4k, HDR, and HEVC - Media Choice Group of Companies€¦ · 4k, HDR, and HEVC Oh my! 13 Sept ... •Builds on concepts from H ... c o m p a tib ility with th e c o n v e n tio n a

1 – Confidential 9/21/2015

4k, HDR, and HEVC Oh my!

13 Sept 2015

Russel Van Der Werff

Director of Product Management


HEVC OVERVIEW Why do we need a new CODEC anyway?


History of Video Compression

• MPEG-2 Video Compression – First widely used video compression algorithm

– SD video at 4 Mbps, HD at 13 Mbps

– Achieves compression rates of 30-50x!

• H.264/AVC/MPEG-4 Compression – Approximately 2x compression of MPEG-2 video

– SD Video at 2.5 Mbps, HD at 7 Mbps

• H.265/HEVC/MPEG-H Compression (2013) – Approximately 2x compression of AVC video?

– SD video at 1 Mbps, HD at 4 Mbps?

MPEG-1 (1993)

MPEG-2 (1994)

AVC (2003)

HEVC (2013)

Today (2014)

?


Video Compression Basics

• Relies on Three Techniques

– Spatial Compression (Intra-frame Prediction)

– Temporal Compression (Inter-frame Prediction)

– Data Compression (e.g. Entropy Coding)

• Lossless data rate reduction – think “.ZIP files”


HEVC Benefits

• Builds on concepts from H.264

• More flexible macroblock (coding tree block) structure

• More granular motion prediction


HEVC Benefits

• Sample picture with HEVC CTB partitions


HEVC Benefits

• Developments may seem obvious, but…

– Require more processing per frame

– Frame-rates remain constant

– More computational power:

• Must be achievable in inexpensive solutions (STB)

40-100(!?)x H.264

Complexity (Encoding)

3x H.264

Complexity (Decoding)


HEVC Terminology and Features

• Profiles

– 25 Profiles!

– Likely Important: • Main

• Main 10

• Main 4:2:2 10

– Other/Of Interest: • Main Still Picture

• 12-bit and 4:4:4

• Scalable & Multiview

– More to come…?

• Levels • 3.1: SD (480/576)

• 4: HD (720/1080)

• 4.1: 1080p60

• 5.1: 4kp60

• Tiers

– Main • “Consumer” applications

• 1080p60 @ 20 Mbps

– High • “Pro” applications

• 1080p60 @ 50 Mbps


HEVC Terminology and Features

• Interlaced Support – SEI message used to indicate interlaced/progressive

– No macroblock or picture adaptive frame/field (MBAFF/PAFF)

– Sequence adaptive transitions (SAFF) supported

Value Indicated Display of Picture

0 (progressive) frame

1 top field

2 bottom field

3 top field, bottom field, in that order

4 bottom field, top field, in that order

5 top field, bottom field, top field repeated, in that order

6 bottom field, top field, bottom field repeated, in that order

7 frame doubling

8 frame tripling

9 top field paired with previous bottom field in output order

10 bottom field paired with previous top field in output order

11 top field paired with next bottom field in output order

12 bottom field paired with next top field in output order


HEVC Compression Efficiency

• Efficiencies with interlaced sequences

– Some debate about necessity of PAFF/MBAFF

– Early research asserted gains of these tools were too negligible to include

– Some encoder manufacturers feel differently…

• General target is 50% gain over H.264

– File-based systems have achieved 40-50% today

– Real-time systems more like 30-40%

– Most encode manufacturers still evolving toward 50%

– Expected to reach target in 2016


METHODS FOR PERCEPTUAL PICTURE QUALITY IMPROVEMENT

What really makes your HD more “Ultra?”


Improvements in Picture Quality

• Numerous new technologies for “better” picture:

– Resolution Improvements • 1080p50/60 -> UHD-1/4k/2160p -> UHD-2/8k/4320p

– High Frame Rates (HFR)

– High Dynamic Range (HDR)

– Improved Color Gamut

– Higher Bit-depth

– Better Compression Systems

• It doesn’t end with 4k (or HDR)!


Perceptual Improvement

• High-res (4k) UHD really shines with screen size

– Human eye’s perception of detail falls off with distance • 50” UHD TV -

sit 1m away!?

• 80” UHD TV - sit 1.5m away!?

SOURCE: RTINGS.COM

SIDE NOTE: 4K 6” SMARTPHONE SCREEN?!


Perceptual Improvement • Some of the other improvements:

– High Frame Rates (HFR)

– High Dynamic Range (HDR)

– Improved Color Gamut

are applicable for small screens, etc.

• Open debate about which improvements are “better”…


INTERFACES FOR 4K …and how do we carry this stuff around?


Interfaces for 4k

• Current Interfaces for UHD/4k

– Quad SDI

• 4x 3G-SDI = 12Gig/sec

• 4x BNC cumbersome/impractical

• Synchronization challenges, etc.

– HDMI Interfaces

• HDMI 1.4 is capable of 2160p24

• HDMI 2.0 goes to 2160p60

• HDMI 2.0a adds HDR signaling


Interfaces for 4k

• SMPTE to the rescue!


Interfaces for 4k

• SMPTE to the rescue!


Interfaces for 4k

• Optical 12-gig SFP

– SMPTE 2082 12 Gb/s Signal/Data Serial Interface - Electrical

– SFP based – Fiber and Multi-Coax

– Starting to show up in some products now


Interfaces for 4k

• Isn’t everything going IP?

– SMPTE 2022-6 – SDI over IP • 1080i60 = 1.5 Gbps

• 1080p60 = 3 Gbps

– 10 Gbps Ethernet getting cheap(er)

• But for 4k

– 2160p60 = 12 Gbps!

– Dual 10 Gbps?

– 40 Gbps – available, but still expensive


Interfaces for 4k

• But everything must be going IP!?

• Back to Compression Schemes – H.264 – Contribution HD in 30 Mbps

– JPEG 2000 – Contribution HD in 200 Mbps

– IntoPix Tico – Contribution HD in 800 Mbps

• Advantages of Light Compression – Bitrates capable for IP transport

– JPEG 2000 et al. “visually lossless” (or close)

– Low-delay suitable for switches, effects, etc.

• “All IP” studios use these, versus SMPTE 2022-6

• Perhaps 4k IRDs will have JPEG 2000 outputs?


HIGH DYNAMIC RANGE “Are we there yet?”


Intro to HDR • Difficult to reproduce certain real-world scenes

– Displays have limited range of luminance (dark->light)

– Some scenes have large range of light and dark

– Extending range creates better reproduction of image

STANDARD DYNAMIC RANGE (SIMULATED) HIGH DYNAMIC RANGE (SIMULATED)

SOURCE: RUSS’S IPHONE


Intro to HDR

• HDR – picture quality improvement from…

– Improved maximum luminance:

• Reference white at 100 nits

• Today’s LCDs support 200-400 nits

• HDR LCDs will have up to 10000 nits

– Improved contrast:

• Typical distribution has supported 8 bits per pixel

• HEVC distribution supports 10 bits per pixel

• HDR may involve 12 bits per pixel


Intro to HDR

• Digital Picture Sampling

– Maps digital numbers (DNs) to brightness values

– Example:

Map between signal level (digital/electronic) and optical brightness: “Electro-Optical Transfer Function”

Digital Number Brightness (nits)

1 0 nits

2 5 nits

3 10 nits

… …

256 1275 nits

RANGE OF DN DEPENDS ON

BIT DEPTH

NOTE: MAPPING DOES NOT NEED TO BE LINEAR (AS SHOWN HERE)


Intro to HDR

• Old assumptions based on CRT display capabilities

• Physical characteristics of modern LCDs allow wider “range of creative expression” (better picture!)

Camera Transmission Display EOTF OETF

SOURCE: AVSFORUMS


HDR Proposals

• Different/competing encoding curves – Several proposals to extend beyond existing max luma (100IRE)

– Notably Dolby “Vision” and BBC

9

BT.2020, this choice of breakpoint allows up to 640% of reference white to be coded before clipping is required. This further extension is discussed in later sections.

A higher value of Lmax would also be possible to provide a higher dynamic range. For example

Lmax=8 implies a breakpoint of μ=0.083822216783. However as the value of Lmax increases, compatibility with the conventional gamma curve decreases. It would also be possible to have different values of Lmax for different applications, such as digital signage, CAD or computer graphics, which use different types of display. For applications that may use the same display, such as television and user generated content or other entertainment, there are obvious advantages to choosing a consistent value of Lmax. Further increased dynamic range, beyond this proposal, may be achieved by other means, such as the use of tone mapping. Overall this proposal suggests a single value for video applications of Lmax=4 (or, alternatively, Lmax=8 to achieve higher dynamic range but with lower backward compatibility).

A 10 bit signal conforming to the proposed OETF, with Lmax=4, provides a sixteen fold increase in dynamic range compared to an 8 bit Rec 709 signal (without taking account of the headroom above V=1). The maximum luminance is increased by a factor of 4 as above. Increasing the signal depth from 8 to 10 bits reduces the difference between successive quantisation levels by a further factor of 4, thereby increasing the dynamic range at low luminance (“in the blacks”).

Figure 3 shows a comparative plot of several OETFs. The Rec 709 curve (partially hidden by the “knee” curve), clips at reference white. Clipping is avoided up to 400% of reference white by using the camera knee characteristic illustrated. The proposed OETF offers the same extension in dynamic range but with a smoother curve. At the bottom end it is (by design) very similar to the Rec 709 curve. At the upper end it is similar to the knee curve. Based on industry experience of cameras with knee curves we may be confident that the proposed curve has a high degree of compatibility with Rec 709. The “perceptual quantiser” OETF proposed by Miller et al (PQ 10000) is included for comparison. Over the range plotted it is extremely similar to a pure logarithmic transfer function with a (10 bit) Weber fraction of about 1% (see below). As a logarithmic transfer function it is clearly highly incompatible with the Rec 709 curve. However the perceptual quantiser is designed to be used over a much wider dynamic range so incompatibility is unavoidable. The comparison is only intended to show that a perceptual quantiser or, equivalently, a pure logarithmic transfer characteristic is not compatible with Rec 709.

Opto Electronic Transfer Functions

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00

luminance (relative to reference white)

sig

na

l

Rec 709

Knee (400%)

Proposed (400%)

Perceptual Quantiser

Figure 3: Opto-electrical Transfer Function

The effect on the image of various OETFs is illustrated below. A high dynamic range image was created from multiple bracketed exposures on a digital stills camera. A reference white level was chosen to provide the best image. For figure 4 the image was created using the standard Rec 709 OETF. Signals brighter than reference white were clipped. Notice that a large part of the sky is “burnt out” obscuring detail in the clouds on the right of the sky. Note also that there is an artificial “rainbow” around the burnt out region of sky. This is due to changes in hue caused by clipping the

D

Dolby BBC “Knee”

Gamma

SOURCE: BBC


© 2012 SMPTE · T e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org

12 bit Rec1886 Curves HDR Proposals

• Why change?

10-bit Gamma Stretched to higher brightness

Banding Returns

SOURCE: SMPTE


HDR Proposals

• Why change?

“Barten” Ramp (Humans notice steps/banding)

Typical 8-bit Gamma (Some banding possible)

10-bit Gamma No Banding to 100 nits

BBC “Knee”

Dolby Vision

SOURCE: ERICSSON


HDR Support through the Signal Chain

• Optics and Sensors • Opto-Electrical Transfer Function (OETF)

• Color Space • CODEC • Signaling

• Signaling Transfer

• Mastering • Color Balance

Camera

Contribution Encode

Receive / Decode

Production Edit

Distribution Encode

STB & TV

• CODEC Support • Signaling

• TV Support • Signaling • Electro-Optical

Transfer Function (EOTF)


HDR Support through the Signal Chain

• Standardization is a “work in progress”

SOURCE: SMPTE


HDR Status and Open Quesitons

• Other Aspects – BBC proposal easier to make “backwards compatible”

– Dolby proposal has more market momentum

• How to carry? – Two streams (SDR and HDR)?

– Single stream with supplementary data

– One stream with “supplement?”

• Carry “remapping” with metadata?

• No standardization on display side… – Samsung “Peak Illuminator Ultimate,” LG “Ultra Luminance,”

Panasonic “Dynamic Range Remaster,” Sony “XDR” and “XDR Pro,” Dolby Vision, Sharp, TCL, etc. etc. etc.


MORE QUESTIONS? Thanks for your time and attention!

Documents

4k, HDR, and HEVC - Media Choice Group of Companies€¦ · 4k, HDR, and HEVC Oh my! 13 Sept ... •Builds on concepts from H ... c o m p a tib ility with th e c o n v e n tio n a