Video Coding --- State of the Art and Beyond Dr. Mona Mathur Advanced Systems Technology,...

Video Coding --- State of the Art and Beyond

Dr. Mona MathurAdvanced Systems Technology,

STMicroelectronics, GNoida, India

mona.mathur@st.com

August 8, 2011

2

Agenda

Overview of concepts in Video Coding

State of the art Compression standards

Evolving Video Coding standards

The Road ahead ---- Challenges

Digital Video characteristics

Frame : is a 2D rectilinear array of pixels

Frame rate number of frames displayed per second

Frame dimensions width and height of the frame in pixels

Pixel depth Number of bits per pixel

Each pixel has two values associated with it Luminance – represents the intensity (Y) Chrominance – represents the colour of the pixel (Cr, Cb)

OBESE

Why compression ???? Images require considerably more storage space than text Audio and Video demand even more!!

Text (VGA screen @ 2 bytes/char) ~ 640 * 480 * 2 / (8 * 8) ~ 9.4 KB

Uncompressed stereo audio ~ 2 * 44100 * 2(44.1 KHz, 16 bits per sample) ~ 172 KB/s

PAL (720 pixels, 576 lines, 25 frames) ~ 249 Mbits/s !!A 2 hour movie ~ 1669GB more than 2000 CDs

Very high secondary storage and transmission requirements.

History of Video Standards

6

Use and Spread of Standards

Critical for widespread adoption of codec technology

The three players Customers

Economies of scale --- so products are affordable

Industry Interoperability between vendors

Content providers Long life and broad demand of the content Ensures widespread adoption

Proprietary standards AVS (China), WMV9 (Microsoft), On2, VP8, Nancy, RV10 (Real

Networks Real Video)

7

Compression rates for diff technologiesIntra-frameMotion Compensation

Integer-pelHalf-pelVariable Block sizeQuarter-pel Multi-frame

8

Evolution of MPEG4

Compression ??? : How

Statistical Redundancy in the signal Spatial: Correlation between adjacent pixels

Temporal: Correlation between pixels in adjacent pictures

Irrelevancy Reduction Omits parts of the signal not perceived by the Human Visual

System (HVS)

No Change in pixels

10

Coding Tools in state of the art codecs

Spatial redundancy Intra prediction

Temporal redundancy Inter prediction

Transform -- DCT Entropy coding

VLC, CABAC

Deblocking Filter

11

Frame – from Macroblock view

12

Spatial Coding

13

Temporal Coding

H.264 Encoder

15

Compression Today??

Is Need for higher Compression Relevant Today?

YES !!!

Why ????

16

Evolving Products

Smart phones Tablets Ipads Net enabled STBs ….

17

Emerging Video Content &Type Video as a medium is changing.

Spatial resolution QCIF -- 176x144 --- pixels HD --- 1920x1080 ---- 2,073,600 pixels UHD --- 4096x3,072 ---- 12,582,912 pixels

Temporal resolution (25- 50 -200 fps) Increasing dynamic range of each component

From 8 bit per component to 16 bit per component New visual modalities

stereoscopic, Multi-view, 3D, holography, the myriad of consumer

devices playing stored

18

Demands of Consumers…

Imagine…

…being able to see high quality, high definition video, on demand. …renting a DVD with choices of standard or high definition

video, with multiple camera views

Applications like Facebook, twitter, Google+

Mobile Video Services like : football highlights, personal video conferencing, or downloads of

movie trailers such services will increase What about spectrum availability

19

Video Coding today ….

New extensions within existing framework SVC, MVC, Free View point TV

Next evolution of the video coding standards KTA, H.265

New Technologies Parametric Video Coding Texture Analysis and Synthesis

20

Video Coding today ….

New extensions within existing framework SVC, MVC, Free View point TV

Next evolution of the video coding standards KTA, H.265

New Technologies Parametric Video Coding Texture Analysis and Synthesis

21

HEVC ---High Efficiency Video Coding

Proposed Video compression standard Successor to H.264/MPEG-4 AVC

A joint team by ISO- MPEG and ITU-T --VCEG has been established JCT-VC --- Joint Collaborative Team on Video Coding in 2010

Nick named --- H.265

Goals: Increase compression by 50% over H.264 at comparable image/video quality At the expense of increased computational complexity

Targeted platforms: Next generation HDTV displays and content capture systems Display resolutions from QVGA (320x240) upto 1080p and Ultra HDTV (7680x4320)

http://en.wikipedia.org/wiki/High_Efficiency_Video_Coding

22

Source --- JCTVC-D181

Testing Results --- BDrate with complexity

23

Benefits of arbitrary unit representation Codec optimized for content,

applications and devices. Examples are:

low resolution video services Videos with large homogenous

regions

A simple & elegant representation LCTB size, max hierarchical depth, & a series of split flags.

Size-independent syntax representation Syntactical representation of

remaining coding tools is simplified Simplifies the standard and the

parsing process

64

64

32

32

16

16

8

8

Resolution: 1920x1080LCU size : 64Maximum depth =2

LCU size : 64Maximum depth = 4

Resolution: 1920x1080

Resolution: 352x288LCU size : 16Maximum depth =2

32

32

16

16

8

8

4

4