Video Coding --- State of the Art and Beyond
Dr. Mona MathurAdvanced Systems Technology,
STMicroelectronics, GNoida, India
August 8, 2011
2
Agenda
Overview of concepts in Video Coding
State of the art Compression standards
Evolving Video Coding standards
The Road ahead ---- Challenges
Digital Video characteristics
Frame : is a 2D rectilinear array of pixels
Frame rate number of frames displayed per second
Frame dimensions width and height of the frame in pixels
Pixel depth Number of bits per pixel
Each pixel has two values associated with it Luminance – represents the intensity (Y) Chrominance – represents the colour of the pixel (Cr, Cb)
OBESE
Why compression ???? Images require considerably more storage space than text Audio and Video demand even more!!
Text (VGA screen @ 2 bytes/char) ~ 640 * 480 * 2 / (8 * 8) ~ 9.4 KB
Uncompressed stereo audio ~ 2 * 44100 * 2(44.1 KHz, 16 bits per sample) ~ 172 KB/s
PAL (720 pixels, 576 lines, 25 frames) ~ 249 Mbits/s !!A 2 hour movie ~ 1669GB more than 2000 CDs
Very high secondary storage and transmission requirements.
History of Video Standards
6
Use and Spread of Standards
Critical for widespread adoption of codec technology
The three players Customers
Economies of scale --- so products are affordable
Industry Interoperability between vendors
Content providers Long life and broad demand of the content Ensures widespread adoption
Proprietary standards AVS (China), WMV9 (Microsoft), On2, VP8, Nancy, RV10 (Real
Networks Real Video)
7
Compression rates for diff technologiesIntra-frameMotion Compensation
Integer-pelHalf-pelVariable Block sizeQuarter-pel Multi-frame
8
Evolution of MPEG4
Compression ??? : How
Statistical Redundancy in the signal Spatial: Correlation between adjacent pixels
Temporal: Correlation between pixels in adjacent pictures
Irrelevancy Reduction Omits parts of the signal not perceived by the Human Visual
System (HVS)
No Change in pixels
10
Coding Tools in state of the art codecs
Spatial redundancy Intra prediction
Temporal redundancy Inter prediction
Transform -- DCT Entropy coding
VLC, CABAC
Deblocking Filter
11
Frame – from Macroblock view
12
Spatial Coding
13
Temporal Coding
H.264 Encoder
15
Compression Today??
Is Need for higher Compression Relevant Today?
YES !!!
Why ????
16
Evolving Products
Smart phones Tablets Ipads Net enabled STBs ….
17
Emerging Video Content &Type Video as a medium is changing.
Spatial resolution QCIF -- 176x144 --- pixels HD --- 1920x1080 ---- 2,073,600 pixels UHD --- 4096x3,072 ---- 12,582,912 pixels
Temporal resolution (25- 50 -200 fps) Increasing dynamic range of each component
From 8 bit per component to 16 bit per component New visual modalities
stereoscopic, Multi-view, 3D, holography, the myriad of consumer
devices playing stored
18
Demands of Consumers…
Imagine…
…being able to see high quality, high definition video, on demand. …renting a DVD with choices of standard or high definition
video, with multiple camera views
Applications like Facebook, twitter, Google+
Mobile Video Services like : football highlights, personal video conferencing, or downloads of
movie trailers such services will increase What about spectrum availability
19
Video Coding today ….
New extensions within existing framework SVC, MVC, Free View point TV
Next evolution of the video coding standards KTA, H.265
New Technologies Parametric Video Coding Texture Analysis and Synthesis
20
Video Coding today ….
New extensions within existing framework SVC, MVC, Free View point TV
Next evolution of the video coding standards KTA, H.265
New Technologies Parametric Video Coding Texture Analysis and Synthesis
21
HEVC ---High Efficiency Video Coding
Proposed Video compression standard Successor to H.264/MPEG-4 AVC
A joint team by ISO- MPEG and ITU-T --VCEG has been established JCT-VC --- Joint Collaborative Team on Video Coding in 2010
Nick named --- H.265
Goals: Increase compression by 50% over H.264 at comparable image/video quality At the expense of increased computational complexity
Targeted platforms: Next generation HDTV displays and content capture systems Display resolutions from QVGA (320x240) upto 1080p and Ultra HDTV (7680x4320)
http://en.wikipedia.org/wiki/High_Efficiency_Video_Coding
22
Source --- JCTVC-D181
Testing Results --- BDrate with complexity
23
Benefits of arbitrary unit representation Codec optimized for content,
applications and devices. Examples are:
low resolution video services Videos with large homogenous
regions
A simple & elegant representation LCTB size, max hierarchical depth, & a series of split flags.
Size-independent syntax representation Syntactical representation of
remaining coding tools is simplified Simplifies the standard and the
parsing process
64
64
32
32
16
16
8
8
Resolution: 1920x1080LCU size : 64Maximum depth =2
LCU size : 64Maximum depth = 4
Resolution: 1920x1080
Resolution: 352x288LCU size : 16Maximum depth =2
32
32
16
16
8
8
4
4
Recommended