View
22
Download
0
Category
Tags:
Preview:
DESCRIPTION
Scalable_Video_Coding_Dept_Talk_Ankit_Bhurane
Citation preview
Scalable Video Coding: An Overview
Ankit A. Bhurane
TIDSP, EE, IITB
Why Video?
• Annual global IP traffic will surpass 1.3 zettabytes/ month threshold by the end of 2016!
• 1, 000, 000, 000, 000, 000, 000, 000 bytes!
• Who is responsible?
…
• Complete list at http://en.wikipedia.org/wiki/List_of_video_hosting_services
Forecast by CISCO: http://www.cisco.com/
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 2
Raw Images
• Store data from an image sensor
• Typical image sensors capture only red, green, or blue for a pixel.
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 3
Courtesy: Adobe Systems
Video?
… …
Source: http://ivpl.eecs.northwestern.edu/files/general/Public_Files587/MC_Frame_Rate_Upconversion/frame_motion.png
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 4
n-1 n n+1
How Hard can it be?
• Size: HD: 1280×720;
• Color
• Frame rate: 30 fps
• 1280×720×3×30×8 bits/sec
• 663.552000 Mbits/sec!
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 5
Raw Video Specs
• Specification:
– Spatial Resolution (aspect ratio)– 4:3, 16:9, …
– Color space: RGB, YUV, …
– Framerate: 30 fps, 50 fps, 60 fps
– ?
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 6
Resoltion Chart: Source: Wikipedia
http://upload.wikimedia.org/wikipedia/en/9/9b/Vector_Video_Standards2.png
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 7
Frame rate
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 8
Color Space
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 9
RGB to YUV
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 10
RGB YUV
𝑌𝑈𝑉
=0.299 0.587 0.114−0.147 −0.289 0.4360.615 −0.515 −0.100
𝑅𝐺𝐵
How to achieve compression?
• Take advantage of human visual system (HVS)!
• Remove redundancy.
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 11
Human Eye Sensitivity
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 12
Shamelessly Taken from: Slides by Yao Wang, Polytechnic University, Brooklyn :P
Courtesy: Yao Wang
Croma Sampling
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 13
Shamelessly Taken from: Slides by Yao Wang, Polytechnic University, Brooklyn :P
Courtesy: Yao Wang
Most video is 4:2:0
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 14
Taken from: http://people.xiph.org/~tterribe/pubs/lca2008/anatomy.pdf
Spot the difference
Department Talk | Ankit Bhurane | TIDSP-IITB April 3, 2013 15
4:4:4 4:2:2
Sequences available at: http://samples.mplayerhq.hu/yuv/
4:2:2 requires 2/3rd the bandwidth of 4:4:4!
How to achieve compression?
• Take advantage of human visual system!
• Remove redundancy.
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 16
Spatial Redudndancy: Correlated data
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 17
Transform
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 18
Taken from: http://people.xiph.org/~tterribe/pubs/lca2008/anatomy.pdf http://www.vcodex.com/files/videocoding_intro_1.pdf
Spatial-Temporal redundancy
n n+1
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 19
MEMC
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 20
Taken from: John G. Apostolopoulos, Image and Video Compression, MIT 6.344, Spring 2002
Motion
Motion Estimation and
Compensation
Department Talk | Ankit Bhurane | TIDSP-IITB April 3, 2013 21
Taken from: Thyagarajan, Still Image and Video Compression.
Department Talk | Ankit Bhurane | TIDSP-IITB April 3, 2013 22
MEMC-Example
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 23
How to achieve compression?
• Take advantage of human visual system!
• Remove redundancy.
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 24
Campbell-Robson
Contrast Sensitivity Chart
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 25
Sinusoidally modulated Luminance
Co
ntr
ast
fro
m 1
00
% t
o a
bo
ut
0.5
%
http://neurovision.berkeley.edu/Demonstrations/VSOC/izumi/CSF/A_What_is_CSF.html
Quantization
• The only lossy step!
• Broadly categorized:
– Scalar
– Vector
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 26
Core of Data Compression
Transform
• Removes spatial redundancy
Quantize
• Removes insignificant data
Entropy coding
• Removes statistical redundancy
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 27
Core of Video Compression
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 28
MEMC
• Removes temporal redundancy
Transform
• Removes spatial redundancy
Quantize
• Removes irrelevant data
Entropy coding
• Removes statistical redundancy
Transform
• Goal: A sparse representation of the pixel data.
– By removing redundancy of highly correlated data.
• Optimal transform: Karhunen-Loève Transform (KLT).
– Data-dependent
– Need to transmit eigen vectors
• More appropriate: DCT
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 29
Transform
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 30
Taken from: http://people.xiph.org/~tterribe/pubs/lca2008/anatomy.pdf http://www.vcodex.com/files/videocoding_intro_1.pdf
Entropy Coding
• Inputs:
– transform coefficients
– motion vectors and
– ‘side’ information (headers, synchronization markers, etc.)
• Techniques:
– Huffman coding
– Arithmetic coding
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 31
Entropy Coding
• Huffman coding
– Optimal– but only as symbol codes
– Needs integer-valued codeword length
• Arithmetic coding
– Codes group/ block of symbols
– Near-optimal for long data
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 32
How it appears?
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 34
Taken from: http://www.w3.org/2008/01/video-container.png http://cdn.reelstatic.com/wp-content/uploads/encoding-video.png
How it comes to us?
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 35
Internet
www.youtube.com 208.65.153.251
Server
Internet Service Provider
How does it look in Market?
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 36
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 37
Todays Need
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 38
Encode: Once at highest resolution, Decode: Different rates depending on application
Single bit stream adaptable to heterogeneous end users!
Scalability, What’s that?
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 39
Scalable Bitstream
Courtesy: Nikola Adami et al., “State-of-the art and trends in Scalable video compression with wavelet-based approaches”
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 40
SVC System
Courtesy: Nikola Adami et al., “State-of-the art and trends in Scalable video compression with wavelet-based approaches”
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 41
Scalable Bitstream
Courtesy: Nikola Adami et al., “State-of-the art and trends in Scalable video compression with wavelet-based approaches”
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 42
Spatial Scalability
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 43
Temporal Scalability
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 44
Quality Scalability
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 45
Performance
April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 46
SVC demo
Courtesy: Georgia Tech.
Recommended