Embed Size (px)
Citation preview
Digital Media
Lecture 10: Video & Compression
Georgia Gwinnett CollegeSchool of Science and Technology
Dr. Jim Rowan
Coping with Video Size
Consider human vision limitations1) Chrominance sub-sampling
Compression - two versions2) Spatial3) Temporal
differencingvectoring
Coping with Video Size
Consider human vision limitations1) Chrominance sub-sampling
Compression - two versions3) spatial4) Temporal
differencingvectoring
Chrominance sub-sampling
Humans can’t distinguish changes in color as well as they can distinguish luminance changes– http://en.wikipedia.org/wiki/Chroma_subsam
pling In our cameras…
– Of every 4 frames– store the luminance for each frame– only store a proportion of the color info– 4:2:0
Chrominance sub-samplinghttp://dougkerr.net/pumpkin/articles/Subsampling.pdf
http://en.wikipedia.org/wiki/Chroma_subsampling#Sampling_systems_and_ratios
Chrominance sub-samplingLuminance, Cr, Cb
Coping with Video Size
Aside from screen size and frame rate...
Consider human vision limitations1) Chrominance sub-sampling
Compression - two versions2) spatial3) Temporal
differencingvectoring
Coping with Video Size
Spatial compression • Individual images can be compressed
using the techniques discussed in the bitmapped section
• Doesn’t result in very much compression for video
• Doesn’t take into consideration the other frames that come before or after it
• Our video cameras do this, compressing each frame to jpeg
Coping with Video Size
Aside from screen size and frame rate...
Consider human vision limitations1) Chrominance sub-sampling
Compression - two versions2) spatial3) Temporal
differencingvectoring
Temporal Compression differencing
Use the Difference in two frames– A naive approach can result in good
compression– Works well for a small amount of movement
• Security cameras spend most of their time “seeing” the same thing all night long
– A Tarantino film? not so much…• Most pixels change with nearly every frame
Image Differencing
To subtract one image from the next Do it one pixel at a time
– red minus red– green minus green– blue minus blue
Store the difference To play it back
– Play frame one 1– Add frame 2 to frame 1
Next, an example in black & white
Temporal Compression differencing
Example 1, the difference of two identical images
Example 1, the difference of two identical images
The result ===>
Example 2, the difference of two similar images
Example 2, the difference of two similar images
The result ===>
Temporal Compressionvectoring
• When an OBJECT moves – compute its trajectory– fill in the resulting exposed background
– BUT there’s a problem...– why isn’t this an easy thing to do?
CapturedFrame 1
CapturedFrame 2
StoredObject
MovementVector
StoredBackground
More on differencing…
The differencing can happen in a forward manner and a backward manner
It might be more economical (in data size) to create a frame from a frame that follows it...
MPEG-2iFrame (interframe prediction)pFrame (forward predicted)bFrame (backward predicted)GOP (group of pictures)
http://en.wikipedia.org/wiki/Inter_frame
http://en.wikipedia.org/wiki/I-frames#Intra_coded_frames_.28or_slices_or_I-frames_or_Key_frames.29
MPEG terminology
iFrame: -a keyframe-spatially compressed(a fully specified image)
pFrame-predicted frame-contains only the differencebetween the current frame andthe previous iFrame(smaller in size than iFrame)
bFrame-bi-predicted frame-contains difference betweencurrent and both the precedingand following iFrames-even smaller in size than iFrame
MPEG terminology
MPEG terminology
All spatially (intra-frame) compressed
Spatially compressedand predictive (difference)
Spatially compressed forward and backward predictive(difference)
The largest
Smaller…but morecomputation
Smallest…but more computationand it is transported out of order!
Play sequence: 1 2 3 4 5 6 7 8 9 10
Transmit sequence: 1 4 2 3 7 5 6 10 8 9
Group(s) Of Pictures (GOPs)
So…
How does this ===> happen?
Video CompressionWhat does it?
Coder/Decoder - Codec– encodes and decodes video
Can be symmetric• it takes as long to compress as decompress
Can be asymmetric • it takes longer to compress or decompress than it
does to decompress to compress
Video Compression:Which codec to choose?
It’s a tradeoff… compression technique
• sorenson• DV• Cinepak• Intel Indeo• MPEG4
the compression result• asymmetric or symmetric
– satellite provider’s choice?• larger or smaller final size• computational complexity• the artifacts generated
http://www.100fps.com/codec_quality_comparison.htm
Which is the original?Notice the artifacts?
http://www.100fps.com/codec_quality_comparison.htm
Center one is the originalLeft is “sharpened” Right is “blurred”
Retrograde motion
http://wiki.ggc.edu/images/8/85/JrowanSpring2012CroppedRetrogradeMotionClip.mov
http://wiki.ggc.edu/images/3/35/RetrogradeGMCarcadiaBroadbandHigh.mov
Two ways to make Moving Pictures:Video & Animation
In this class:– Video
• shot with a camera • captures images from the world• then play them back
– Animation • create frames individually• using inkscape and blender• play them back
Reading in the supplemental text:
Moving images: Video
