Upload
taylor-owens
View
48
Download
9
Tags:
Embed Size (px)
DESCRIPTION
CSC 8610 & 5930 Multimedia Technology. Lecture 6 Digital Video Representation. Today in class (3/19). 6:15Recap, Reminders 6:20Lecture – Digital Audio Repr. (cont’d) 6:40Lecture – Digital Video Representation 7:30Break 7:45Digital Video Repr. (cont’d) - PowerPoint PPT Presentation
Citation preview
CSC 8610 & 5930Multimedia Technology
Lecture 6
Digital Video Representation
2
Today in class (3/19)
6:15 Recap, Reminders 6:20 Lecture – Digital Audio Repr. (cont’d) 6:40 Lecture – Digital Video Representation 7:30 Break 7:45 Digital Video Repr. (cont’d) 8:00 Project 1 status, discussion 8:15 Audacity discussion & demos? 9:00 Wrap
3
BASIC TERMINOLOGY
4
Lower Field vs Upper Field
5
Interlaced vs. Progressive Scan Interlaced
– Developed for CRT (Cathode Ray Tube) technology– Divides scans into odd and even lines– Alternately refreshes odd lines, then even lines– Slight delay between refreshes causes “jaggedness” or INTERLACE
ARTIFACTS– DEINTERLACING can compensate somewhat
Progressive– Computer monitors– Scans entire picture line by line– Eliminate flicker seen in interlaced
6
Deinterlace To remove the interlace artifact
Common method:– discard one field– fill in the gaps by duplicating or interpolating the other field
7
Deinterlaced Result
8
Sampling and Quantization of Motion Temporal:
– sampling rate: how frequent you take a snapshot of the motion frame rate higher sampling rate: higher frame rate higher frame rate more frames for the same duration larger file size
9
Sampling and Quantization of Motion Each snapshot: a frame
– an image– digitized based on the same concepts of sampling and quantization of
images
10
Counting Time in Digital Video Timecode: to number frames SMPTE (Society of Motion Pictures and Television
Engineers) video timecode– number frames in hours, minutes, seconds, and frames– drop-frame timecode– non-drop-frame timecode
11
Frame Size Resolution of the frame image
Measured in pixel dimensions
No ppi setting:Unlike digital images, there is no pixel per inch (ppi) setting for video because video is not intended for print but for on screen display.
12
Frame Size Examples
Frame size
NTSC standard definition
720 x 480 pixels
high definition HDV format
1280 x 720 pixels1440 x 1080 pixels
PAL standard definition
720 x 576 pixels
13
Frame Aspect Ratio the ratio of a frame's viewing width to height
NOT equivalent to ratio of the frame’s pixel width to height.
14
Frame Aspect Ratio Examples
4:3 16:9Example:•Standard definition NTSC standard format
Examples:•Standard definition NTSC wide-screen format
•High definition digital video
•High definition TV
15
VIDEO IMAGE WILL BE DISTORTED IF IT IS DISPLAYED ON A SYSTEM WITH A DIFFERENT PIXEL ASPECT RATIO.
16
Distortion
Pixel Apect Ratios Distortion
video frame's = display system's none
video frame's < display system's stretched horizontally
video frame's > display system's stretched vertically
17
VIDEO STANDARDS AND COLOR MODELS
18
Digital Video Standards Standard definition High definition Digital Television
19
The Term DV Common simple abbreviation for digital video
DV compression and DV format: specific types of digital video compression and format respectively
In this context– DV refers to the specific types of digital video– DV is NOT used to abbreviate digital video
20
High Definition HDV Format
Picture Format 720/25p, 720/30p, 720/50p, 720/60p
1080/50i1080/60i
Pixel Dimensions 1280 720 1440 1080
Frame Aspect Ration 16:9
Pixel Aspect Ratio 1.0 1.33
Data Rate Video data only: approx. 19 Mbps approx. 25 Mbps
Color Sampling Method YUV 4:2:0
Audio Setting Sampling rate and bit depth:
48 kHz16-bit
Bit rate after compression:
384 kbps
21
HDV Picture Format Notation
1080 / 60 i
frame height
frame/field frequency
"i": interlaced"p": progressive
22
Frame Size (Resolution) Comparison between Standard Definition and High Definition
By viewing frame size
23
Frame Size (Resolution) Comparison between Standard Definition and High Definition
By pixel dimensions
24
Frame Size (Resolution) Comparison between Standard Definition and High Definition
A frame from a 1080i video
25
Frame Size (Resolution) Comparison between Standard Definition and High Definition
Same frame as 720p
26
Frame Size (Resolution) Comparison between Standard Definition and High Definition
Same frame as standard definition DV wide-screen (16:9)
27
Frame Size (Resolution) Comparison between Standard Definition and High Definition
Same frame as standard definition DV standard 4:3
28
Digital Television (DTV) Signals of DTV are broadcast and transmitted digitally Need a digital TV set to watch
29
Digital Television (DTV) Standard definition
– 704 480, 16:9 and 4:3, progressive and interlaced– 640 480, 4:3, progressive and interlaced
High definition– 1920 1080, 16:9, progressive and interlaced– 1280 720, 16:9, progressive and interlaced– MPEG-2
30
VIDEO FILE TYPES
31
Common Video File Types
File Type Acronym For Originally Created By
File Info & Compression
Platforms
.mov QuickTime movie Apple • Also audio-only• Can be streamed• "Fast start"
• Common compression methods: H.264, Sorenson Video, Animation
Apple QuickTime player, which is available for Mac and Windows
.avi Audio Video Interleave
Intel Common compression methods: Microsoft RLE, Intel Indeo Video
Primarily used on Windows but Apple QuickTime player can play AVI files
.mpg
.mpegMPEG Motion Picture
Experts Group• For DVD-video• High definition HDV
Cross-platform
32
Common Video File Types
File Type Acronym For Originally Created By
File Info & Compression
Platforms
.flv Flash Video Adobe • Progressive download• Can be streamed• Common compression methods: H.264, Sorenson Spark, On2 VP6
• Cross-platform• Requires Adobe Media Player to play
.rm Real Video Real System • Can have very high degree of compression• Choose the compression level based on network connection speed• Can be streamed
• Cross-platform• Requires Realplayer to play
.wmv Windows Media Microsfot Requires Windows Media Player to play
33
Considerations for File Type File size restriction Intended audience Future editing
34
File Size Restriction For Web:
– high compression– streaming video
CD-ROM or DVD-ROM playback:– use data rate that can be handled by your target audience's computer
DVD-video:– MPEG-2
35
Intended audience Multiple platforms
– cross-platform formats: Apple QuickTime, MPEG, Flash video, Real Video
How your target audience is going to watch your video?
36
Future Editing If the video will be used as a source for future editing:
– Lower compression level– Choose uncompressed, if
the frame size is small the video duration is extremely short you have enough disk space
37
Digital Video File Size Optimization Video tends to have very large file size compared to
other media.
Why should we care file size optimization?– A large file requires more disk space.– A large file takes longer to transfer.– Data transfer can be expensive (because data plans are not unlimited)– High data rate may cause choppy playback of the video.
(Data rate will be explained later in this lecture.)
38
Effect of File Size vs. Data Rate on Video Playback
Data rate:– If high: choppy playback– Amount of data to be processed per second
Larger file size can have a low data rate if it is a long video Smaller file size can have a high data rate if it is a short video
File size:– If high:
Requires larger storage space Not unnecessary choppy playback
– The impact of file size on smoothness of playback also depends on the video duration.
39
General Strategies for Reducing Video Data Rate
Basic ideas:
A video is a sequence of images + audio
Apply strategies for reducing digital image and audio file size.
40
General Strategies for Reducing Video File Size
General Strategies for reducing digital image file size– reduce frame size– reduce frame rate– choose a video compressor that allows higher compression– choose the lower picture quality option
Reduce duration of the video so you have less frames– not always possible– will not impact data rate
41
Why General Strategies for Reducing Digital Image File Size Work for Video
Reduce frame size because:– you have less pixels for each frame
Reduce frame rate because:– you have less frames
Video compression with high compression because:– some data are discarded
Lower picture quality option because:– some data are discarded
42
Strategies Least Used for Reducing Video Data Rate
Reduce bit depth– Not all video formats support lower bit depth– Live videos need 24-bit to look natural– Some compressors do not support lower bit depth
Reduce sampling rate, bit depth, and channel numbers of the audio
– size of the audio is insignificant compared to that of the picture component in a video
43
COMPRESSION OF VIDEO
44
Compression Basic idea:
Want to represent the same content by using less data
45
Compression and Decompression Compression:
– To reduce file size– Takes time– Often takes more time for higher compression
Decompression:– A compression video file must be decompressed before it is played.– The decompression method or algorithm depends on how it is
originally compressed.
46
Compression and Decompression Compression and decompression always go together as
a pair.
Codec: compressor/decompressor
47
Types of Compression Methods Spatial compression Temporal compression Lossless vs. lossy compression Symmetrical and asymmetrical compression
48
Spatial Compression Compact individual frames as if they are independent
digital images
Examples of algorithms:– Run-length encoding (RLE)– JPEG compression
Example codecs:– QuickTime Animation– QuickTime PlanarRGB– Microsoft RLE
49
Spatial Compression Types of video that spatial compression is good for:
– contain large areas of solid colors, such as cartoon animation
Disadvantage:– Less compressed, i.e., relatively large file size compared to other types
of compression
50
Temporal Compression Exploits the repetitious nature of image content over
time in video
Saving more information for selected frames, i.e. less compressed. These are called key frames.
All other frames stores only the difference from the previous key frame, instead of full frame
Advantage:– Effective if the change between a frame and its previous key frame is
small
51
Temporal Compression Compressed well for:
– video that contains continuous motion
Not compressed well for:– video with frequent flickering and scene changes
Example codecs that use temporal compression:– H.264– Sorenson Video
52
Lossy vs. Lossless Compression Lossy compression:
Reduce data by discarding or altering some of the original data
Lossless compression:Preserve the original data but reduce file size by encoding the data specially
53
Lossy Compression Usually much smaller file size than lossless
compression
Lower picture quality
Often try to maintain perceptual quality when deciding what data are to be discarded
Discarded data cannot be recovered
54
Lossless Compression Usually much larger file size than lossy compression
Example codecs:– QuickTime Animation– PlanarRGB (set at the maximum quality setting)
55
Symmetrical and Asymmetrical Compression
Symmetrical codec:Same amount of time in compression and decompression
Asymmetrical codec:– Amount of time to compress and decompress are significantly different– Preferable: Fast decompression so less wait time to play back the
video
56
MPEG
57
MPEG Moving Pictures Experts Group
Committee who derives standards for encoding video
Allow high compression
MPEG-1, MPEG-2, MPEG-4
58
What Happend to MPEG-3? NOT MP3 (which is audio format)
Intended for HDTV
HDTV specifications was merged into MPEG-2
59
MPEG-1 Video quality comparable to VHS
Originally intended for Web and CD-ROM playback
Frame sizes up to 352 240 pixels
Video format for VCD (VideoCD) before DVD became widespread
60
MPEG-2 Supports DVD-video, HDTV, HDV standards
For DVD video production:Export video into DVD MPEG-2 format
For HDV video production:Export video into HDV's MPEG-2 format
61
MPEG-4 Newer standard of MPEG family
Different encoding approach from MPEG-1 and MPEG-2(will discuss after MPEG-1 and MPEG-2 compression in this lecture)
62
Applications of MPEG-4 Cover a wide range of data rate
Low end of the data rate: Video playback on mobile devices
High end of the data rate:– HDTV– Handheld and portable game devices (e.g., Sony PSP)
63
WAYS OF PLAYING VIDEO
64
Two Ways of Playing Video Play from disk Play over a network
65
Play from Disk An entire clip needs to be on disk before it can be
played
Played from hard drive, CD, or DVD
66
Play over a Network The video can be played while it is being downloaded
Can be played from disk
Streaming video
Progressive download
67
Streaming Video Play video as soon as enough data has arrived
Examples:– Streaming QuickTime– Real Video– Window Media Video (WMV)
68
Streaming Video Require a streaming server to stream video
Allow saving several different compression levels of a video in a single file
The server chooses the compression level to match the speed of network connection
Buffering: Wait time depends on network speed
69
Progressive Download Play video as soon as enough data has arrived
Does not require special servers
Example: – QuickTime fast-start
Created by saving the QuickTime movie as self-contained using QuickTime Pro