Alain Bouffioux - Philips Leuven 1 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Audio & Video Compressionand its Application inConsumer Products
Alain Bouffioux - Philips Leuven 2 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Agenda• Introduction - The evolution of Audio/Video
consumer products and the role of compression techniques.
• Audio & Video compression principles• Audio compression• Video compression• Audio/Video synchronisation• The MPEG model and its situation in a
communication context• Application to DVD (Digital Versatile Disc)• Application to DVB (Digital Video Broadcasting)• Conclusion
Alain Bouffioux - Philips Leuven 3 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Agenda• Introduction - The evolution of Audio/Video
consumer products and the role of compression techniques.
• Audio & Video compression principles• Audio compression• Video compression• Audio/Video synchronisation• The MPEG model and its situation in a
communication context• Application to DVD (Digital Versatile Disc)• Application to DVB (Digital Video Broadcasting)• Conclusion
Alain Bouffioux - Philips Leuven 4 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Moore’s law
Number of transistors per square inch doubles every 18 months
Alain Bouffioux - Philips Leuven 5 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Moore’s law today
Cost of a transistor divided by one million in 30 years
Alain Bouffioux - Philips Leuven 6 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Moore’s law today (2)
• “Self-fulfulling prophecy” (not automatic)= roadmap for the semiconductor industry
Alain Bouffioux - Philips Leuven 7 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Moore’s law today (3)• Roadmap for semiconductor industry
= only certainty in the current undefined future• Moore’s law will continue to apply: 20 years
– Economical limitation ?– Power consumption (Moore’s low in reverse direction)– Architectural gap between IP-blocks & application
(middleware still more complex…)• Progresses in semiconductors
= fuel the innovation = fuel the software revolution= fuel the wireless revolution
(WLAN, WPAN, WBAN, …) • …• Examples:
WBAN & sensors, RFID applications, camera to swallow, flexible display…
Alain Bouffioux - Philips Leuven 8 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The evolution of some CE products (1)
ConsumerConsumerComputerComputer
CommunicationCommunication
CDCDDCCDCC
CDCD--ii
DVDDVD
STBSTB
Alain Bouffioux - Philips Leuven 9 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The evolution of our CE products (2)• The Residential Gateway (Set-Top-Box) as the link
between the home and the world-wide information infrastructure.
STB Home NetworkWorld-wide
communicationinfrastructure
Alain Bouffioux - Philips Leuven 10 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The evolution of our CE products (3)
• The STB (in home) as the gateway to various services. Local Server provides 2 kind of services:– Broadcast
Analogue & digital TV, NVOD, PPV– Point-to-point (Home to local server)
Home shopping, VOD, e-mail, Web browsing, PC connection...
Network
Local server
Internet
Local server
Up to 800 homes
Alain Bouffioux - Philips Leuven 11 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The evolution of our CE products (4)• The STB as a key element of the home network
To telephone Network
Home Network
Residential Gateway
Computer
DVD Jukebox
Television
DiskRecorder
To satelliteNetwork
To cable Network
Alain Bouffioux - Philips Leuven 12 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The evolution of our CE products (5)
• 3C Convergence - Progressive• New products combine all 3 functions• Products always more and more complex• Products have always new features• Lifetime of products is always shorter
Alain Bouffioux - Philips Leuven 13 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Factors enabling such evolutionCompression is one among the various factors (all
powered by semiconductor progresses) that enable multimedia.
Software (methodology,user interface ...)
International cooperation(interoperability &economy of scale)
Disc capacity (DVD), communicationgoing digital (Modems, xDSL, ATM ...)
Multimedia
Electronics(Memory capacity,clock frequency,µP architecture, ...--> decoding at low cost
Audio/VideoCompression(e.g. CCIR601 vs MPEG)
Alain Bouffioux - Philips Leuven 14 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
BUT !!• Convergence of technologies (consumer,
communication, computer) All products combine all three technologies
• BUT !
• Divergence of applications– Home consumer, Multimedia phone, Camera, PDA,
Office computer, Automotive…– High number of potential products
Technology push Market pull (user centric approach)
Alain Bouffioux - Philips Leuven 15 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Agenda• Introduction - The evolution of Audio/Video
consumer products and the role of compression techniques.
• Audio & Video compression principles• Audio compression• Video compression• Audio/Video synchronisation• The MPEG model and its situation in a
communication context• Application to DVD (Digital Versatile Disc)• Application to DVB (Digital Video Broadcasting)• Conclusion
Alain Bouffioux - Philips Leuven 16 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Compression in first A/V Products (1)• First Audio/Video products made compression without
knowing it was compression.• How ?
By removal of irrelevancies• Audio and Video characteristics
Audio VideoSpectral
SensitivityGood Bad
SpatialSensitivity
Bad Good
Alain Bouffioux - Philips Leuven 17 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Compression in first A/V Products (2)• Audio products
From 2 to 7.1 channels are enough to provide the spatial resolution.
• Video productsThree colours (RGB) are enough to provide the spectral resolution.
Alain Bouffioux - Philips Leuven 18 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The need for more compression (1/5)• Audio: Compression needed in spectral domain• Bitrate of a stereo audio source (CD-DA encoding)
Sampling frequency : 44.1 kHzStereo16-bit per sampleBitrate = 44100 * 2 * 16 = 1.41 Mbit/sec
Audio waveform (time)
time
Alain Bouffioux - Philips Leuven 19 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The need for more compression (2/5)• Video: Compression needed in spatial domain• Bitrate of a video source (CCIR 601 - 50 Hz countries)
25 images per secondYUV coding (Y: luminance - U,V : Chrominance)Y: 8 bit per pixel - U,V: 1 pixel on 2 coded, 8 bit per pixelBitrate = (576*720)*25*16 = 166 Mbit/sec
720 samples
576lines
Video image
Alain Bouffioux - Philips Leuven 20 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The need for more compression (3/5)• Channels availables for AV transmission
– Analog television channel (compatibility)Cable (bandwidth = 8 MHz) Satellite (Bandwidth = 30-40 MHz)
⇒ Capacity around 40 Mbit/sec
– Compact disc (CD)For 74 min. play time : 1.41 Mbit/sec
Alain Bouffioux - Philips Leuven 21 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The need for more compression (4/5)• MPEG-1 target (Moving Picture Expert Group)
(Video-CD : 74 min. constraints)
But quality was judged too poor (about VHS quality)
Compression
Video : 166 Mbit/sec
Audio : 1.4 Mbit/sec1.4 Mbit/sec
Alain Bouffioux - Philips Leuven 22 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The need for more compression (5/5)• MPEG-2 target
– Program stream (DVD)
– Transport stream (DVB)
Compression
1 program(video, multichannelaudio, ....)
= motivation for the capacityincrease of the CD (--> DVD)
3-9 Mbit/sec (variable bitrate)(but higher quality than MPEG-1)
Compression
n programs(video, multichannelaudio, ....)
about 40 Mbit/sec (constant bitrate)(DVB-Satellite & DVB-Cable)
Alain Bouffioux - Philips Leuven 23 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Principles of compression (1/2)• Compression (or source coding) is achieved by suppressing
information : – redundant information – irrelevant information
• Suppression of redundant information ⇒ lossless compression example: PCM to DPCM,DCT
The original signal and the one obtained after encoding and decoding are identical
DecompressionCompressionFc(x,y,t)
Rc kbps Ri < Rc
Fp(x,y,t) = Fc(x,y,t)
Rp = Rc
Alain Bouffioux - Philips Leuven 24 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Principles of compression (2/2)• Suppression of irrelevant information
⇒ lossy compression Example: bandwidth limitation, masking in audio
•
The original signal and the one obtained after encoding and decoding are different but are perceived as identical
DecompressionCompressionFc(x,y,t)
Rc kbps Ri < Rc Rp = Rc
Fp(x,y,t) <> Fc(x,y,t)
Alain Bouffioux - Philips Leuven 25 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Audio Demonstration
From “Borderline” Madonna - Stereo - 16 bit/channelCompression used AAC
Compression
Decompression
Original 705 kbps
-
128kbps
64kbps
32kbps
16kbps
Alain Bouffioux - Philips Leuven 26 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
MOS scale (1/2)• Signal distortion is not a good measure of the
performance of a lossy compression method⇒ an other method is necessary: MOS scale (Mean Opinion Score)
• The five-grade CCIR impairment scale (Rec.562)1(Very annoying), 2(Annoying), 3(Slightly annoying), 4(Perceptible but not annoying), 5(Imperceptible)
• Example:Double blind test
Alain Bouffioux - Philips Leuven 27 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
MOS scale (2/2)
Original signal
Listener answers to :1. Which signal is the original ? 2 or 3 ?2. Grade the other one ?
Exchange box(Random for listener)
Selector(controlled by listener)
Compressed signal
Compressed
3
1
4
Impairmentscale5
Min value
Sequence2 3
Original
Mean value
Max value
Alain Bouffioux - Philips Leuven 28 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Compression to VBR or CBR
Constantbit rate
Bitrate
Complex
Distorsion
Simple
Constant quality
• CBR (Constant Bit Rate) vs VBR (Variable Bit Rate)• Scene more complex ⇒ Higher bit rate for same quality• CBR ⇒ variable quality (example : Video CD artefact)• Constant quality ⇒ VBR necessary (e.g.: DVD-Video)
Alain Bouffioux - Philips Leuven 29 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Video demonstration
Alain Bouffioux - Philips Leuven 30 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The compression trade-off• Compression techniques are still making progress• Trade-off Complexity/Quality/Bit Rate• New technique may result in new trade-off
ComplexityComplexity
MPEG Layer 1MPEG Layer 1
MPEG Layer 2MPEG Layer 2
MPEG Layer 3MPEG Layer 3
MPEG AACMPEG AACOther TechniqueOther TechniqueSpeech codingSpeech coding
QualityQuality
BitrateBitrate
Alain Bouffioux - Philips Leuven 31 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Agenda• Introduction - The evolution of Audio/Video
consumer products and the role of compression techniques.
• Audio & Video compression principles• Audio compression• Video compression• Audio/Video synchronisation• The MPEG model and its situation in a
communication context• Application to DVD (Digital Versatile Disc)• Application to DVB (Digital Video Broadcasting)• Conclusion
Alain Bouffioux - Philips Leuven 32 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Audio compression in MPEG (1/5)• Based on psycho-acoustics• Compress the bit rate without affecting the quality
perceived by the human ears (based on the imperfection of human ears)
• Removal of irrelevancies• 4 main principles :
– Threshold of audibility– Frequency masking– Critical bands– Temporal masking
Alain Bouffioux - Philips Leuven 33 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Audio compression in MPEG (2/5)• Principle 1: Threshold of audibility
⇒ Not all frequency components need to be encoded with the same resolution. Nr_bit(f) = (signal/threshold)db/6
0.2Frequency (kHz)
20
00.02 0.10.05
100
80
60
40
Sound level(db)
50.5 1 2 2010
Alain Bouffioux - Philips Leuven 34 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Audio compression in MPEG (3/5)• Principle 2: Frequency masking
⇒ Analysis of the incoming signal
0.2Frequency (kHz)
0
20
0.050.02 0.1
60
40
80
Sound level(db) 100
Masked signal
0.5 1 2 5 10 20
New threshold of audibilitydue to masker signal
Masker signal
Alain Bouffioux - Philips Leuven 35 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Audio compression in MPEG (4/5)• Principle 3: Critical bands
– Human ear may be modelled as a collection of narrow band filters– Bandwidth of these filters = critical band– critical band
(<100 Hz) for lowest audible frequencies(≈ 4 kHz) for highest audible frequencies
– The human ear cannot distinguish between two sounds having two different frequencies in a critical band.Example : when we hear 50 & 60 Hz at the same time we cannot distinguish them.
– Consequence : Noise masking threshold depends solely of the signal energy within a limited bandwidth domain.The largest sound is taken as the representative of the critical band.Necessity to analyse the signal at 100Hz resolution at low-frequency
Alain Bouffioux - Philips Leuven 36 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Audio compression in MPEG (5/5)• Principle 4: temporal masking
⇒ selection of the frame duration for frequency analysis and encoding.
0~(-10msec) 0
Envelope of masker
Pre masking
Level(db)
Post masking
Simultaneous masking
Time~100msec
Alain Bouffioux - Philips Leuven 37 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
An enabling tool : the filter bank (1/2)
Digital audio signal, Fs
ENCODING / DECODING
Reconstructed signal, Fs
SynthesisINTERPOLATION
Analysis
DECIMATION
n subband signals, Fs
Filter bank
n subband signals, Fs
n subband signals, Fs/n
n subband signals, Fs/n
Filter bank
Alain Bouffioux - Philips Leuven 38 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
An enabling tool : the filter bank (2/2)• After decimation, same bit rate as original signal, but
signal decomposed in various frequency ranges ⇒ possibility of frequency based compression
• Filter-bank:Aliasing occurs due to decimation
• It exists a class of filter-bank such that aliasing is compensated in synthesis filter : QMF (QuadratureMirror Filter) but high complexity
• Pseudo-QMF (Polyphase filter bank) is used. Has good compromise between computation cost and performances
• Remark : Aliasing may occur if signal in a adjacent band is not reconstructed with an adequate resolution.
Alain Bouffioux - Philips Leuven 39 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The MPEG encoder
Signal to mask level for each subband(quantisation information)
Psychoacousticmodel.
Digital audioinput
Subbandsamples
FilterBank
Ancillary data
Quantisedsamples
Bitallocation
Bitstreamformatting
Encodedbitstream
Alain Bouffioux - Philips Leuven 40 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The MPEG filter bank • In MPEG, 32 equal-width subbands are used• For each subband, necessity to define the maximum
signal level and the minimum mask level.• BUT, at low frequencies :
bandwidth of subbands > critical bands• ⇒ Necessity to rely on an FFT in order to
compensate the lack of frequency selectivity of filterbank at low frequencies
Alain Bouffioux - Philips Leuven 41 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Psychoacoustic model & Bit allocation(1/2)
• An FFT compensates the lack of frequency selectivity of filterbank at low frequencies
• FFT : 512 samples (layer 1) & 1024 samples (layer 2)resolution for layer 1 : Fs/512 < 100 Hz
• A psychoacoustic model based on the FFT computes the signal to mask ratio for each subband (1 bit = 6db)
• Ideally, after allocation, quantisation noise < masking level• The scale factors are computed for each subband from the
filterbank output (floating point representation of samples)• The bit allocator adjust the bit allocation in order to meet the
bitrate requirement.• The bitstream syntax is dependent of the MPEG layer (See later)
Alain Bouffioux - Philips Leuven 42 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Psychoacoustic model & Bit allocation(2/2)
Frequency
Signal/MaskLevel(db)
Level(db)
Signal level
Bit allocation= 1 bit
Frequency
Mask level
Alain Bouffioux - Philips Leuven 43 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The MPEG decoder• Decoder is simple (Complexity is at encoder side)• Remark 1: DCC is MPEG-1 but DCC encoder has no FFT,
relies only on power in the 32 subbands ⇒ Higher bit rate (320 kbps) to reach transparent quality
• Remark 2: MPEG specifies bitstream syntax only. Encoder are given for information. Possibility of improvement.
SubbandSamplesrecons- truction
Bitstreamunpacking
Encodedbitstream
Ancillarydata
Subbandsamples
Filterbank
Decoded audio signal
Alain Bouffioux - Philips Leuven 44 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Audio features in MPEG• MPEG1 :
– Mono/stereo/dual/joint stereo (Possibility Dolby surround)
– Sampling frequencies : 32, 44.1 & 48 kHz– 3 layers : trade-off complexity/delay versus
coding efficiency of compression– Various bit rate : trade-off quality versus bitrate
• MPEG2 :– 5.1 channels– Sampling frequencies extended to 16, 22.05 &
24 kHz
Alain Bouffioux - Philips Leuven 45 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Dolby surround principles (1/5)• 4 channels carried by stereo pair ⇒ same tools as for stereo• Compatible with stereo installation
+90°
-90°
Phaseshifter
Lt
RtSurround Surround
Left
Configuration
Center Right
Center(C)
- 3 dB- 3 dB
Right (R)
Surround(S) 100 Hz -
7000 Hz
BP
- 3 dB- 3 dB
From 4 channels to 2 channels
Left (L)
Alain Bouffioux - Philips Leuven 46 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Dolby surround principles (2/5)
Encoding equations Simple decoder
Lt L .1
2C ..j 1
2S L Lt ⇒ Ld L .1
2C ..j 1
2S
Rt R .1
2C ..j 1
2S R Rt ⇒ Rd R .1
2C ..j 1
2S
C Rt Lt
2⇒ Cd .1
2L .1
2R C
S Lt Rt
.2 j⇒ Sd ..j 1
2L ..j 1
2R S
Alain Bouffioux - Philips Leuven 47 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Dolby surround principles (3/5)
3 dBS
3 dB
3 dB
L C3 dB
R
• Simple decoder provides only 3 dB channel separation(See previous equations)⇒ Need for improvement⇒ Dolby Surround pro-logic decoder (next slide)
Alain Bouffioux - Philips Leuven 48 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Dolby surround principles (4/5)Dolby surround pro-logic decoder
BP
C'
S'
R'
L'
Rt
-3 dB
Lt
-3 dB Direc-tionCom-pen-sation-90°
Rd
Sd
VCA
VCA
Cd
Ld
VCA
VCA
Alain Bouffioux - Philips Leuven 49 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Dolby surround principles (5/5)Performance of Dolby pro-logic decoder
Channel separation larger than 35 dB
> 35 dBS
> 35 dB
> 35 dB
L
> 40 dB
C> 35 dB
> 40 dB
R
Alain Bouffioux - Philips Leuven 50 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
5.1 surround soundMPEG-2 surround configurations (front/back)
• 3/2• 3/0 + 2/0• 3/1• 2/2• 2/0 + 2/0• 3/0• 2/1• 2/0• 1/0+ LFE (opt.)
(Fs/96) 15-120Hz
SurroundLeft
SurroundRight
(+LFE)
CenterLeft Right
Alain Bouffioux - Philips Leuven 51 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Virtualisation• Virtualisation has no direct relation with the MPEG
standard.It is considered here only because it may be implemented in some of the future audio products (DVD, STB ...)
• Virtualisation is a product feature.• It allows reproduction of surround information (5.1,
3/1) on a stereo installation.
Alain Bouffioux - Philips Leuven 52 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Virtualisation principle
Listener
Real loudspeakerProcessing
Mono signal
Virtualloudspeaker
• Virtualisation = processing of the signal in such a way the source of the signal is perceived at a selected position outsidethe loudspeaker axis (virtual loudspeaker).
• Drawback : very sensitive to listener position (stability) • Remark : a mono signal coded in normal stereo is perceived
between the two loudspeakers
Alain Bouffioux - Philips Leuven 53 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Stereo widening
Listener
Real loudspeakerProcessing
Stereo signal
Virtualloudspeaker
• Also called Q-sound ™, incredible sound, azimuth positionning...
• The stereo sources are positionned at virtual locations for improving the stereo effect (cheap analog solution exists)
• Real sound comes from real loudspeakers. Perceived sound is as if stereo signals were coming from virtual loudspeakers
Alain Bouffioux - Philips Leuven 54 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Virtual surround• Virtual surround gives on a stereo installation the
subjective effect of a multichannel configuration.• Each channels is virtually positionned at a location
around the listener. The stereo installation performs the addition of the processed signals for each audio channel.
• Real sound comes from a stereo installation. Perceived sound is as if the various surround signals were coming from some virtually located loudspeakers.
Alain Bouffioux - Philips Leuven 55 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Summary of surround aspects
Stereo receiver Multichannel receiver
Stereo source Stereo widening(incredible sound,azimuth positioning)
3-D special effects(hall, theatre, stadium)
multichannel source Virtual surround Multichannel output
Remarks about Dolby surround pro-logic :Only carrier is stereo, source & presentation are multichannelCompatible with stereo installation (no surround effect except in
the case of surround virtualisation)
Alain Bouffioux - Philips Leuven 56 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Agenda• Introduction - The evolution of Audio/Video
consumer products and the role of compression techniques.
• Audio & Video compression principles• Audio compression• Video compression• Audio/Video synchronisation• The MPEG model and its situation in a
communication context• Application to DVD (Digital Versatile Disc)• Application to DVB (Digital Video Broadcasting)• Conclusion
Alain Bouffioux - Philips Leuven 57 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Video compression in MPEG (1/6)• Principles
– removal of intra-picture redundancy : Image is decomposed in 8*8 pixels sub-images.Each sub-image contains redundant information DCT transformation (in frequency domain) decorrelates the input signal.( most energy in low spatial frequencies)
– removal of interpicture redundancy :coding of difference with an interpolated picture (moving vectors)
– high frequent spatial frequencies quantized with lower resolution than low ones(remove irrelevancy)
– zig-zag scan and VLC (remove redundancy)
Alain Bouffioux - Philips Leuven 58 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Video compression in MPEG (2/6)• Result
– 4:2:2 CCIR 601 resolution : 166 Mbps (=25images/sec *576lines* 720pixels* 2(lum & chrom) *8bits)⇒ ± 3-4 Mbps (mean) in MPEG2
– 4:2:0 SIF resolution : 30 Mbps (=25 images/sec *288 lines *352pixels* 1.5(lum & chrom) *8bits)⇒ ±1.2 Mbps (CBR) in video CD (MPEG1)
Alain Bouffioux - Philips Leuven 59 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Video compression in MPEG (3/6)• Spatial redundancy reduction (DCT example)
158 0 -1 0 0 0 0 0 -1 -1 0 0 0 0 0 0 -1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
139 144 149 153 155 155 155 155144 151 153 156 159 156 156 156150 155 160 163 158 156 156 156159 161 162 160 160 159 159 159159 160 161 162 162 155 155 155161 161 161 161 160 157 157 157162 162 161 163 162 157 157 157162 162 161 161 163 158 158 158
158 0 -1 -1 -1 -1 EOBzig-zag scan
1260 -1 -12 -5 2 -2 -3 1 -23 -17 -6 -3 -3 0 0 -1 -11 -9 -2 2 0 -1 -1 0 -7 -2 0 1 1 0 0 0 -1 -1 1 2 0 -1 1 1 2 0 2 0 -1 1 1 -1 -1 0 0 -1 0 2 1 -1 -3 2 -4 -2 2 1 -1 0
DCT
Quantisation
Alain Bouffioux - Philips Leuven 60 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Video compression in MPEG (4/6)• Temporal redundancy reduction
B
5
Bi-directional prediction
I : Intra-coded pictureP: Predicted pictureB: Bi-directionally interpolated picture
4
B
Order ofpresentation
Order oftransmission
BI P
0 3
B P
1 2 6
B
Prediction
I B P B
Increase of compressionrate
0 1 2 3 4
7
B P B
5 9
B I P
8
P B B P I B
86 7 9
Alain Bouffioux - Philips Leuven 61 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Video compression in MPEG (5/6)• Model of a possible encoder
BufferData
Regulator
Motion estimation
Motionvectorsand modes
Format conversion and picture reorder
Inputpictures
Picturestore andprediction
1/DCT
DCT VLC
1/Q
Q Multi-plex
Alain Bouffioux - Philips Leuven 62 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Video compression in MPEG (6/6)• MPEG1 en MPEG2 video features
– MPEG1• sequential picture• resolution : SIF format 288(240)*356*24,25 or 30 Hz
– MPEG2• sequential or interlaced• various levels : low level (SIF: 288*356), main level
(CCIR601: 576 * 720), high 1440 level (HDTV: 1152*1440), high level (EQTV: 1152*1920)
• various profiles (toolboxes) : simple profile (No B picture), main profile (=MPEG1+interlaced), SNR scalable profile (allows graceful degradation (noise improvement at same resolution), spatial scalable profile (hierarchical coding : improvement at higher resolution), high profile.
Alain Bouffioux - Philips Leuven 63 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Agenda• Introduction - The evolution of Audio/Video
consumer products and the role of compression techniques.
• Audio & Video compression principles• Audio compression• Video compression• Audio/Video synchronisation• The MPEG model and its situation in a
communication context• Application to DVD (Digital Versatile Disc)• Application to DVB (Digital Video Broadcasting)• Conclusion
Alain Bouffioux - Philips Leuven 64 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Synchronisation
Synchronisation in the multimedia context refers to the mechanism that ensures a temporal
consistent presentation of the audio-visualinformation to the user
Alain Bouffioux - Philips Leuven 65 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Intramedia synchronisation
Network
∆
∆ T1T2 = ∆
T1
Capture time
Audio signal Encoder
∆T = Constant
Decoder
Presentation time
Audio signal
∆T between capture & presentation = Constant≡ Same clock frequency & Data on time ⇒ Need for corresponding tools
Alain Bouffioux - Philips Leuven 66 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Intermedia synchronisation
Network
T_audio = Constant
T_video = Constant = Capture time
∆
Video signal
Audio signal
Encoder
Capture time∆
T_Audio ∆Presentation time
DecoderVideo signal
Audio signal
Presentation time
∆T_Audio = ∆T_Video≡ (Sampled at the same time ⇒ Presented at the same time)⇒ Possible tools : common time base and presentation control (media
synchronisation with the common time base)Ex.: Lip_sync (requirement: |delay_difference| < 80msec)
Alain Bouffioux - Philips Leuven 67 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Recovery of clock in CBR• CBR = Constant Bit Rate • if the clock to recover is synchronous with transport clock ⇒
Recovery of clock but not of common time base• Remark : possibility to slave DSM to local clock
Filter
CBR stream
Phaseerror
Filling level50%
Time Information carriedby each sample
VCO
Processing
Recoveredclock
time
Alain Bouffioux - Philips Leuven 68 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Recovery of clock and time base in VBR
• VBR = Variable Bit Rate • Need for insertion of time stamps (OUTPUT TIME)
Output time stamp says for example : “It is now 16h25”Receiver adjusts its own horloge to the received time stamp
• Recovery of clock & of common time base
Time information carriedonly by time stamps
Time stamp extraction
Time counter
Recovery of clock & time
First time stamp
Others
Clock
Data stream
Data streamTime stamps
Counter
Recovered clock
Error Filter VCO
VBR Stream
Counter sample(=Time stamps)
Channel
Time stamps
Alain Bouffioux - Philips Leuven 69 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Synchronisation with common time base• Insertion of time stamp (=INPUT TIME)
Input time stamp says : “Input has been sampled at 16h29”.Receiver presents the sample at (its input time stamp + maximum encoding and decoding delay).Alternative: transmission of presentation time stamp (input time+delay)
BufferingMediaoutput
Media input
Time clock(Recovered)
Comparison of time clockwith sampled time clock
Sample "Time clock"Assemble frame
Time clockTimestamp
Processing
Processing
Channel
Alain Bouffioux - Philips Leuven 70 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Getting data on time• “On time” ⇒ Not too late, not too early
No buffer over- or underflow • Flow control : not applicable in broadcasting• Common time base and
Definition of a standard target decoder that describes the data consumption pattern of the receiver.
• Remark: Direct MPEG (Microsoft) does not use time information for clock recovery but relies on flow control
Alain Bouffioux - Philips Leuven 71 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Streams• Idea of continuity (pipelining)• Carry time information for clock recovery • No flow control (allows broadcasting)
The emitter must have a precise knowledge of the receiver data consumption pattern (explicit in MPEG STD)
• Just-in-timeShorter delay and smaller buffer size than with flow control
• Two aspects in synchronisation :Clock recovery & timing control (model & buffering)
Alain Bouffioux - Philips Leuven 72 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Requirement on for stream transport
• Data information ⇒ BER (Bit Error Rate) requirementNo repetition of frame possible ⇒ FEC (Forward Error Correction)
• Time information ⇒ No jitter
Alain Bouffioux - Philips Leuven 73 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Agenda• Introduction - The evolution of Audio/Video
consumer products and the role of compression techniques.
• Audio & Video compression principles• Audio compression• Video compression• Audio/Video synchronisation• The MPEG model and its situation in a
communication context• Application to DVD (Digital Versatile Disc)• Application to DVB (Digital Video Broadcasting)• Conclusion
Alain Bouffioux - Philips Leuven 74 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
What is MPEG ? (1/2)
• Moving Picture Expert Group• Still active (MPEG-21 is currently in development)• International standard (ISO/IEC)
⇒ Interoperability & economy of scale• Compression of audio and video and multiplexing in a
single stream• Definition of the interface not of the codecs
⇒ room for improvement • MPEG-1 : until 1.5 Mbps, for DSM
Progressive picture, stereo (Dolby surround)
Alain Bouffioux - Philips Leuven 75 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
What is MPEG ? (2/2)• MPEG-2 : Various bit rates (CBR & VBR)
Program stream for DSM, transport stream for networkInterlaced picture, 5.1 audio channels Definition of various video levels (e.g. CCIR601 resolution: 4-9 Mbps, HDTV:15-25 Mbps) and profiles
• MPEG-3 : Cancelled, integrated in MPEG-2(Initially : for HDTV)
• MPEG-4 : standard for audio, video and graphics in interactive 2D and 3D multimedia communication. (Initially : low bit rate for real-time personal communication)
• MPEG-7 : Multimedia contents description interface• MPEG-21 : Focus on multimedia distribution and on DRM
aspects.
Alain Bouffioux - Philips Leuven 76 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The MPEG model (1/2)
Audiodecoder
Audio signal
Videosignal
Presented signals
Multiplexer
Videodecoder
Captured signals
Audioencoder
Videoencoder
Audio signal
Videosignal Digital storage medium
orNetwork
Transmission channel
Demulti-plexer
Alain Bouffioux - Philips Leuven 77 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The MPEG model (2/2)• Compression of audio & video and multiplexing in a single
stream• Guarantees intramedia and intermedia synchronisation.• MPEG defines an interface
– bitstream syntax– timing of the bitstream ⇒ STD specifying timing stream ⇒ STD specifying timing requirement (ideal
Alain Bouffioux - Philips Leuven 78 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Components of the MPEG standard• The MPEG standard is composed of 3 main parts :
– Audio : Specifies the compression of audio signals– Video : Specifies the compression of video signals– System : specifies how the compressed audio and video
signals are combined in the multiplexed stream (program stream or transport stream).
• Each part specifies :– The bitstream syntax – The timing requirement and the related information (bit
rate, buffer needs)
Alain Bouffioux - Philips Leuven 79 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Synchronisation Mechanism (1/2)
System decoderanddemultiplexer
Multiplexer and systemencoder
Audiodecoder
Videodecoder
ComparisonPTS and STCand presentation
Videooutput
ComparisonPTS and STCand presentation
Audiooutput
Extractionof PCR (SCR)
STC
Audioencoder
Videoencoder
Assemble pictures,Sample STC for PTS
Videoinput
Assembleaudio frames,Sample STCfor PTS
Audioinput
Sample STCfor PCR(SCR)
STC
Transmission channel
Alain Bouffioux - Philips Leuven 80 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Synchronisation Mechanism (2/2)• PCR for TS & SCR for PS (but same concept)• Clock & time base recovery: Time-stamping at OUTPUT (PCR
included in TS multiplex, SCR in pack header)• Audio & video clock locked to STC ⇒ easy recovery (see next
slide)• Synchronisation of audio & video to common time base (Time
stamping at Input)• STD is defined (because of the absence of flow control)
streams are such that STD buffers never over- or underflow• In TS, many program in a single stream but unique clock per
program.• Time information ⇒ “No Jitter” requirement for transport
Alain Bouffioux - Philips Leuven 81 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Clock recovery in receiver
VCO Audio clockdivider
Video clockdivider Video
clock
Audioclock
Audiooutput
STC
ComparisonPTS and STCandPresentationDecoded
audio
PTS
STC(Counter)
PCR
STC
Error Low Pass Filter(Integrator)
Load first PCR
Alain Bouffioux - Philips Leuven 82 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
MPEG program & transport streams• Program streams:
– Relatively error free environment– program stream packet may have variable and great
length– Single time base
• Transport streams:– environment where errors are likely– many programs (independent time base)– Transport stream packet : fixed, 188 bytes– Contains tables
Alain Bouffioux - Philips Leuven 83 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
MPEG in a communication context (1)• “Typical” communication system
Channelencode
Channeldecode
Bit-stream
Sourcedecode
Informationsink
Digitaloutput
Format Decrypt
SourceencodeFormat
Informationsource
Digitalinput
Encrypt
Synchro-nisation
Digitalwaveform
Otherdestination
Demodu-late
Demulti-plex
Multipleaccess
Modu-late
Multi-plex
Channelbits
Othersources
Multipleaccess
Channel
Alain Bouffioux - Philips Leuven 84 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
MPEG in a communication context (2)• MPEG : Source coding only (bit rate reduction) +
multiplexing• The MPEG stream must be adapted to the channel in what
concern its physical characteristics and in order to get the required QoS (Quality of Service) & Security– Encryption – Channel coding (forward error correction, interleaving,
modulation codes)– multiplexing & formatting– modulation (frequency allocation)– multiple access method
• Some channels : CD/DVD - satellite - cable - ATM - 1394
Alain Bouffioux - Philips Leuven 85 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
MPEG in a communication context (3)
VideoEncoder
MPEG2 compression layer
Audioencoder
Audio,videosources
ES(ElementaryStream)
Adap-tationto thechannel
PS(1 pro-gram)
MPEG2 system layer
PSMulti-plexing
Adap-tationto thechannel
DVB, DVD ...
Disc
Satellite
TSMulti-plexing
TS(n pro-grams)
Adap-tationto thechannel Cable
TS (Transport Stream)orPS (Program Stream)
• A simple view of MPEG in the communication context
Alain Bouffioux - Philips Leuven 86 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Agenda• Introduction - The evolution of Audio/Video
consumer products and the role of compression techniques.
• Audio & Video compression principles• Audio compression• Video compression• Audio/Video synchronisation• The MPEG model and its situation in a
communication context• Application to DVD (Digital Versatile Disc)• Application to DVB (Digital Video Broadcasting)• Conclusion
Alain Bouffioux - Philips Leuven 87 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
CD : Some concepts
• Hard disk vs compact disc : more differences than just storage technique.HD developed for data storage and recording, CD developed for stream storage (CD-DA) ⇒ their basic differences
• Questions– track form? – read direction? Why?– CAV or CLV? Why?– Access time : CD-ROM vs HD?– Data storage: on which face? – Production method?– Capacity?– Sensitivity to error? Diameter of a possible hole?– Standard = Interface definition : CD vs HD ?
Alain Bouffioux - Philips Leuven 88 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
CD-DA: Encoder model (1/3)
Modulation
6 samples = 24 bytes= 1 frame
EFM + 3 merging bits
Synchronisation pattern27 bit/frame
561 bits / frame
CD-ROM1 sector = 98 frames75 sectors/sec.
588 channel bit/frame
Right
LeftA/D conversion
PCM 44.1 kHz16 bit/sample/channel
Subcode(1 byte / frame)
Physical layer
Error correctionencoding
32 bytes/frame
Alain Bouffioux - Philips Leuven 89 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
CD-DA: Encoder model (2/3)• The CD-DA physical layer adapts the input stream (audio) to the
requirements of the channel
– Modulation : EFM (Eight to fourteen modulation + 3 merging bits) Pit & land length (number of successive 0 or 1 as written to disc): between 3 and 11 channel bits DC free code for adaptation to the channel bandwidth & for clock recovery considerations.
– Error correction (Cross-interleaved Reed-Solomon code)Interleave placed between C1 & C2 ECC.Next slide presents only principles and not real CD implementation.
Alain Bouffioux - Philips Leuven 90 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
CD-DA: Encoder model (3/3)– Error correction : addition of redundancy in order to be
able to correct errors (e.g. RS(28,24,5)*RS(32,28,5))Principle :
– Interleaving : time diversity in order to deal with error burst.Successive erroneous channel bits (burst error) do not damage the same Reed-Solomon table.
4 bytesC2 codewords
24 bytes28 bytes
Data
4 bytes
C1code-words
Alain Bouffioux - Philips Leuven 91 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
CD-ROM encoder model
2336 bytes
Mode 1
Mode 2
Mode 1
2336 bytes
Additionalerror detection &error correctionencoding
Optional EDC 4 bytes
User data 2324 bytes
Subheader 8 bytes
Subheader 8 bytes
User data 2048 bytes
Sync Pattern 12 bytes
Additionalerror detection &error correctionencoding
User data 2048 bytes
Zeroes 8 bytes
Header 4 bytes
Header 4 bytes
2340 bytes
For EDC only
Video-CD uses CD-ROM mode 2 sectors
OR
12 bytes
2340 bytesScramblingOR1 CD-ROM sector= 2352 bytes
Alain Bouffioux - Philips Leuven 92 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
From CD to DVD : the motivation• Motivation = increase the capacity• Why ? - Requirement of the motion picture industry
– Playback time : more than 135 min. (duration of 90% of films)
– Picture quality : superior to laser disc– Audio quality : 5.1 channels surround– Language/subtitles : 3 languages minimum.⇒ capacity needs : more than 4.7 Gbytes
• Where ? - In physical layer• DVD : developed specifically for audio/video
( ≠ video CD).
Alain Bouffioux - Philips Leuven 93 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The DVD physical layer (1/2)
Recorded sectors2366 bytes(13 *182 bytes)
8/16modu-lation
37856 channel bits
Synchronisation(2*13)*32 channel bits
38688 channel bits(eqv. to 2418 bytes)
EDC : Error Detection CodeECC : Error Correction CodeCPR-MAI : CoPyRight MAnagement Information
CPR-MAI - 6 bytes
ID (incl. sector#)4 bytes
Data2048 bytes
+EDC
Scrambling
EDC4 bytes
6 bytes
ECC(per group of 16 sectors)
Data sector2064 bytes(12 * 172 bytes)
RowInter-lea-ving
Alain Bouffioux - Philips Leuven 94 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The DVD physical layer (2/2)• Objective was the storage of 2K sectors • Error Correction Code (Reed-Solomon) - add redundancy
• Modulation - time diversity(Number of consecutive 0 : between 2 and 10)Pit and land length : between 3 and 11 (Idem CD)
• Synchronisation : for sector reconstruction.
16 bytesPO (Outer Parity)
172 bytes192 bytes
16 data sectors
(12 * 172 bytes)*16
10 bytes
PI(InnerParity)
Alain Bouffioux - Philips Leuven 95 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
DVD: the capacity improvement (1/4)
• Increase of channel bit density ( gain = 4.50)Min pit length : (0.83µ ⇒ 0.4µ)Track pitch : (1.6µ ⇒ 0.74µ)Diameter of laser spot (÷ wavelength/NA)Wavelength (780µ ⇒ 640 nm) ⇒ gain = 1.5NA (0.45⇒ 0.60) ⇒ gain = 1.78reduced margin ⇒ gain = 1.68
• Modulation:EFM (8 to 17 bit) ⇒ 8 to 16 ⇒ gain = 1.06
• Error correctionRS(32,28,5)*RS(28,24,5) ⇒ RS(182,172,11)*RS(208,192,17)
⇒gain = 1,16
Alain Bouffioux - Philips Leuven 96 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
DVD: the capacity improvement (2/4)
• No subcode ⇒gain = 1.03• Sync pattern ⇒gain = 1.03• Better sector formatting
sector length (2352 bytes ⇒ 2064) ⇒gain = 1.14• Other (e.g. recorded area) ⇒gain = 1.07
Total gain : 7.2Capacity per side : 650 MBytes (mode 1) ⇒ 4.7 Gbytes
Alain Bouffioux - Philips Leuven 97 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
DVD: the capacity improvement (3/4)
0.6 mm
0.6 mm
0.6 mm
0.6 mm
For layers 0 and 1
A side
Single-layer double-sided disc
Single-layer single-sided disc
0.6 mm
0.6 mm
B sideDual-layer double-sided disc
0.6 mm
0.6 mm
Dual-layer single-sided disc
Alain Bouffioux - Philips Leuven 98 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
DVD: the capacity improvement (4/4)• Capacity of the various types
Single-layer single-side 4.7 GbytesDual-layer single-side 8.5 GbytesSingle-layer double-side 9.4 GbytesDual-layer double-side 17 Gbytes
Alain Bouffioux - Philips Leuven 99 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The 3 components of the DVD-V standard• DVD = DVD (= 3 random letters) (previously :
Digital Versatile Disc, Digital Video Disc)• DVD-V : DVD - Video
Part 1 : Physical specification
Part 2 : File system specification
Part 3 : Video specification
How blocks may be retrieved. Definition of the file andvolume structure.
How blocks of 2048 bytesare stored on the disc
Contents of the data block.How audio and video are mapped to the block, file and volume structure
Alain Bouffioux - Philips Leuven 100 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Some DVD-V features (1/2)Presentation data = MPEG program stream, VBR, max peak bit rate = 10.08 Mbps)
Video data 1 stream Mpeg1Mpeg2 (ML@MP)16:9 or 4:3 aspect ratioNTSC or PAL
Audio data max 8 streams Mpeg2 + 7.1 extension(50 Hz countries)AC-3 (60 Hz countries)Linear PCM (incl. 96 kHz- 24 bits)
Sub picture data max 32 streams Run length encoded(subtitles) Bit map
Alain Bouffioux - Philips Leuven 101 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Some DVD-V features (2/2)• Seamless playback
Languageparental lockMulti-angle cameraStill pictureRegional coding (6 regions)
• System menuAudio stream selectionSubtitle selectionAngle selection
• EncryptionDecryption key hidden on the disc.
Alain Bouffioux - Philips Leuven 102 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The DVD family of products
UDF & ISO9660
Part 3:Application Video
specificationPart 2:File system
Part 1:Physical
UDF
Read only
DVD-ROM DVD-Video
Audiospecification
Write-once
UDF | (UDF & ISO9660)
Rewritable
DVD-Audio DVD-R DVD-RAM
Alain Bouffioux - Philips Leuven 103 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Recording on disk - principle• Products: CD-R, CD-RW, DVD+/-R(W) …• CD principle: reflectivity of pits & lands are different.
Pits and lands are used to store 0 and 1.• CD-RW principle: reflectivity of the two phases of the recording
material (amorphous, crystalline) are different.Controlling the phase allows storage of 0 or 1.
• To Amorphous state (low reflectivity):T above melting point (600°C) & fast cooling
• To Crystalline state (high reflectivity):T above 200°C for a sufficient time
• Recording: by the laser heating the recording layer • Reading: by laser as for CD (-> compatibility)
Alain Bouffioux - Philips Leuven 104 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Blu-Ray DVD
CD DVD Blu-Ray DVD Owner Philips & Sony many
companies Philips, Sony, Hitachi…
Data depth 1.2 mm 0.6 mm 0.1 mm Wavelength 780 nm 650 nm 405 nm NA 0.45 0.60 0.85 Spot dimension (relative)
1 1/2.6 1/13.2
Spot dimension (relative)
1 1/5.2
Capacity 650 MB (data) 4.7 GB (per side)
22.5 GB
Duration 70 min @1.4 Mb/sec
135 min @ 4.6 Mb/sec
150 min @ 20 Mb/sec
Focus VHS quality Standard TV HDTV
Alain Bouffioux - Philips Leuven 105 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
SACD• = Super-Audio-CD• = Response of Philips/Sony on the thread DVD-Audio
brings on the revenues from CD portfolio • Multi-layer hybrid scheme
– One layer for playback in CD player at standard quality– One layer for playback in SACD player at enhanced
quality (DVD-like, 4.38 Gbytes)• Already on the market and in consumer homes
(marketing argument)• DSD technology (Direct Stream Digital)• delta sigma DAC to decode the 2.82 Mbps PDM
stream • Lossless compression, 5.1 multichannel, encrypted• 120 db (=20-bit), 100 kHz BW
Alain Bouffioux - Philips Leuven 106 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Agenda• Introduction - The evolution of Audio/Video
consumer products and the role of compression techniques.
• Audio & Video compression principles• Audio compression• Video compression• Audio/Video synchronisation• The MPEG model and its situation in a
communication context• Application to DVD (Digital Versatile Disc)• Application to DVB (Digital Video Broadcasting)• Conclusion
Alain Bouffioux - Philips Leuven 107 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Adaptation to the DVB channel
Sequence of bits
(Encrypted) TS Channelcoding
Analog waveform
Modulation Physicalchannel
• Channel coding : transforms the TS in an other sequence of bits containing the same information than the input stream but more robust against the imperfections of the transmission on the physical channel cost : a higher bit rate
• Modulation : transforms an input sequence to an analogwaveform for transmission over the physical channel
Alain Bouffioux - Philips Leuven 108 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Channel coding (1/3)• Unlike source coding that removes redundancy,
channel coding adds redundancy in a structured way so that the decoder be able to detect and/or correct the errors introduced by the physical channel.
Channeldecoding
Source decoding
Sink
Source coding
Source Channelcoding
Quasi-error-freechannel (e.g.BER<1E-10)
Demodulation
Error-pronechannel (e.g.BER=1E-3)
Modulation
Physicalchannel
Alain Bouffioux - Philips Leuven 109 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Channel coding (2/3)• Channel coding may include :
– Spectral modification of the signalfor adaptation to the channel (e.g. remove DC, spectrum shaping like uniform distribution in the frequency space ...)
– FEC : Forward Error CorrectionAddition of redundancy in order to allow error detection and/or correction (example : The total of bought articles is similar to a parity byte)
188 bytes
188 bytes
Allows correction of 8 erroneous bytes
After error correctionencoding
Original sequence
Message Parity bytes
16 bytes
Alain Bouffioux - Philips Leuven 110 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Channel coding (3/3)– Interleaving
Time diversity in order to deal with error bursts.The successive bytes of information are dispersed in time on thetransmission channel in such a way that an error burst does not affect neighbouring bytes. Interleaving is often combined with FEC so that error bursts could be corrected by the FEC.
Example :
AEIMBFJNCGKODHLPQ.....
AEIMBFJ&&&KODHLPQ.....
----> A burst of errors affects bytes belonging to different error correction blocks
Channel
Deinterleaving
AB&DEF&HIJKLM&OPQ.......|---|---|---|---|
Interleaving
ABCDEFGHIJKLMNOPQ.......
& : Erroneous byte| : Beginning of an error correcting block- : Element of an error correcting block
Alain Bouffioux - Philips Leuven 111 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Modulation in DVB (1/3)• Different modulation techniques :
– Cable : QAM– Satellite : QPSK– Terrestrial : OFDM
• Why ?Modulation technique depends on :– Physical characteristics of the channel– Compatibility constraints with actual analog
transmission
Alain Bouffioux - Philips Leuven 112 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Modulation in DVB (2/3)
• Example : influence of SNR on modulation technique selected⇒ QPSK for satellite and QAM for cable
141E-7
4
1E-5
1E-6
6 8 10 12
QPSK
BER
0.001
1E-4
0.01
1.0
0.1
16 18
16QAM
20 22
32-QAM 64-QAM
24 26 28 SNR
Alain Bouffioux - Philips Leuven 113 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Modulation in DVB (3/3)• Satellite
Bandwidth : generally 27-36 MHzSNR low : about 10 db (power transmitted by satellite)direct path
• CableBandwidth : 8 MHz (50Hz countries) - 6 MHz (60Hz countries)SNR strong (about 25 db)Echoes from impedance mismatch in the network
• Terrestrial Bandwidth : idem as cable Multipath interference, signal level variation, ...
Alain Bouffioux - Philips Leuven 114 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
From TS to the DVB channel
QAMmodulation
QPSKmodulation
OFDMmodulation
DVB-C
DVB-S
DVB-T
Spectrumshaping
TS Outercoding
Inner inter-leaver
Innercoding
Inter-leaving
• Some blocks are identical for all standards (Cable, Satellite & Terrestrial)
• Inner & outer : terminology is derived from the view of the quasi-error-free channel composed of a transmitter and a receiver.
• Satellite & Terrestrial : More sensitive to error ⇒ inner coder is added
Alain Bouffioux - Philips Leuven 115 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Agenda• Introduction - The evolution of Audio/Video
consumer products and the role of compression techniques.
• Audio & Video compression principles• Audio compression• Video compression• Audio/Video synchronisation• The MPEG model and its situation in a
communication context• Application to DVD (Digital Versatile Disc)• Application to DVB (Digital Video Broadcasting)• Conclusion
Alain Bouffioux - Philips Leuven 116 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Questions
Alain Bouffioux - Philips Leuven 117 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
APPENDICES
Alain Bouffioux - Philips Leuven 118 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Agenda
Conditional access
• What is cryptography• Symmetric & public-key cryptography• Why cryptography for DVB ?• Conditional access information in MPEG/DVB• Conditional access mechanism• Conditional access interfaces
Alain Bouffioux - Philips Leuven 119 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
What is cryptography (1/2)• Why cryptography ?
– CONFIDENTIALITY - The message is not listened– INTEGRITY - The message is not modified– AUTHENTICITY - The message has been sent by Alice– NON-REPUDIATION - Alice cannot falsely deny she has
sent the message
ALICE
MESSAGE
BOB
Alain Bouffioux - Philips Leuven 120 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
What is cryptography (2/2)• Basic terminology
cryptographic algorithm or cipher
CiphertextEncryption
Plaintextor cleartext
Key Key
DecryptionOriginalplaintext
Alain Bouffioux - Philips Leuven 121 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Symmetric & public-key cryptography(1)
EncryptionPlaintextor cleartext
CyphertextDecryption
Originalplaintext
Key 1 Key 2
Symmetric cryptography Public-key cryptography Key1 = Key2 Key 1 ≠ Key 2
• Public-key cryptographyOne Public-key (known by everybody) : PKOne Private-key or Secret-key (kept secret) : SK
• C = EKey1(M) ⇒ M = DKey2(C) = DKey2(EKey1(M)) In public-key cryptography, key1 may be PK or SK and key2 is the other key.
Alain Bouffioux - Philips Leuven 122 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Symmetric & public-key cryptography(2)• Example of symmetric cryptography
Pseudo-randomnumber generator Secure
channelKey
+Plaintext Ciphertext
Pseudo-randomnumber generator
Key
Original plaintext+
– Key stream as long as message– Key stream = pseudo-random sequence (easy to
break)– Low security should be compensated by frequent
change of keys ⇒ necessity of secure channel⇒ 2 channels : one for the message & one for the key
Alain Bouffioux - Philips Leuven 123 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Symmetric & public-key cryptography(3)• Example of public-key cryptography
Alice encrypts messageusing Bob's public key
Public key of Bob
Encrypted message
Public Keys
Bob decrypts messageusing his secret key
Alain Bouffioux - Philips Leuven 124 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Symmetric & public-key cryptography(4)• Symmetric cryptography example : DES• Public-key cryptography example : RSA (1977)• Symmetric versus public-key cryptography
– Symmetric cryptography is faster (about 1000 times).– Low security of symmetric cryptography (due to the necessity
of key transport) is improved by a frequent change of the key.– In Public-key cryptography the secret-key may be kept secret.
It is never transported ⇒ High security.– Different usage : In DVB, symmetric key algorithm for
encrypting data, public-key algorithm for key management (secure channel).
• Hybrid cryptosystemExample : DES for message and RSA for key encryption
Alain Bouffioux - Philips Leuven 125 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Cryptography and DVB (1/2)• Cryptography prevents unauthorised receiver from
decoding the program.• DVB compared with banking or military secret
– high information rate– low information value– decryption must be cheap
• Cost of cracking the system should be higher than the benefits gained from the cracking
• Cryptography in DVB is a trade-off between cost/complexity versus piracy-proof.
• CA (Conditional Access) = very sensitive subject. Some service providers want their own CA system.
Alain Bouffioux - Philips Leuven 126 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Cryptography and DVB (2/2)• MPEG does not specify a conditional access (CA) system but
defines a frame to support CA.• DVB characterises some aspect left undefined by MPEG,
It defines a CA interface.• The broadcaster develops its CA system using a CA interface. • DVB is based on
– symmetric cryptography for audio-visual transmission– frequent key change to increase security– Public-key cryptography for key-exchange
• DVB relies on – stream of ECM’s (Entitlement Control Message)– stream of EMM’s (Entitlement Management Message)
Alain Bouffioux - Philips Leuven 127 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
CA information in MPEG TS (1/2)
Multiplexing
VideoEncoder
ECM'sPES
STC
EMM'sPES
SYSTEMLAYER
COMPRESSIONLAYER
Transport SublayerPES Sublayer
ComputePCR
VideoPES
Packetising
PTS
Sampling
Video elementarystream (E.S.)
Audio elementarystream (E.S.)
Program SpecificInformation (PSI,SI)
MPEG2 Transport Stream
Packetising
PTS
AudioPES
TRANSPORTSTREAM
PACKETISEDELEMENTARYSTREAM
ELEMENTARYSTREAM
AudioEncoder
Uncoded video
Sampling
Uncoded audio
Alain Bouffioux - Philips Leuven 128 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
CA information in MPEG TS (2/2)
Table
MPEG TSMultiplex
Program Association Table (PAT) (PID=0)
Program Map Table (PMT) (PID=x)
Conditional Access Table (CAT) (PID=1)
Some possible tables
Program Number 1 2 ......
Stream-type Audio Video PCR's ECM's
Stream-PID aa bb cc dd
EMM's mm
PMT-PID x y......
EMM's
PCR
ECM's
Payload184 bytes
Header4 bytes
PID
Video
Audio
Alain Bouffioux - Philips Leuven 129 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The CA mechanism : illustration
DecryptionEncrypted AV data Clear AV Data
SMARTCARD
Decryption
Decryption
ECM’s(Program related)
EMM’s(CA systemrelated)
IK
EntitlementSK
Access control parameters
SK
CW’s
PDK1 PDK2PDK
Alain Bouffioux - Philips Leuven 130 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The CA mechanism (1/2)
ACP = Access ControlParameters.
EncryptionCW (Control Word)
SK (Service Key)
Base Key
MPEG TS(clear) Scrambler
ECM's
Encryption EMM's
AV streams
Decryption
Decryption
ECM'sMPEG TS
EMM's
IK (Issuer Key)
AV streams
ACP
PDK (Programmer Distribution Key)
SKEntitlementPDK,SK
Entitlement,PDK, SK
DescramblerMPEG TS(clear)
CW
• AV streams are scrambled with Control Words (CW) using symmetric cryptography
• CW are encrypted using Service Keys (SK), are placed in ECM’s and are securely transmitted to the receiver
Alain Bouffioux - Philips Leuven 131 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
The CA mechanism (2/2)• SK are encrypted using public-key cryptography -
Keys are IK (unique key internal to the smartcard) or PDK (transmitted via EMM’s in order to define user’s group)
• ECM’s carries (informations related to a single program ⇒ PID of ECM’s in PMT) – enciphered CW– access parameters
• ECM’s are decoded to CW if the receiver contains the required entitlements
• EMM’s carries (information related to a conditional access system ⇒ PID of EMM’s in CAT)– New entitlements, SK’s (Service Keys)– Programmer distribution key
Alain Bouffioux - Philips Leuven 132 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
About DVB scrambling• Encryption occurs after compression (at the location in
the stream where the redundancy is at its lowest value) in order to have a robust encryption system.
• Encryption may occur at PES level or at TS level.• DVB scrambling is transparent (a valid TS remains
valid after scrambling) ⇒ facilitates transport and manipulation.
• Synchronisation based on PCR’s ⇒ constant time required for scrambling/descrambling.
• Security device should authenticate EMM’s origin.• CA is only one aspects of cryptography usage in
DVB. An other may be copy protection by (watermarking) and authentication (by signature).
Alain Bouffioux - Philips Leuven 133 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Agenda
Some video format
Alain Bouffioux - Philips Leuven 134 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Some video formats (1)
• Max. component video signal bandwidth: 6 MHz.• CCIR601 (CCIR is now ITU-R): Video sampling
frequency: 13.5 MHz for 525 & 625 line standards(Shannon requirement)
• Synchronous with line (& image) sampling frequencyFsampling= 864*Fh for 625 line system (50Hz countries) Fsampling= 858*Fh for 525 line system (60Hz countries)
• Why synchronous? Points at the same place• RGB format
Alain Bouffioux - Philips Leuven 135 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Some video formats (2)• YCbCr format
Cb = B-Y, Cr = R-YEye is more sensitive to luminance than to chrominance (lower resolution needed for chrominance)
Red
BlueGreen
MatrixLPF
Y
Cb
Cr
ModulatorSubC
+Composite
Video
Alain Bouffioux - Philips Leuven 136 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Some video formats (3)• The 4:2:2 format
– Y sampling @ 13.5 MHz– C sampling @ 6.75 MHz– 8 bits per pixel– 720 active points per line– 576 lines active lines per image (2 fields) (625 lines)
and 480 active lines (525 lines) – Pixels are not square (e.g. for 480 lines, only 640 active
points are needed - VGA format)– Image size 720*576 or 720*480
• The 4:2:0 format– Vertical luminance resolution reduced by a factor 2
(average on two successive lines)
Alain Bouffioux - Philips Leuven 137 A/V compression & Consumer Products - ULGPhilips Digital Systems Laboratories 09/12/2004
Some video formats (4)• SIF format (Source Intermediate Format)
Half the vertical & horizontal resolution of 4:2:0For 50Hz countries:– Luminance: 360*288– Chrominance: 180*120
• CIF format (Common Intermediate Format)– Intermediate format used in videoconferencing
(communication between US & Europe)– resolution: 360*288 – Sampling frequency: 30 Hz
• QCIF (Quarter CIF)– Half the vertical & horizontal resolution of CIF.