View
234
Download
0
Category
Preview:
Citation preview
MPEG-4 streams
Comp-ress
Comp-ress
Comp-ress
Comp-ress
DE LIVERY
INTERFACE
(DAI)Comp-ress
Scene Des.
decomp-ress
decomp-ress
decomp-ress
decomp-ress
DE LIVERY
INTERFACE
(DAI)
decomp-ress
COMPOSITOR
Obejct-des
Synchronization of elementary streams with the scene
Synchronization of multiple elementary streams
Based on two well known concepts
_______ references – Convey the speed of the encoder clock
_________ stamps – Convey the time at which an event should happen
Time stamps and clock references are – defined in the _____________ model – conveyed on the ______ layer
System Decoder Model
Object-based compression and delivery
Comp-ress
Comp-ress
Comp-ress
Comp-ress
DE LIVERY
INTERFACE
(DAI)
Comp-ressScene Des.
decomp-ress
decomp-ress
decomp-ress
decomp-ress
DE LIVERY
INTERFACE
(DAI)
decomp-ress
COMPOSITOR
System Decoder Model
decoderCompositionMemory CM1
DecoderBuffer DB1D
E LIVERY
INTERFACE
(DAI)
COMPOSITOR
decoderCompositionMemory CM1
DecoderBuffer DB1
decoderCompositionMemory CM1
DecoderBuffer DB1
decoderCompositionMemory CMn
DecoderBuffer DBn
System Decoder Model
Ideal model of the decoder behavior – Instantaneous ______________
Incorporates the__________ model – Decoding & composition time
Manage decoder buffer resources – Useful for the _______ – Ignores delivery ______
Designed for a rate-controlled “______” scenario – Applicable also to flow-controlled “pull” scenario
Defines composition memory (CM) behavior – A random access memory to the current composition unit – CM resource management not implemented
So, how does synchronization work?
Synchronization of elementary streamswith time events in the scene description
How are time events handled in the scene description?
How is this related to time in the elementary streams?
Which time base is valid for the scene description?
Time line (“object time base”) for the scene
Scene description stream with time stamped BIFS access units Object descriptor stream with pointers to all other streams
Video stream with (decoding & composition) time stamps
Audio stream with (decoding & composition) time stamps
Alternate time line for audio and video
Where are time stamps carried?
The synchronization layer
Synchronization layer (short: sync layer or SL) SL packet = one packet of data consists of header and __________
Indicates boundaries of __________ units – AccessUnitStartFlag, AccessUnitEndFlag, AULength
Provides consistency checking for ____ packets Carries object _______ reference (OCR) stamps Carries decoding and composition time stamps
(DTS, CTS)
Elementary Stream Interface
Compression Layer
SL-Packetized stream
Sync Layer
Elementary stream Access Unit AU Access Unit AU
Elementary Stream Interface
Elementary stream Access Unit AU Access Unit AU
Elementary stream Access Unit Access Unit
Elementary stream AU AUAUAU AUAU
Elementary Stream Interface
Elementary stream Access Unit Access Unit
ENCODER
This guy knows about• Access unit _________ • Random access points • Desired __________ time • The _________ time
Elementary Stream Interface
Access Unit Access Unit
ENCODER
Interface may be hidden in the Encoder
Access Unit
How is the sync layer designed?
The sync layer design
SL Packet SL Packet SL Packet
Access units are conveyed in SL packets
The sync layer design
SL P SL PacketSL Packet SL PacketSL P SL P
Access units may use more than one SL packet
The sync layer design
SL packets have a header to encode the information conveyed through the ESI
SL packets that don’t start an AU have a smaller header
SL P SL PacketSL Packet SL PacketSL P SL P
How is the sync layer designed?
As flexible as possible to be suitable for – a wide range of ______________ – a wide range of different media __________s
Time stamps have – variable length – variable _______________
Same for clock reference (OCR) values – OCR may come via another stream
Alternative to time stamps exists for lower bitrate Indication of ______ time and ________ of units (accessUnitDuration, compositionUnitDuration)
How is the sync layer flexibility signaled?
SLConfigDescriptor within ES_Descriptor Determines all variable parameters Allows to select from predefined _________s Must be known to parse an SL-packetized
stream
SLConfigDescriptor syntax example
class SLConfigDescriptor {
uint (8) predefined;
if (predefined==0) {
bit(1) useAccessUnitStartFlag;
bit(1) useAccessUnitEndFlag;
bit(1) useRandomAccessPointFlag;
bit(1) usePaddingFlag;
bit(1) useTimeStampsFlag;
uint(32) timeStampResolution;
uint(32) OCRResolution;
uint(6) timeStampLength;
uint(6) OCRLength;
if (!useTimeStamps) {
................
SDL-Syntax Description
Language
How to ship the SL packets?
Wrap SL packets in a suitable layer!
SL Packet SL Packet SL Packet
SL Packet SL Packet SL Packet
SL Packet SL Packet SL Packet
e.g. FlexMux
e.g. RTP/UDP/IP
e.g. MPEG-2 TS
MPEG-4 stream delivery (a sneak preview to DMIF ;-)
MPEG-4 delivery framework
The delivery layer
The “delivery layer” is an abstraction to refer generically to – any ________ multiplex (a. k. a. file format) – any transmission __________ – and the associated control functionality
The DMIF (Delivery Multimedia Integration Framework) part of MPEG-4 describes the _________ to any multiplex (DAI - DMIF Application Interface)
Adaptations of MPEG-4 to, e.g., RTP, MPEG-2, H.223 are being defined by the responsible group (with MPEG help)
Adaptation of the sync layer syntax
“Wrapping” of SL packets was the original design goal
MPEG-2 TS adaptation (Amendment 7 to MPEG-2 Systems)and FlexMux encapsulation really work like that
RTP encapsulation still in development – “Payload formats” specify the adaptation – They may use “Header compression”, i.e., – Syntax elements of SL are replaced by their RTP counterparts
e.g., composition time stamp --> RTP time stamp The MPEG-4 file format stores elementary streams
– conversion to SL-packetized streams during playout (if needed)
Multiplex of elementary streams
Multiplex of elementary streams
Not a core MPEG task
Just respond to specific needs for MPEG-4 content ______________
– Low delay – Low overhead – Low complexity
This prompted the design of the “___________” tool
One ___________ file format desirable – This lead to the design of the MPEG-4 file format
The MPEG-4 content access procedure
MPEG-4 content access procedure
Locate an MPEG-4 content item (e.g. by URL) and connect to it
How? – Via the _______________________– Reminder: This abstraction is being
mapped to actual signaling protocols (RTSP, MPEG-2 PSI, etc.)
Retrieve the _____________________ This Object Descriptor points to _____ +
_____________– Open these streams via DAI
The Scene Description points to other streams through __________ Descriptors
– Open the required streams via DAI Start playing!
A second attempt – more graphics
Recommended