Upload
noel-bennett-perkins
View
216
Download
0
Embed Size (px)
Citation preview
1
RUSProjects Communication Systems
BeWü Development
RTP Payload for MPEG-4 Payload for MPEG-4
with Flexible Error Resiliencywith Flexible Error Resiliency draft-ietf-avt-mpeg4streams-00.txt
proposed experimental RFC
formerly draft-guillemot-avt-genrtp-03.txt
RTP Payload for MPEG-4 Payload for MPEG-4
with Flexible Error Resiliencywith Flexible Error Resiliency draft-ietf-avt-mpeg4streams-00.txt
proposed experimental RFC
formerly draft-guillemot-avt-genrtp-03.txt Christine Guillemot, Paul Christ, Stefan Wesner [, Anders Klemets]
2
RUSProjects Communication Systems
BeWü Development
draft-guillemot-avt-genrtp-03.txt => new title Flexmux section added in discussion with authors - (see draft-
rgcc-flexmuxmpeg4-00.txt);
Payload Type (PT): Different payload types should be assigned for MPEG4 ES, MPEG4 SL-PDU MPEG-4 FlexMux streams. A payload type in the dynamic range should be chosen.
Same format, same interface ES, SL, Flexmux
TSOFFSET removed, Ebits included
draft-guillemot-avt-genrtp-03.txt => new title Flexmux section added in discussion with authors - (see draft-
rgcc-flexmuxmpeg4-00.txt);
Payload Type (PT): Different payload types should be assigned for MPEG4 ES, MPEG4 SL-PDU MPEG-4 FlexMux streams. A payload type in the dynamic range should be chosen.
Same format, same interface ES, SL, Flexmux
TSOFFSET removed, Ebits included
Modifications since WashingtonModifications since Washington
3
RUSProjects Communication Systems
BeWü Development
Multiple Dimensions of MPEG-4/RTPMultiple Dimensions of MPEG-4/RTP
Systems (OD) or non-systems (non-OD) framework
Various types of streams with different QoS requirements (MPEG-4 A/V, OD, BIFS, IPMP, other compressed AV
formats, eg. H.263 streams)
Mixed live and streamed (pre-encoded) content
Session Types Client - Server (unicast), “MU” (Multi) Peer-to-Peer - (multicast)
Systems (OD) or non-systems (non-OD) framework
Various types of streams with different QoS requirements (MPEG-4 A/V, OD, BIFS, IPMP, other compressed AV
formats, eg. H.263 streams)
Mixed live and streamed (pre-encoded) content
Session Types Client - Server (unicast), “MU” (Multi) Peer-to-Peer - (multicast)
... in applications relying on
4
RUSProjects Communication Systems
BeWü Development
Unified solution for the transport of MPEG-4 MPEG-4 SL packet streams - and for MPEG-4 ES
Common solution for both live and pre- encoded content
Applicable to the transport of FlexMux PD (submitted draft-rgcc-avt-mpeg4flexmux-00.txt)
Unified solution for the transport of MPEG-4 MPEG-4 SL packet streams - and for MPEG-4 ES
Common solution for both live and pre- encoded content
Applicable to the transport of FlexMux PD (submitted draft-rgcc-avt-mpeg4flexmux-00.txt)
Summary Summary
5
RUSProjects Communication Systems
BeWü Development
SummarySummary Abstraction of
QoS monitoring and Congestion Contro “intelligence” unique interface architectural simplicity and consistent stream “adaptation” in high number of streams applications (mixing real-time & pre-encoded content)
Packet Loss flexible protection mechanisms Full and partial AUs Segments with types or priority
Abstract media idiosyncrasies Assuming a media and network aware adaptation layer Towards UEP and/or Diffserv marking based on AUs
Abstraction of QoS monitoring and Congestion Contro “intelligence” unique interface architectural simplicity and consistent stream “adaptation” in high number of streams applications (mixing real-time & pre-encoded content)
Packet Loss flexible protection mechanisms Full and partial AUs Segments with types or priority
Abstract media idiosyncrasies Assuming a media and network aware adaptation layer Towards UEP and/or Diffserv marking based on AUs
6
RUSProjects Communication Systems
BeWü Development
Sample RTP packet for MPEG-4 ESsSample RTP packet for MPEG-4 ESs
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0|0| Res | | +-+-+-+-+-+-+-+-+ | | . . AU or partial AU . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
If no additional data-based protection needed
- One byte difference with draft-ietf-avt-mpeg4-es-00.txt,Byte offering additional flexibility in terms of protection- The packet would be of this form if protection supported by compression layer in the case of real-time
7
RUSProjects Communication Systems
BeWü Development
Sample RTP packet for MPEG-4 ESsSample RTP packet for MPEG-4 ESs
If additional data-based protection needed (eg. duplicatedheaders), (no fragmentation used).
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |G|E| XT | LENGTH |EBITS| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | . + Extension data (e.g. VOP header) +-+-+-+-+-+-+-+-+ . |G|E|0| res | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | . . Media Payload | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
8
RUSProjects Communication Systems
BeWü Development
Sample RTP packet for MPEG-4 SL-PDUsSample RTP packet for MPEG-4 SL-PDUs
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |G|E|F| res | optional SL header paramaters as indicated by . +-+-+-+-+-+-+-+-+ the SLConfigDescriptor . | . . Media payload | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
If no additional data-based protection needed
9
RUSProjects Communication Systems
BeWü Development
Sample RTP packet for MPEG-4 SL-PDUsSample RTP packet for MPEG-4 SL-PDUs
If additional protection needed
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |G|E| XT | LENGTH |EBITS| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | . . Extension data . | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |G|E|F| res | optional SL header paramaters as indicated by | +-+-+-+-+-+-+-+-+ the SLConfigDescriptor . | . . Media payload | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
10
RUSProjects Communication Systems
BeWü Development
Usage of the format for MPEG-4 Flexmux-PDUs in draft-rgcc-avt-mpeg4flexmux-00.txt
Usage of the format for MPEG-4 Flexmux-PDUs in draft-rgcc-avt-mpeg4flexmux-00.txt« What is suggested here is to define objects in the RTP payload, asit is proposed in draft-ietf-avt-mpeg4streams-01.txt. The RTP payload being a succession of ob-jects. Each object is byte aligned. Each object starting with its length. »
« A payload object is either a protection data object, or a completeFlexMux packet. Each payload object follows an identification byte,its object type byte. The RTP payload starts with one object typebyte. »
« The object type byte syntax: 0 1 2 3 4 5 6 7 +-+-+-+-+-+-+-+ |G|E| XT | +-+-+-+-+-+-+-+ »
11
RUSProjects Communication Systems
BeWü Development
MPEG-4/RTP recent historyMPEG-4/RTP recent history 12/98 Orlando,
draft-ietf-avt-rtp-mpeg4-00.txt, SL-packetized streams draft-guillemot-avt-genrtp-00.txt, ES streams + flexible additional
protection (if needed, eg. for streamed content)
04/99 NY + 07/99 Oslo joint IETF/MPEG (phone) meetings “No non-SL- and non-Systems ES RTP-mapping needed”: “2 experimental RFCs ...”
10/99 ISO Melbourne JNB non-System AV ES-mapping proposed
12/99 Washington draft-jnb-mpeg4av-rtp-00.txt „the normative way on how MPEG-4 Audio/Visual streams ... mapped ... to RTP“
12/98 Orlando, draft-ietf-avt-rtp-mpeg4-00.txt, SL-packetized streams draft-guillemot-avt-genrtp-00.txt, ES streams + flexible additional
protection (if needed, eg. for streamed content)
04/99 NY + 07/99 Oslo joint IETF/MPEG (phone) meetings “No non-SL- and non-Systems ES RTP-mapping needed”: “2 experimental RFCs ...”
10/99 ISO Melbourne JNB non-System AV ES-mapping proposed
12/99 Washington draft-jnb-mpeg4av-rtp-00.txt „the normative way on how MPEG-4 Audio/Visual streams ... mapped ... to RTP“
12
RUSProjects Communication Systems
BeWü Development
MPEG-4/RTP recent historyMPEG-4/RTP recent history 03/2000 Adelaide,
draft-rgcc-avt-flexmuxmpeg4-00.txt, flexmux-packetized streams -
– withdrawn last week in the Netherlands ??
.... more to come?
03/2000 Adelaide,
draft-rgcc-avt-flexmuxmpeg4-00.txt, flexmux-packetized streams -
– withdrawn last week in the Netherlands ??
.... more to come?
13
RUSProjects Communication Systems
BeWü Development
Implementation statusImplementation status
Mapping/de-mapping completed Integrating into IM1 as a DMIF plug-in (with
CSELT) is ongoing FEC implemented: duplicated headers, parity codes Adaptivity for QoS monitoring under development R-S under development
Mapping/de-mapping completed Integrating into IM1 as a DMIF plug-in (with
CSELT) is ongoing FEC implemented: duplicated headers, parity codes Adaptivity for QoS monitoring under development R-S under development
14
RUSProjects Communication Systems
BeWü Development
Information - DistributionInformation - Distribution current code documentation
http://www-ks.rus.uni-stuttgart.de/PROJ/GP ACTS COMIQS project ended february 2000
http://www.ccett.fr/comiqs/welcome.htm Used in 1 National Project (F, RNRT-VISI) Continue in 2 European Projects (IST-SONG, IST-
OPENISE) Distributed to 4 Companies outside the above
projects
current code documentation http://www-ks.rus.uni-stuttgart.de/PROJ/GP
ACTS COMIQS project ended february 2000 http://www.ccett.fr/comiqs/welcome.htm
Used in 1 National Project (F, RNRT-VISI) Continue in 2 European Projects (IST-SONG, IST-
OPENISE) Distributed to 4 Companies outside the above
projects
15
RUSProjects Communication Systems
BeWü Development
ConclusionConclusion Complexity of (future) MPEG-4 usage
Multiple streams (live and pre-encoded) with different QoS requirements so far, multiple ways for handling protection in the different ESs (HEC specific to visual)
Stresses the need, for the sake of architectural « simplicity »,
simple, single interface, avoiding parsers of the ESs
draft-ietf-avt-mpeg4streams-01.txt has the potential for an homogeneous way of handling efficient transport of the different types of MPEG-4 streams
Complexity of (future) MPEG-4 usage Multiple streams (live and pre-encoded) with different QoS requirements so far, multiple ways for handling protection in the different ESs (HEC specific to visual)
Stresses the need, for the sake of architectural « simplicity »,
simple, single interface, avoiding parsers of the ESs
draft-ietf-avt-mpeg4streams-01.txt has the potential for an homogeneous way of handling efficient transport of the different types of MPEG-4 streams
16
RUSProjects Communication Systems
BeWü Development
Application Case of
draft-ietf-avt-mpeg4streams-01.txt
to MPEG-4 Visual streams
in draft-gc-avt-mpeg4visual-00.txt
Application Case of
draft-ietf-avt-mpeg4streams-01.txt
to MPEG-4 Visual streams
in draft-gc-avt-mpeg4visual-00.txt
Christine Guillemot, Paul Christ, Stefan Wesner
17
RUSProjects Communication Systems
BeWü Development
Application of drat-ietf-avt-mpeg4streams-01.txt to the transport of MPEG-4 visual streams
Common solution for both live and pre- encoded content
Common solution for the different profiles
Application of drat-ietf-avt-mpeg4streams-01.txt to the transport of MPEG-4 visual streams
Common solution for both live and pre- encoded content
Common solution for the different profiles
Summary Summary
18
RUSProjects Communication Systems
BeWü Development
Simple Visual Profile: efficient, error resilient coding of rectangular
video objects, suitable for applications on mobile networks.
Simple Scalable Profile: adds support for coding of temporal and spatial scalable objects, useful for applications which provide services at more than one level of quality due to bit-rate or decoder resource limitations.
Core Visual Profile: adds support for coding of arbitrary-shaped and temporally scalable objects, useful for applications such as those providing content-interactivity (Internet multimedia applications).
Simple Visual Profile: efficient, error resilient coding of rectangular
video objects, suitable for applications on mobile networks.
Simple Scalable Profile: adds support for coding of temporal and spatial scalable objects, useful for applications which provide services at more than one level of quality due to bit-rate or decoder resource limitations.
Core Visual Profile: adds support for coding of arbitrary-shaped and temporally scalable objects, useful for applications such as those providing content-interactivity (Internet multimedia applications).
Visual Profiles Visual Profiles
19
RUSProjects Communication Systems
BeWü Development
Main Visual Profile: adds support for coding of interlaced, semi-
transparent, and sprite objects to the Core Visual Profile; useful for
interactive and entertainment-quality broadcast and DVD applications.
N-Bit Visual Profile: adds support for coding video objects having
pixel-depths ranging from 4 to 12 bits to the Core Visual Profile.
…. More for synthetic and for natural/synthetic hybrid visual content
Main Visual Profile: adds support for coding of interlaced, semi-
transparent, and sprite objects to the Core Visual Profile; useful for
interactive and entertainment-quality broadcast and DVD applications.
N-Bit Visual Profile: adds support for coding video objects having
pixel-depths ranging from 4 to 12 bits to the Core Visual Profile.
…. More for synthetic and for natural/synthetic hybrid visual content
Visual Profiles (continued) Visual Profiles (continued)
20
RUSProjects Communication Systems
BeWü Development
Very elaborate and powerfull syntax.
HEC mechanism useable initially for protecting header parameters needed in the simple profile
Lately corrigendum (DCOR1) in order to cover header parameters for arbitrary shape objects
Not (yet?) for enhancement layers when scalability is used in simple scalable, core, main
Very elaborate and powerfull syntax.
HEC mechanism useable initially for protecting header parameters needed in the simple profile
Lately corrigendum (DCOR1) in order to cover header parameters for arbitrary shape objects
Not (yet?) for enhancement layers when scalability is used in simple scalable, core, main
Impact of Profiles on SyntaxImpact of Profiles on Syntax
21
RUSProjects Communication Systems
BeWü Development
Configuration information global configuration information refering to the whole group
of visual objects (visualobjectsequence()), object configuration information refering to a single visual
object (visualobject()) object layer configuration information (visualobjectlayer()) video object plane header (videoObjectPlane()) (for video)
Configuration information global configuration information refering to the whole group
of visual objects (visualobjectsequence()), object configuration information refering to a single visual
object (visualobject()) object layer configuration information (visualobjectlayer()) video object plane header (videoObjectPlane()) (for video)
Elements of syntaxElements of syntax
22
RUSProjects Communication Systems
BeWü Development
Two modes of transmission for visual object
sequence layer, visual object layer and video
object layer: separate mode (in containers provided by MPEG-4 system (Ods)
combined mode (as part of the ES)
video object plane header as part of ES
Two modes of transmission for visual object
sequence layer, visual object layer and video
object layer: separate mode (in containers provided by MPEG-4 system (Ods)
combined mode (as part of the ES)
video object plane header as part of ES
Transmission of key segmentsTransmission of key segments
23
RUSProjects Communication Systems
BeWü Development
To avoid to have to support parsers in streaming applications
Same way of handling protection for real-time and pre-encoded content
simple and same interface Esdata dataLength number of padding bits at the end (if needed) degradationPriority and/or segment type
(videoobjectlayer(), GOV header, video object plane header, video packet)
To avoid to have to support parsers in streaming applications
Same way of handling protection for real-time and pre-encoded content
simple and same interface Esdata dataLength number of padding bits at the end (if needed) degradationPriority and/or segment type
(videoobjectlayer(), GOV header, video object plane header, video packet)
Uniform way of handling protection of key segmentsUniform way of handling protection of key segments
24
RUSProjects Communication Systems
BeWü Development
Sample RTP packet for MPEG-4 visualstreamsSample RTP packet for MPEG-4 visualstreams
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |G|E| XT | LENGTH |EBITS| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | . + eg. Video Object Plane header +-+-+-+-+-+-+-+-+ . |G|E|0| res | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | . . Media Payload | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+