MPEG-4MPEG-4in a nutshellin a nutshell

[Howard Stateman][Howard Stateman][[AV TeamAV Team]]

10/13/2003 MPEG-4 in a nutshell 2

MPEG-4 MissionMPEG-4 Mission

InternetInternet WirelessWireless

PDAsPDAs Cell PhonesCell Phones

Lower data ratesLower data ratesDramatically smaller file size Dramatically smaller file size (~45%)(~45%)

Deliver DVD-quality Deliver DVD-quality

Focus is on streaming mediaFocus is on streaming media

10/13/2003 MPEG-4 in a nutshell 3

MP3 vs. MPEG-xMP3 vs. MPEG-x MP3 is MP3 is notnot MPEG-3 MPEG-3

MP3 is the audio part of MPEG-1 and MP3 is the audio part of MPEG-1 and MPEG-2MPEG-2

The MPEG group The MPEG group skippedskipped MPEG-3 MPEG-3–Went directly from MPEG-2 to MPEG-4Went directly from MPEG-2 to MPEG-4

Layer 3 of MPEG-1 and MPEG-2Layer 3 of MPEG-1 and MPEG-2a.k.a. MPEG Layer 3a.k.a. MPEG Layer 3

10/13/2003 MPEG-4 in a nutshell 4

OverviewOverviewMMoving oving PPicture icture EEngineering ngineering GGrouproup

More than 200 experts from 20 countriesMore than 200 experts from 20 countries Creates standards for digital coding of Creates standards for digital coding of

audio and video contentaudio and video content

MPEG-1 MPEG-1 MPEG-2 MPEG-2 MPEG-4 MPEG-4(No MPEG-3)(No MPEG-3)

10/13/2003 MPEG-4 in a nutshell 5

Time LineTime Line MPEG-1: MPEG-1: Proposed 1989, Adopted 1992Proposed 1989, Adopted 1992 MPEG-2: MPEG-2: Proposed 1991, Adopted 1994Proposed 1991, Adopted 1994 MPEG-3: MPEG-3: HDTV standard dropped in favor of MPEG-2HDTV standard dropped in favor of MPEG-2 MPEG-4: MPEG-4: Proposed 1995, Adopted 1998-99Proposed 1995, Adopted 1998-99

Committee still activeCommittee still activeMPEG-4, Part 10 adopted 10/23/2003MPEG-4, Part 10 adopted 10/23/2003

–Advanced Video Coding (a.k.a. MPEG-4 AVC H.264)Advanced Video Coding (a.k.a. MPEG-4 AVC H.264)Considering merging MP3 audio into MPEG-4 Considering merging MP3 audio into MPEG-4 standardstandard

10/13/2003 MPEG-4 in a nutshell 6

What’s in a Format?What’s in a Format? MPEG-1 (.mpg, .m1v, .m1a, .mp3)MPEG-1 (.mpg, .m1v, .m1a, .mp3)

<= 1.5 Mbps<= 1.5 Mbps VCD quality & sizeVCD quality & size

MPEG-2 (.mpg , .m2v, .m2a, .mp3)MPEG-2 (.mpg , .m2v, .m2a, .mp3).5 – 8Mbps.5 – 8MbpsDVD quality & sizeDVD quality & size

MPEG-4 (.mp4)MPEG-4 (.mp4)Modem to 3MbpsModem to 3MbpsStreaming media @ DVD qualityStreaming media @ DVD quality

10/13/2003 MPEG-4 in a nutshell 7

MPEG-4 ArchitectureMPEG-4 Architecture

Illustration by Fernando Pereira Instituto Superior Técnico - Instituto de Telecomunicações, Lisbon Portugal

10/13/2003 MPEG-4 in a nutshell 8

Scene DescriptionMPEG-4 ComponentsMPEG-4 Components

InteractivitySynchronization

Data Transport(Flexmux/Transmux)

Intellectual Property Protection & Management

MPEG-J(Java)

Audio•Speech•General•Synthetic Speech•Synthetic Audio

Video•Movies•Still images•Text•2D/3D Graphics•Face & Body Animation

File Format(QuickTime)

10/13/2003 MPEG-4 in a nutshell 9

The Six Parts The Six Parts of MPEG-4of MPEG-4

1. Systems2. Visual 3. Audio4. Conformance Testing5. Reference Software6. Delivery Multimedia Integration

Framework (DMIF)

10/13/2003 MPEG-4 in a nutshell 10

SystemsSystems Scene descriptionScene description MultiplexingMultiplexing SynchronizationSynchronization Buffer managementBuffer management Intellectual property management Intellectual property management

and protectionand protection

10/13/2003 MPEG-4 in a nutshell 11

VisualVisual Specifies the coded representation Specifies the coded representation

of natural and synthetic visual of natural and synthetic visual objects objects

10/13/2003 MPEG-4 in a nutshell 12

AudioAudio Specifies the coded representation Specifies the coded representation

of natural and synthetic audio of natural and synthetic audio objects objects

10/13/2003 MPEG-4 in a nutshell 13

Conformance TestingConformance Testing Defines conformance conditions for Defines conformance conditions for

bitstreams and devicesbitstreams and devices Used to test MPEG-4 Used to test MPEG-4

implementations implementations

10/13/2003 MPEG-4 in a nutshell 14

Reference SoftwareReference Software Software ToolsSoftware Tools

For implementing compliant productsFor implementing compliant products

10/13/2003 MPEG-4 in a nutshell 15

DMIF Delivery Multimedia Integration

Framework Defines a session protocol Used to manage multimedia

streaming over generic delivery technologies

Internet Wireless

10/13/2003 MPEG-4 in a nutshell 16

Making & Playing MPEG-4Making & Playing MPEG-4 Encoder software costs $$Encoder software costs $$

Many licensing issuesMany licensing issues Patents held by:Patents held by:

– MPEGLA (licensing pool)MPEGLA (licensing pool)– Dolby LabsDolby Labs– VIAVIA– ThomsonThomson

Player software is freePlayer software is free Envivio plug-in (WMP, ROP, QT)Envivio plug-in (WMP, ROP, QT) QuickTime 6.3QuickTime 6.3

10/13/2003 MPEG-4 in a nutshell 17

New MPEGs in ProgressNew MPEGs in Progress MPEG-7MPEG-7

Description and search of audio and Description and search of audio and visual contentvisual content

MPEG-21MPEG-21 "Multimedia Framework" "Multimedia Framework"

10/13/2003 MPEG-4 in a nutshell 18

Links to detailed Links to detailed information information

MPEG-4 is a very complex standard, encompassing a wide MPEG-4 is a very complex standard, encompassing a wide range of applications. Here are some links to more detailed range of applications. Here are some links to more detailed information:information:

Italy Telecom:http://mpeg.telecomitalialab.com/events&tutorials/mpeg-4_si/index.htmFraunhofer:http://www.iis.fraunhofer.de/amm/techinf/mpeg4/MPEG Committee:http://www.chiariglione.org/mpeg/standards/mpeg-4/mpeg-4.htmUC Berkeley:http://www.sims.berkeley.edu/courses/is224/s99/GroupG/report1.html#_Toc447982102

MPEG Committee Home Page:http://www.chiariglione.org/mpeg/index.htm

10/13/2003 MPEG-4 in a nutshell 19

Links to PlayersLinks to Players Envivio TV plug-inEnvivio TV plug-inhttp://www.envivio.com/products/etv/http://www.envivio.com/products/etv/

download.jspdownload.jsp

QuickTime 6.4 playerQuickTime 6.4 playerhttp://www.apple.com/quicktime/download/http://www.apple.com/quicktime/download/

standalone/standalone/

10/13/2003 MPEG-4 in a nutshell 20

Links to EncodersLinks to EncodersEnVivio EnVivio www.envivio.com/productswww.envivio.com/productsVanguard Software Vanguard Software www.vsofts.comwww.vsofts.comMoonlight Moonlight www.moonlight.co.ilwww.moonlight.co.ilPixelTools PixelTools www.pixeltools.comwww.pixeltools.comProVideoProVideo www.provideo.com.twwww.provideo.com.twDSP Research DSP Research www.dspr.comwww.dspr.comHikVision HikVision www.hikvision.com/enwww.hikvision.com/en

10/13/2003 MPEG-4 in a nutshell 21

MPEG-4 Mailing ListMPEG-4 Mailing ListList for MPEG-4, MPEG-7 and MPEG-21 List for MPEG-4, MPEG-7 and MPEG-21

non-technical discussions:non-technical discussions:

http://lists.mpegif.org/mailman/listinfo/discusshttp://lists.mpegif.org/mailman/listinfo/discuss

10/13/2003 MPEG-4 in a nutshell 22

The EndThe End

10/13/2003 MPEG-4 in a nutshell 23

Original goals - addendumOriginal goals - addendum The following 4 slides describe the The following 4 slides describe the

original goals of the MPEG-4 original goals of the MPEG-4 committeecommittee

Removed from the original Removed from the original presentation due to time presentation due to time considerationsconsiderations

10/13/2003 MPEG-4 in a nutshell 24

MPEG-4 Goals [1 of 4]MPEG-4 Goals [1 of 4] Efficiently represent a number of data Efficiently represent a number of data

types: types: Video from very low to very high bitratesVideo from very low to very high bitrates Music and speech across a wide bitrate Music and speech across a wide bitrate

rangerange Generic dynamic 3-D objects Generic dynamic 3-D objects

Specific objects such as human faces and bodiesSpecific objects such as human faces and bodies Support for 3-D audio spacesSupport for 3-D audio spaces

Text and graphicsText and graphics

10/13/2003 MPEG-4 in a nutshell 25

MPEG-4 Goals [2]MPEG-4 Goals [2] Error correction in the encoding Error correction in the encoding

layerlayer For all types of dataFor all types of data Emphasis on difficult conditionsEmphasis on difficult conditions

Mobile phones, palmtopsMobile phones, palmtops Object-oriented data handlingObject-oriented data handling

Each discrete object can be managed Each discrete object can be managed separatelyseparately

10/13/2003 MPEG-4 in a nutshell 26

MPEG-4 Goals [3]MPEG-4 Goals [3] Audiovisual scenes can combine:Audiovisual scenes can combine:

AudioAudio VideoVideo NaturalNatural

AnalogAnalog– MusicMusic– NatureNature– SpeechSpeech

SyntheticSynthetic DigitalDigital

– MIDIMIDI– Computer-generatedComputer-generated

10/13/2003 MPEG-4 in a nutshell 27

MPEG-4 Goals [4]MPEG-4 Goals [4] Interactive Interactive HyperlinkingHyperlinking Protect intellectual propertyProtect intellectual property Transparently cross the borders of Transparently cross the borders of

different delivery environments. different delivery environments.