© Copyright Khronos Group, 2006 - Page 1 Open Standards for Media Authoring and Acceleration Neil Trevett Vice President Embedded Content, NVIDIA President,

© Copyright Khronos Group, 2006 - Page 1

Open Standards for Media Open Standards for Media Authoring and AccelerationAuthoring and Acceleration

Neil TrevettVice President Embedded Content, NVIDIA

President, Khronos


Khronos - Creating Open StandardsKhronos - Creating Open Standards

Open MembershipAny company is welcome

Funded by membership dues - $5K / year

Open Standards Publicly available on web-site

Royalty-free

Khronos has a PROVEN reputation for the TIMELY creation of HIGH-QUALITY,

ROYALTY-FREE standards

Khronos has a PROVEN reputation for the TIMELY creation of HIGH-QUALITY,

ROYALTY-FREE standards

Open Standards for Media Authoring and Acceleration


Khronos Tech Talks AgendaKhronos Tech Talks Agenda

Khronos TechTalk Schedule - Siggraph 2006 - Wednesday August 2ndSession Time Finish DescriptionNeil Trevett, NVIIDA 10:00 10:30 Khornos Overview and OpenKODEDan Ginsburg, ATI 10:30 11:00 Advanced Rendering Techniques with OpenGL ES v1.1+Eisaku Ohbuchi, DMP 11:00 11:30 Samurai Enlightened: OpenGL ES 1.1 + Techniques and BeyondKristof Beets, Imagination 11:30 12:00 OpenGL ES Performance RecommendationsTom Olson, TI 12:00 12:25 Shaders Go Mobile: An Introduction to OpenGL ES 2.0Steve Wilkinson, TI 12:25 12:50 Now and Then: Moving from OpenGL ES 1.1 to ES 2.0Yaki Tebeka, Graohic Remedy 12:50 13:20 OpenGL ES debugging and profiling techniques using gDEBugger ESHans-Martin Will, Vincent 13:20 13:45 Open Source Efforts around OpenGL ES APIYoshio Kuniyoshi, Acrodea 13:45 14:15 User Interface and other mobile applications using OpenGL ESHwanyong Lee, Huone 14:15 15:00 OpenVG: How to Program advanced 2D vector graphicsKristof Beets, Imagination 15:00 15:15 OpenVG SDK for PowerVR MBX


Khronos News – SIGGRAPH 2006Khronos News – SIGGRAPH 2006OpenGL ARB has voted to pass control of OpenGL to KhronosOpenGL 2.1 specification released here at SIGGRAPHgDEBugger Academic Program – free OpenGL tools for academia

COLLADA gains strong Momentum since joining Khronos one year agoUsed by all major authoring tools as asset exchange standardAdopted as an import format by Google Earth

Defining a complete native API set for handheld media applicationsLike DirectX for cell phones – except cross-platform and an open standard

Gaining strong support from Wireless Operators – Vodafone joins Khronos

Khronos membership hits close to 120 – Apple, Google and Dell joinIntel and Samsung now have seats on Board of Promoters

Khronos now driving dynamic media standards for embedded and desktop


Khronos Dynamic Media EcosystemKhronos Dynamic Media Ecosystem

Embedded Media Acceleration APIs

2D/3D Vector 2D Streaming Media Enhanced Audio

“DirectX-like” set of native APIsIncludes mixed media acceleration and OS portability APIs

3D Authoring

Dynamic Media Authoring

Dynamic Media Authoring Standards

Cross platform 2D/3D

Cross-platform graphics authoring/acceleration

Ecosystem


Over 100 companies creating media authoring and acceleration standards


OpenGL in KhronosOpenGL in Khronos• Can synergize resources and outreach- Common Conformance tests, marketing and web-site, tool chains etc.

• API collaboration- OpenGL, OpenGL ES, COLLADA, OpenKODE (EGL and debugging), OpenVG

• OpenGL and OpenGL ES Working Groups will remain independent- Both groups will be able to make decisions that best serve their own markets- OpenGL Working Group for desktop graphics- OpenGL ES Working Group for embedded graphics

Next Generation architectural firepower

Market feedback on streamlining functionality

Momentum - hundreds of millions of OpenGL ES devices

Integrated Authoring and Effects Framework

Embedded Markets

Desktop Markets


OpenGL Roadmap SynergyOpenGL Roadmap Synergy• OpenGL is now foundation of coherent family of market-focused APIs- Strong roadmap synergy between OpenGL, OpenGL ES and COLLADA

• “OpenGL 3.0” could use OpenGL ES 2.0 design for lean and mean core- Add nexgen performance, shaders (geometry, sample, uniforms), tool integration, object model

• Both APIs can share same conformance test foundation- Significant recent investment in OpenGL ES conformance tests by Khronos

OpenGL 1.5/2.0Architectural Foundation

OpenGL ES 1.1/2.0Functional

Streamlining

“OpenGL 3.0”Next generation functionality starting with OpenGL ES 2.0 streamlined core

“OpenGL ES 3.0”Streamlining next

generation functionality for embedded markets

Inter-generational Roadmap Synergy


COLLADA = 3D Asset InterchangeCOLLADA = 3D Asset Interchange• COLLADA is designed to transport assets between applications- Binds diverse DCC and 3D processing tools into a production pipeline- Developers and tool makers build on each other's strengths without reinventing the wheel

• COLLADA is a XML database schema for 3D assets- COLLADA is an interchange format- COLLADA is an intermediate format- COLLADA is not a delivery format and not a scene graph

• COLLADA can be lossless- COLLADA enables interoperability between DCC tools- COLLADA can be a source format that retains all information- Even multiple versions of the same asset

• COLLADA is comprehensive- COLLADA can encode visual scenes – including shaders and physics- COLLADA can contain libraries of assets

• COLLADA is an open, archive-grade format that retains meta information- When your DCC tool upgrades, you keep your assets


COLLADA MomentumCOLLADA Momentum• COLLADA 1.4 – released in 1H06 - supports advanced effects- COLLADA FX – shader effects, COLLADA Physics - physics effects

• Already supported by all major tools and thousands of users- 3ds Max, Blender, DAZ|Studio, Feeling Viewer, FX Composer - Google Earth, Houdini, Maya, Sketchup, and XSI

• Google Earth v4 imports COLLADA - KML v2.1 imports COLLADA models such as buildings, monuments, and statues- Full support for textures and level of detailing

• Unreal Engine 3 using COLLADA- To enable assets to be imported from any authoring tool


COLLADA Milestones – What’s NextCOLLADA Milestones – What’s Next

• Adding support for OpenGL ES 2.0 shaders• Full shader FX framework for OpenGL and OpenGL ES• Richer asset types• Asset Conditioning – scaling assets to platform capabilities• Integration of audio and video assets• Run-time and delivery formats

2H04 1H05 2H05 1H06 2H06

COLLADA Becomes Khronos Working Group

COLLADA Adopted by Google as import format for Google Earth

COLLADA 1.0 released by Sony at SIGGRAPH

COLLADA 1.4 Released

1H07

First COLLADA Textbook

OpenGL ES 2.0 supported

Conformance Tests Released


Embedded API OverviewEmbedded API Overview


Mid-2005OpenVG 1.0 Spec release

Adoption of Khronos Embedded Adoption of Khronos Embedded APIsAPIs

Silicon Designs in progress

Rapid adoption when silicon ships

Widespread, cross-platform availability

Market adoption in media-accelerated handsets

Mid-2004OpenGL ES 1.1

Spec release

Beginning-2006OpenMax IL 1.0

Spec release

End-2006OpenSL ES 1.0

Spec release

We Are Here - OpenGL ES is widespread

- OpenVG is in rapid adoption- OpenMAX is being implemented- OpenSL ES is being designed

100%

2D/3

D

Vect

or 2

D

Stre

amin

g M

edia

Enha

nced

Aud

io

End-2007


OpenGL ES – Two Track StandardOpenGL ES – Two Track Standard• Two tracks - manage mobile graphics through programmable transition- With maximized portability and minimized platform costs

• OpenGL ES 2.0 ruthlessly eliminates redundancy – just like 1.X- Deprecates all fixed functionality that can be replaced by shaders- Significant reduction in engine cost and driver complexity

• Platforms can ship either or both 1.X and 2.X libraries- Cheaper, more flexible than one large driver with both fixed and programmable functions- With full backwards compatibility maintained in each track

• OpenGL ES 2.X does NOT replace OpenGL ES 1.X- Will always need lowest cost, non-programmable hardware for certain high-volume devices

OpenGL ES 1.X – Fixed Function AccelerationOpenGL ES 1.1- For software and fixed functionality hardware- All 1.X specifications are backwards compatible

OpenGL ES 2.X – Programmable AccelerationOpenGL ES 2.0- Vertex & pixel shaders through GLSL ES shading language- All 2.X specifications will be backwards compatible


OpenGL ES 2.0 StatusOpenGL ES 2.0 Status• OpenGL ES 2.0 draft specification released at SIGGRAPH 2005- Final specification planned for 4Q06

• Making sure the standard is rock solid when released- Conformance tests will ship with the final specification- Requiring two working implementations to shake out the spec

• Raising the bar for OES extensions- Require conformance test before promotion to OES status- Require one working implementation


OpenVG – Accelerated Vector GraphicsOpenVG – Accelerated Vector Graphics• OpenVG ACCELERATES existing formats – such as Flash- NOT a competitor to Flash, SVG etc.- Enables popular vector formats to run with faster performance and less power

• OpenVG 1.0 released at Siggraph 2005- Open, royalty-free standard- Developed in just 12 months

• Uses OpenGL-style syntax- Easy to learn for OpenGL developers

• Can be supported by EXISTING silicon- Dedicated 2D engines AND full 3D GPUs

Applications

2D Hardware Acceleration

SVG/Flash/Font Packages etc..


OpenVG TimelineOpenVG Timeline

• OpenVG 1.1 is being defined now- Accelerated text rendering– in-driver glyph positioning- Flash-compatible rendering – compound shapes, additional blending modes

• OpenVG 2.0- Shaders?

3Q05 4Q05 1Q06 2Q06 1Q07

OpenVG 1.0 ratified in August,2005

OpenVG 1.0.1 to be ratified mid-06 – clarifications only. Conformance tests released

Target OpenVG 1.1 Release

4Q063Q06


Solving an Audio CrisisSolving an Audio Crisis• Many different proprietary audio APIs- Even playing a simple sound is different on different platforms

• No standard way to access available audio acceleration hardware- Significant source fragmentation for developers to re-write code for every platform

• Industry needs a unified, low-level audio acceleration API- Handheld target devices: mobile phones, personal media players & handheld gaming consoles

• Cross-platform foundation for a wide range of higher-level audio APIs- OpenAL, Java sound APIs (JSR-135 and JSR-234)

Applications

Audio Hardware

High-level Audio Libraries


OpenSL ES CapabilitiesOpenSL ES Capabilities• Low-latency, interactive sound generation for games- Advanced audio functionality: 3D positional audio, reverb effects, SP-MIDI

• Streaming playback of audio files- Mandated audio format across platforms providing consistency for developers

• Interactive music and ring-tones- SP-MIDI, Mobile DLS, Mobile XMF

• Effects and Playback Controls- Music and media player effects- Advanced environmental effects for gaming

Digital Audio

SP-MIDI

3D Audio

Effects

Controls

CODECs


Working Group and Milestones Working Group and Milestones

4Q05 1Q06 2Q06 3Q06 4Q06

Draft Specification External ISV review

First face-to-face meeting

Requirements finalized

1Q07

OpenSL ES 1.0 Public release


OpenMAX - Three Layer SolutionOpenMAX - Three Layer Solution

”Application Level”Media Application PortabilityApplications programmed using

cross-vendor interfaces

“Integration Level”Media Graph Portability

Integrate media networks using standard interconnect protocols

“Development Level” Media Component Portability

Develop portable media components using low-level media APIs

More media applications available on more platforms

for more end-user value!

Component vendors can ship more advanced functionality

across more processors. Media silicon vendors reduce

costs and time to market

Portable and powerful media processing graphs can

flexibly leverage available platform media componentsMedia Infrastructure

PortabilityOpen, royalty-free standard

IL

DL

AL

OpenMAX defines three holistically designed media open standards to provide complete media infrastructure portability


OpenMAX Milestone OverviewOpenMAX Milestone Overview• Created with strong industry consensus and participation

- ARM, ATI, Beatnik, Broadcom, Emuzed, Fraunhofer, Freescale, Infineon, Intel, Motorola, Nokia, NVIDIA, Philips, SKY MobileMedia, Samsung, Sasken, Siemens, STMicroelectronics, Symbian, Texas Instruments

• ST Microelectronics “Bellagio” OpenMAX IL released on Source Forge- Includes GStreamer plug-ins

4Q05 1Q06 2Q06 3Q06 4Q06

OpenMAX IL 1.0 Specification and Conformance Process Released

OpenMAX DL 1.0 Specification Released

OpenMAX IL 1.0 Linux Sample Implementation on Source Forge

OpenMAX DL/IL 1.0 Conformance Tests to be released

OpenMAX IL 1.1Target Date

1Q07

OpenMAX AL 1.0 Specification Target Release Date


Its like DirectX™ for mobile phones!Except its an open standard, cross-platform, royalty-free and streamlined for handheld devices

is a set of C-native APIs for handheld games and media applications

2D, 3D, video and audio media types are all seamlessly accelerated by

minimizes source changes when porting games and applications from phone to phone


3D Game Engines

Media Accelerator SiliconGPU / DSPs / CPUs

TV/Video/Audio Players

Kernel Operating System (e.g. Rex, Symbian OS, Windows Mobile, Linux, Nucleus, Synergy)

CPU

User Interface

NativeApplications

Flash/SVG Players

Software Platform (e.g. Brew, Symbian UIQ, S60, WIPI)

JavaApplications

Java JVM

Handheld devices will need advanced

graphics and media acceleration for user

interfaces, 3D gaming, Flash, TV…

Different OS APIs on every handset

Bindings

Proprietary media APIs – not tested for mixed-media operation BUT

Severe API fragmentation to access OS resources

and media acceleration = HUNDREDS of source

variants per game

Before OpenKODEBefore OpenKODE


3D Game Engines

Media Accelerator SiliconGPU / DSPs / CPUs

TV/Video/Audio Players

Kernel Operating System (e.g. Rex, Symbian OS, Windows Mobile, Linux, Nucleus, Synergy)

CPU

User Interface

NativeApplications

Flash/SVG Players

Software Platform (e.g. Brew, Symbian UIQ, S60, WIPI)

JavaApplications

Java JVM

Bindings

The OpenKODE SolutionThe OpenKODE Solution

SystemAbstraction

APIs

Khronos media APIs for state-of-the-art media acceleration

2D/3D Vector 2D Streaming Media Enhanced AudioMediaAPIs

Robust Trans-API Mixed Media Acceleration

=

+

Trans-API data and event coordination – WITH full trans-API Conformance Tests

+

System Abstraction APIs for portable

access to operating system resources, input devices and

displays

=

A coherent, cross-platform API set for

NATIVE portable media applications


Will result in advanced, multi-threaded, rich-media experiences

Coming to a handset near you!

OpenKODE 1.0 specification is due for public release in early

2007


Khronos Events Here at SIGGRAPHKhronos Events Here at SIGGRAPH

• OpenGL ES BOF- Thursday 3rd August 10AM-12PM Room 251

• COLLADA BOF & Social Event - Wednesday 2nd August 6-8PM Room 206A

• Tech Talk: COLLADA- Thursday 3rd August 12-2PM Room 251

• OpenGL BOF - Wednesday 2nd August 4-6PM Room 206A

Documents

© Copyright Khronos Group, 2006 - Page 1 Open Standards for Media Authoring and Acceleration Neil Trevett Vice President Embedded Content, NVIDIA President,