Open Standards for Today’s Gaming Industry€¦ · Accelerating OpenGL Innovation DirectX 10.1 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 OpenGL 3.1 DirectX 9.0c DirectX 10.0 DirectX 11 OpenGL

© Copyright Khronos Group 2013 - Page 1

Open Standards for Today’s Gaming Industry

Erik Noreke VP of Business Development, Khronos Group


Gaming Evolution

New high performance games

continue to evolve on high-end platforms

Mobile is the new platform for game innovation.

Mobile APIs unlock hardware acceleration and

conserve battery life, making portable gaming

possible

Diverse platforms – mobile, TV, embedded

– means HTML5 will become

increasingly important as a universal

gaming platform

DESKTOP MOBILE


Courtesy of PC Gaming Alliance

www.pcgamingalliance.org







US$28.7B gaming revenues generated on platforms supporting

Khronos native graphics and compute APIs


OpenGL 3D API Family Tree

OpenGL ES 1.0

OpenGL ES 1.1 OpenGL ES 2.0 OpenGL ES 3.0

2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

OpenGL 1.5 OpenGL 2.0 OpenGL 4.3 OpenGL 2.1

OpenGL 3.0

OpenGL 3.1

OpenGL 3.2

OpenGL 3.3

OpenGL 4.0

OpenGL 4.1

OpenGL 4.2

2002

OpenGL 1.3

ES-Next

GL-Next

OpenGL ES 2.0

Content OpenGL ES 1.1

Content

OpenGL ES 3.0

Content

ES3 is backward compatible

so new features can be

added incrementally Fixed function

3D Pipeline

Programmable vertex

and fragment shaders

WebGL 1.0

OpenGL 4.4 is a

superset of DX11

WebGL-Next

Desktop 3D

Mobile 3D

OpenGL 4.4


Mobile OpenCL Shipping • Android ICD extension released in latest extension specification

- OpenCL implementations can be discovered and loaded as a shared object

• Multiple implementations shipping in Android NDK

- ARM, Imagination, Vivante, Qualcomm, Samsung …


Active Gamers by Platform



CAMERA


Sensors & Cameras • Single sensor RGB cameras are just the start of the mobile visual revolution

- IR sensors – LEAP Motion, eye-trackers

• Multi-sensors: Stereo pairs -> Plenoptic array -> Depth cameras

- Stereo pair can enable object scaling and enhanced depth extraction

• Hybrid visual sensing solutions

- Different sensors mixed for different distances and lighting conditions

Dual Camera LG Electronics

Plenoptic Array Pelican imaging

Capri Structured Light 3D Camera PrimeSense


Typical Imaging Pipeline • Pre- and Post-processing can be done on CPU, GPU, DSP…

• ISP controls camera via 3A algorithms

Auto Exposure (AE), Auto White Balance (AWB), Auto Focus (AF)

• ISP may be a separate chip or within Application Processor

Pre-processing Image Signal Processor

(ISP)

Post-

processing

CMOS sensor

Color Filter Array

Lens

Bayer RGB/YUV

App

Lens, sensor, aperture control 3A

Need for advanced

camera control API!

CAMERA


Low Power Environment Scanning • Many sensor use cases would consume too much power to be running 24/7

- Environment aware use cases have to be very low power

• ‘Scanners’ - very low power, always on, detect things in the environment

- Trigger the next level of processing capability

ARM 7 1 MIP and accelerometers can

detect someone in the vicinity

DSP Low power activation of camera

to detect someone in field of view

GPU GPU acceleration for precision

gesture processing


StreamInput Sensor Fusion Stack

OS Sensor OS APIs (E.g. Android SensorManager or

iOS CoreMotion)

Low-level native API defines access to

fused sensor data stream and context-awareness

…

Applications

Sensor Sensor

Sensor

Hub Sensor

Hub

StreamInput implementations

compete on sensor stream quality,

reduced power consumption,

environment triggering and context

detection – enabling sensor

subsystem vendors to increased

ADDED VALUE

Middleware (E.g. Augmented Reality engines,

gaming engines)

Platforms can provide

increased access to

improved sensor data stream

– driving faster, deeper

sensor usage by applications

Middleware engines need platform-

portable access to native, low-level

sensor data stream

Mobile or embedded

platforms without sensor

fusion APIs can provide

direct application access

to StreamInput

Hardware transport

interfaces are defined

by each system, e.g.

IIO or HID sensor


Worldwide Game Users – 2012 (Active and Casual)

0

100

200

300

400

500

N.America Europe Asia Other

WORLDWIDE GAME USERS BY REGION AND PLATFORM (IN MILLIONS)

Console PC Dedicated Handheld Mobile




C/C++

SDK Dalvik (Java)

Objective C C#

DirectX

HTML/CSS HTML/CSS HTML/CSS

Cross-OS, Cross Platform Portability

HTML5 provides cross

platform portability. GPU

accessibility through

WebGL available soon on

~90% mobile systems

Preferred development

environments not

designed for portability

Native code is portable-

but apps must cope with

different available APIs

and libraries


WebGL Availability in Browsers

- Microsoft – “where you have IE11, you have WebGL – turned on by default and working all the time” - Microsoft - WebGL also enabled for Windows applications - web app framework and web view - Apple - WebGL must be explicitly turned on MAC Safari and only exposed on iOS for iAds - Chrome OS - WebGL is the only cross-platform API to program the GPU - Google IO announcement - Chrome on Android will soon launch with WebGL

Much WebGL content uses three.js library:

http://threejs.org/

http://threejs.org/

http://threejs.org/


WebCL – Parallel Computing for the Web • JavaScript bindings to hardware acceleration

- Giving performance gaming in a web browser

http://www.youtube.com/user/SamsungSISA#p/a/u/1/9Ttux1A-Nuc

http://www.youtube.com/user/SamsungSISA



© Copyright Khronos Group 2013 - Page 18 Remember Me image courtesy of CapCom


3D Needs a Transmission Format! • Compression and streaming of 3D assets becoming essential

- Mobile and connected devices need access to increasingly large asset databases

• 3D is the last media type to define a compressed format

- 3D is more complex – diverse asset types and use cases

• Needs to be royalty-free

- Avoid an ‘internet video codec war’ scenario

• Eventually enable hardware implementations of successful codecs

- High-performance and low power – but pragmatic adoption strategy is key

Audio Video Images 3D

MP3 H.264 JPEG ? !

An effective and widely adopted codec ignites previously

unimagined opportunities for a media type


glTF Goals • Binary file format for efficient transmission for 3D assets

- Reduce network bandwidth and minimize client processing overhead

• Run-time neutral - DO NOT IMPLY OR MANDATE ANY RUN-TIME BEHAVIOR

- Can be used by any app or run-time – usually WebGL accelerated

• Scalable to handle compression and streaming

- Though baseline format does not include compression

• ‘Direct load efficiency’ for WebGL

- Little or NO processing to drop glTF data into WebGL client

• Carry conditioned data from any authoring format

- Prototyping and optimizing efficient handling of COLLADA assets

A standards-based

content pipeline for

rich native and Web 3D

applications Playback Authoring


Game On! • Gaming and Animation is a complex and expanding application domain and

coherent multiple standards across multiple domains are needed to enable the

market

• Khronos family of standards allow you to focus your efforts on game

development, not porting between platforms

• Now is a great time to get involved with the standards initiatives

that effect your business and game development

• www.khronos.org

• [email protected]

http://www.khronos.org/

mailto:[email protected]



WebGL First Wave Application Categories • Maps and Navigation

• Modeling Tools and Repositories

• Games

• 3D Printing

• Visualization

• Music Videos and Promotion

• Education

• Photo Editors

• Music Visualizers

• Vision/Video Processing


COLLADA and glTF Open Source Ecosystem

Tool Interop

Three.js glTF Importer. Rest3D initiative

COLLADA2GLTF

Translator

OpenCOLLADA

Importer/Exporter

and COLLADA

Conformance Tests

On GitHUB

Pervasive WebGL deployment

Other

authoring

formats

Web-based Tools

https://github.com/KhronosGroup/glTF

https://github.com/KhronosGroup/OpenCOLLADA

https://github.com/KhronosGroup/COLLADA-CTS

https://github.com/KhronosGroup/glTF

https://github.com/KhronosGroup/OpenCOLLADA





Google Maps • All rendering (2D and 3D) in Google Maps uses WebGL


WebCL – Parallel Computing for the Web • JavaScript bindings to OpenCL APIs

- Enables initiation of Kernels written in OpenCL C within the browser

• Bindings stay close to the OpenCL standard

- Maximum flexibility to provide a foundation for higher-level middleware

- Minimal language modifications for 100% security and app portability - E.g. Mapping of CL memory objects into host memory space is not supported

• API definition underway – public draft released - https://cvs.khronos.org/svn/repos/registry/trunk/public/webcl/spec/latest/index.html

• Compelling use cases

- Physics engines for WebGL games

- Image and video editing in browser River Trail

JavaScript binding to OpenCL APIs to enable initiation of OpenCL C kernels

Data-parallel language extensions to JavaScript

for offload at run-time

https://cvs.khronos.org/svn/repos/registry/trunk/public/webcl/spec/latest/index.html




Leveraging Proven Native APIs into HTML5 • Khronos and W3C liaison

- Leverage proven native API investments into the Web

- Fast API development and deployment

- Designed by the hardware community

- Familiar foundation reduces developer learning curve

Native APIs shipping

or Khronos working group

JavaScript API shipping,

acceleration being developed

or work underway

WebVX? Vision

Processing

WebCAM(!) Camera

control and

video

processing

Possible future

JavaScript APIs or

acceleration

WebStream? Sensor Fusion

Native

JavaScript Canvas

Path Rendering

Camera

Control

HTML


OpenGL ES 3.0 Highlights • Better looking, faster performing games and apps – at lower power

- Incorporates proven features from OpenGL 3.3 / 4.x

- 32-bit integers and floats in shader programs

- NPOT, 3D textures, depth textures, texture arrays

- Multiple Render Targets for deferred rendering, Occlusion Queries

- Instanced Rendering, Transform Feedback …

• Make life better for the programmer

- Tighter requirements for supported features to reduce implementation variability

• Backward compatible with OpenGL ES 2.0

- OpenGL ES 2.0 apps continue to run unmodified

• Standardized Texture Compression

- #1 developer request!


DirectX 11.1

2004 2006 2008 2009 2010 2005 2007 2011

Accelerating OpenGL Innovation

DirectX 10.1

OpenGL 2.0 OpenGL 2.1 OpenGL 3.0

OpenGL 3.1

DirectX 9.0c DirectX 10.0 DirectX 11

OpenGL 3.2

OpenGL 3.3/4.0

OpenGL 4.1

Bringing state-of-the-art functionality to cross-platform graphics

2012

OpenGL 4.2

OpenGL 4.4

2013

OpenGL 4.3


OpenGL 4.3 Compute Shaders • Execute algorithmically general-purpose GLSL shaders

- Can operate on uniforms, images and textures

• Process graphics data in the context of the graphics pipeline

- Easier than interoperating with a compute API IF processing ‘close to the pixel’

• Standard part of all OpenGL 4.3 implementations

- Matches DX11 DirectCompute functionality

Physics AI Simulation Ray Tracing Imaging Global Illumination


OpenGL Conformance Test Suite released!

Conformance submissions are required for GL 4.4 implementations encouraged for earlier driver versions

Shared codebase with OpenGL ES 3.0 CTS additional desktop-specific tests

Core profile functionality

Enhancements underway to add more coverage


OpenCL and OpenGL Compute Shaders • OpenGL compute shaders and OpenCL support distinctly different use cases

- OpenCL provides a significantly more powerful and complete compute solution

Enhanced 3D

Graphics apps

“Shaders++”

Pure compute

apps touching

no pixels

Compute Shaders

1. Full ANSI C programming of

heterogeneous CPUs and GPUs

2. Utilize multiple processors

3. Precisely defined IEEE accuracy

1. Fine grain compute operations

inside OpenGL

2. GLSL Shading Language

3. Execute on single GPU only

Imaging

Video

Physics

AI


Gaming Evolution

New high performance games still evolve on high-

end platforms

Mobile is the new platform for game innovation.

Mobile APIs unlock hardware acceleration and

conserve battery life, making portable gaming

possible

Diverse platforms – mobile, TV, embedded

– means HTML5 will become

increasingly important as a universal

gaming platform

DESKTOP MOBILE

Documents

Open Standards for Today’s Gaming Industry€¦ · Accelerating OpenGL Innovation DirectX 10.1 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 OpenGL 3.1 DirectX 9.0c DirectX 10.0 DirectX 11 OpenGL