Mobile 3D Graphics API M3G En

Mobile 3D Graphics API (M3G)Janne Levula

Forum Nokia

Phone screens may be flat…but the world certainly isn’t!

Why 3D Graphics?

Objective

Gain an insight to the Mobile 3D Graphics API and take your games and applications to the next level

What’s Good to Know!

• Fundamentals of 3D graphics• transformations, texturing, blending,

• OpenGL® or some other modern 3D API• OpenGL is the premier environment for developing interactive

2D and 3D graphics applications. • Introduction in 1992, today industry’s most widely used and

supported 2D and 3D graphics application programming interface.

• Java™ technology, preferably MIDP 2.0• GameCanvas

Agenda

Overview?

Getting started

Scene graph

Low level features

Dynamic meshes

Animation

Demo

New dimension of Java games

Agenda

Overview?

Getting started

Scene graph

Low level features

Dynamic meshes

Animation

Demo


Overview

• Java M3G API (JSR-184) provides a scalable, small-footprint, interactive 3D graphics Java API for J2ME™.

• For a wide range of applications, including games, animated messages, screen savers, custom user interfaces, product visualization…

• Java M3G API is an optional package that can be adopted to existing J2ME profiles. The main target platform of the API is J2ME/CLDC, used with profiles such as MIDP 1.0 or MIDP 2.0.

• JSR-184 was selected by the Java Community Process (JCP) as “The most innovative JSR” in year 2003.

Mobile 3D Graphics API (M3G)

• Also known as JSR-184 (Optional for J2ME)

• Designed for mobile devices• Primarily CLDC 1.1 / MIDP

• But also CDC

MIDlets

Mobile 3D Graphics API MIDP

Graphics Hardware Java Virtual Machine (CLDC 1.1)

The API Advantages• Benefits for developers

• Improve rendering speed by using native Libraries to leverage the underlying hardware Acceleration

• Mask hardware differences (3D HW, FPU, DSP,…). Exploit any new hardware automatically

• Makes easier application development

• Benefits for hardware vendors• Gives a concrete target which features to offer

Benchmarked on an ARM9 processor

Why a New Standard?• OpenGL (ES) is too low-level,

(OpenGL for Embedded Systems) is based on OpenGL)• Lots of Java code needed for simple things• Bigger MIDlets, slower speed

• Java 3D™ API is too bloated• A hundred times larger than M3G• Does not fit together with MIDP

• Now knew what we wanted!• Basic Java 3D™ ideas: nodes, scene graph• Add file format, keyframe animation• Remain compatible with OpenGL ES

Agenda

Overview?

Getting started

Scene graph

Low level features

Dynamic meshes

Animation

Demo


IntroM3G can be described as a unified immediate/retained mode API with a configurable rendering pipeline at the low level and an animated scene graph layered on top of it.

Flexibility is guaranteed by rich set of parameters for configuring the rendering pipeline. There are dozens of mutually independent parameters in the pixel pipeline alone, yielding billions of unique configurations and different visual effects that can be applied to polygons.

Key Classes

World

3D graphics context Performs all renderingGraphics3D

Can load individual objects and entire scene graphs (M3G and PNG files)

Loader

Scene graph root node

Graphics3D• Contains global state

• Rendering targets (Canvas, Image, Image2D, CustomItem)• Camera, Viewport, Light sources, Depth buffer, Rendering quality hints

• Rendering Modes:• Immediate mode

• Low level, allows the user to define every detail of rendering process• Render a branch or an individual node at a time• Explicitly give the Camera and Lights to Graphics3D

• Retained mode• Hides low level from the user allowing animated 3-D content to be

loaded and displayed with few lines of code• Render a scene graph, rooted by the World• Take the Camera and Lights from the World

Graphics3D• Immediate vs. retained mode

• Immediate mode• Render individual scene graph nodes or groups of nodes,

or individual submeshes

• Used lights, active camera, and background are provided separately

• Retained mode• Render an entire world represented as a scene graph

• Used lights, active camera, and background are members of the scene graph

• Difference between immediate and retained mode is vague as they can be freely mixed and matched

Graphics3D: How-to-Use• Bind a target to it, render, release the target• Tip: Everything is synchronous

• Tip: There are no callbacks

• Tip: Never mess with a bound target

• Tip: Graphics3D is a singleton (threads!)

void paint(Graphics g) {

myGraphics3D.bindTarget(g);

myGraphics3D.render(world);

myGraphics3D.releaseTarget();

}

Example: “Hello, World”

A simplified animation playerimport javax.microedition.midlet.MIDlet;import javax.microedition.lcdui.Display;import javax.microedition.lcdui.game.GameCanvas;import javax.microedition.m3g.*;

public class Player extends MIDlet{public void pauseApp() {}

public void destroyApp(boolean b) {}

public void startApp() {PlayerCanvas canvas = new PlayerCanvas(true);Display.getDisplay(this).setCurrent(canvas);try { canvas.run(); } catch (Exception e) {}notifyDestroyed();

}}

Example: “Hello, World”

class PlayerCanvas extends GameCanvas {PlayerCanvas(boolean suppress){super(suppress);}

public void run() throws Exception {Graphics3D g3d = Graphics3D.getInstance();World w = (World) Loader.load("/file.m3g")[0];long start, elapsed, time = 0;while (getKeyStates() == 0) {start = System.currentTimeMillis();g3d.bindTarget(getGraphics());try {w.animate(time);g3d.render(w);

} finally { g3d.releaseTarget(); }flushGraphics();elapsed = System.currentTimeMillis()-start;time += (elapsed < 100) ? 100 : (int)elapsed;if (elapsed < 100) Thread.sleep(100-elapsed);

}}

}

Example: Rotating CubeDefine raw data for a cube// Corners of a cube as (X,Y,Z) tripletsstatic short[] cubeVertices = {-1, -1, 1, 1, -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, 1, -1

};

// A color for each corner as an (R,G,B) tripletstatic byte[] cubeColors = {(byte) 255, (byte) 0, (byte) 0, // red…(byte) 0, (byte) 255, (byte) 0, // green

};

// Define the cube as a single triangle stripstatic int[] indices = { 6,7,4,5,1,7,3,6,2,4,0,1,2,3 };static int[] stripLen = { 14 };

Example: Rotating CubeCopy raw data into Objects// Fill in VertexArrays from short[] and byte[]int numVertices = vertices.length/3;VertexArray pos = new VertexArray(numVertices, 3, 2);VertexArray col = new VertexArray(numVertices, 3, 1);pos.set(0, numVertices, cubeVertices);col.set(0, numVertices, cubeColors);

// Attach the VertexArrays to a VertexBuffer// Note the scale (1.0) and bias (0,0,0)vertices = new VertexBuffer();vertices.setPositions(pos, 1.0f, null);vertices.setColors(col);

// Fill in the triangle striptriangles = new TriangleStripArray(cubeIndices, stripLen);

// Create a Mesh with default Appearancecube = new Mesh(vertices, triangles, new Appearance());

Example: Rotating CubeSet up a CameraCamera cam = new Camera();

// 60-degree field of view, screen aspect ratio// Near clipping plane at 1.0, far plane at 1000float aspect = (float) getWidth() / (float) getHeight();cam.setPerspective(60.0f, aspect, 1.0f, 1000.0f);

// Place the camera at z=5.0 in world space// View vector is along the negative Z axistransform = new Transform();transform.postTranslate(0.0f, 0.0f, 5.0f);g3d = Graphics3D.getInstance();g3d.setCamera(cam, transform);

Example: Rotating Cube

Clear the buffers, render, animateg3d = Graphics3D.getInstance();g3d.bindTarget(getGraphics());try {g3d.clear(null);g3d.render(cube, transform);

} finally { g3d.releaseTarget(); }

xAngle += 1; yAngle += 2; zAngle += 3;transform.setIdentity();transform.postRotate(xAngle, 1.0f, 0.0f, 0.0f);transform.postRotate(yAngle, 0.0f, 1.0f, 0.0f);transform.postRotate(zAngle, 0.0f, 0.0f, 1.0f);

Agenda

Overview?

Getting started

Scene graph

Low level features

Dynamic meshes

Animation

Demo


The Scene GraphActually, it’s just a tree

World

SkinnedMesh

Group

Group Camera

Group MorphingMesh

Group

Group Mesh

Sprite3D

Light

Not allowed!

Scene Graph: Nodes• The Scene graph itself is composed by Node objects:

• Camera

• Mesh: SkinnedMesh and MorphingMesh

• Sprite 3D

• Light

• Node is the type of object commonly manipulated in an application:

• translations, rotations, scaling, transformation

• Nodes can be grouped together, makes manipulation easier

“Nodes of the Scene Graph”Screenshot from Swerve spy window in 3D Studio Max:showing topmost nodes of the scene graphs (Example).

Agenda

Overview?

Getting started

Scene graph

Low level features

Dynamic meshes

Animation

Demo


Renderable Objects

Made of triangle stripsBase class for meshes (two subclasses: MorphingMesh and SkinnedMesh)

Mesh

2D image placed in 3D spaceGood for labels, etc.Sprite3D

MeshComposed of submeshes

• Common buffer of vertex data

• Submesh has triangle strip indices to vertices

• One Appearance for each submesh

IndexBuffer

Mesh VertexBuffer coordinates

normals

texturecoords

colors

Appearance

Appearance Components

Material colors for lightingCan track per-vertex colors

MaterialBlending, depth bufferingAlpha testing, masking

CompositingModeWinding, culling, shadingPerspective correction hint

PolygonModeFades colors based on distanceLinear and exponential mode

FogTexture matrix, blending, filteringMultitexturing: One Texture2D for each unit

Texture2D

Sprite3D2D image with a position in 3D space

• Scaled mode for billboards, trees, etc.

• Unscaled mode for text labels, icons, etc.

Sprite3D Appearance CompositingMode

Image2DImage2D Fog

Agenda

Overview?

Getting started

Scene graph

Low level features

Dynamic meshes

Animation

Demo


MorphingMesh

• A base mesh and several morph targets

• Result = weighted sum of morph deltas

• Can morph any vertex attribute• Vertex positions

• Colors

• Normals

• Texture coordinates

( )∑ −+=i

iiw BTBRFor vertex morphing animation

MorphingMesh

Example: Animating a rabbit’s face

Base Target 1eyes closed

Target 2mouth closed

Animate eyesand mouth

independently

∑=i

iii vwv BM'• Stretch a mesh over a hierarchic “skeleton”

• The skeleton consists of scene graph nodes

• Each node (“bone”) defines a transformation

• Each vertex is linked to one or more bones

SkinnedMeshArticulated characters without cracks at joints

SkinnedMesh

Bone BBone A

"skin"shared vertex,weights = (0.5, 0.5)

non-sharedvertexWeighted skinning

Neutral pose, bones at rest

Bone A

Bone B

position in A'scoordinate system

position in B'scoordinate system

interpolatedposition

Bone B rotated 90 degrees

SkinnedMesh Example

No skinning Local skinning Smooth skinning

Agenda

Overview?

Getting started

Scene graph

Low level features

Dynamic meshes

Animation

Demo


Animation: Relevant Classes

Storage for keyframesDefines interpolation modeKeyframeSequence

Controls the playback of one or more animationsAnimationController

A link between sequence, controller and targetAnimationTrack

Base class for all objects that can be animatedObject3D

Demo

• Creature in Wonderland• Document and example codes for this demo

available at www.forum.nokia.com

• Remember also Nokia:• Nokia Reference Implementation,

Binary For JSR-184 3D Graphics API For J2ME™, “Emulator”

• Sun Wireless Toolkit (WTK) 2.2

40

http://www.forum.nokia.com/


Agenda

Overview?

Getting started

Scene graph

Low level features

Dynamic meshes

Animation

Demo


New Dimension for Java Games

• Downloadable Java games will be literally transformed into a newdimension when the Mobile 3D Graphics API hits the market. This new standard will bring Java on par with the Nokia N-Gage™ game deck in terms of visual quality.

• Java is the most widely deployed platform for third-party applications in mobile terminals. Nokia alone expects to ship approximately 100 million color-screen Java devices in 2004.

• 3D graphics makes games look more impressive, but it also enables new genres by giving more freedom to designers and artists. It even helps to bring down costs: being inherently scalable, it allows games to be easily targeted at different display sizes and color depths.

New Dimension for Java GamesThe difference in visual quality between state-of-the-art Java games ( left) and Symbian games (middle). The Mobile 3D Graphics API bridges this gap (right)

Developer Tools• Modeling tools for making *.m3g files = 3D scenes

• 3DS Max™ 7 includes a built-in M3G exporter. 3DS Max™ is the most popular 3D modeling and animation package in the game industry.

• Digital Element provides exporter that works also with 3DS Max™ version 6

• HI Corporation has exporters for all versions of 3DS Max, as well asLightwave 8. - utility for previewing and examining M3G files, useful for programmers and designers.

• Superscape: A basic version of their Swerve Exporter freely available. The exporter works with 3DS Max™ versions 5.1, 6, and 7. The professional version of Swerve Exporter is only available to game developers who sign a publishing deal with Superscape.

• Follow the open source projects:At least two independent developers (with no Web sites) are making exporters for Blender http://www.blender3d.com, an open-source 3D modeling package.

Summary

• M3G is the 3D API for J2ME™ technology• Will be supported in millions of devices

Currently supported in Nokia 6630, (6255i CDMA device, US model)

• Not just an API• M3G also defines a binary file format

• M3G is flexible• Using the immediate mode gives you full control

• High-level features make your code faster and smaller

More Informationhttp://java.sun.com

http://www.forum.nokia.com

http://java.sun.com/

http://java.sun.com/



Thank You

Have a nice time with 3D development

Documents

Mobile 3D Graphics API M3G En