HTML5 Gaming

JAM40

Erik Oros & Jonathan Feldstein

September 25-27, 2012

Outline

WebWorks + HTML5

Canvas 2D

WebGL

How to draw a triangle

Frameworks

TunnelTilt

Peaks and Valleys

2

WebWorks

3

WebWorks + HTML5

Monetize Your Application

Create and deploy apps as .bar files

Hide Source

Useful APIs

Web Sockets

Device Motion

File Access

GPS

Payment

ActionScript Extensions

4

Canvas 2D

5

What is Canvas 2D?

A surface for programmatic rendering of shapes/images.

Unlike SVG, no scene graph.

<canvas id=“canvas2D” width=“640” height=“480”>

HTML5 canvas is not supported.

</canvas>

var canvas = document.querySelector(‘#canvas2d’);

var context = canvas.getContext(‘2d’);

6

Canvas 2D Functionality

Rects, Paths, Images, Text, Transforms, Shadows, Line Caps and Joins, Colors and Styles, Animations, Clipping

All images attributed to Mozilla Contributors from https://developer.mozilla.org/en-US/docs/Canvas_tutorial and used under the Creative Commons Attribution-Sharealike license. 7

Basic Example

function drawPath(context) {

context.beginPath();

context.moveTo(3, 0);

context.lineTo(10, 0);

context.quadraticCurveTo(30, 0, 20, 13);

context.quadraticCurveTo(0, 14, 0, 13);

context.closePath();

context.fillStyle = "#c0c0c0";

context.fill();

}

8

Basic Example

function init() {

var elem = document.getElementById('myCanvas');

var canvasContext = elem.getContext('2d');

canvasContext.fillRect(0, 0, 125, 95);

var transX = [ 0.15, 0, 1.15, 1, 0.85, 2, 1.85 ];

var transY = [ 0.0, 0.7, 0, 0.7, 1.4, 0.5, 1.2 ];

canvasContext.translate(20,20);

for(var i=0; i<7; i++) {

canvasContext.save();

canvasContext.translate(30*transX[i], 30*transY[i]);

drawPath(canvasContext);

canvasContext.restore();

}

} 9

Hardware Accelerated

BlackBerry PlayBook 2.0

Hardware acceleration through GPU.

Paths are tessellated and passed to OpenGL ES 2.0.

BlackBerry PlayBook 2.1

Updated graphics drivers for increased performance.

BlackBerry 10

Hardware and software improvements will continue the trend.

10

Cocos2d-HTML5 Example

Custom ControlsOverlay.js created.

Joysticks and buttons.

Sprite art by Clint Bellanger

http://www.OpenGameArt.org

Tile art by Daniel Cook

http://www.LostGarden.com

Work In Progress

Documentation.

Open source.

11

WebGL

12

What is WebGL?

Khronos open standard

Brings 3D drawing to the HTML5 canvas tag

JavaScript bindings to OpenGL ES 2.0

13

Pipeline Overview

14

Vertex Data Transformed Vertex Data

Triangle( , , );

Connectivity

Vertex Shader

Primitives

Rasterization

Fragments Colored Fragments

Fragment Shader

Assembly

Vertex Data Transformed Vertex Data

WebGL – Big Picture

15

WebGL Program

GPU

Upload Data,

Modify State,

Run Commands

Query Data

CPU

How To Draw In 5 “Easy” Steps!

1. Create Canvas

2. Setup Render Loop

3. Buffer Geometry

4. Setup Shaders

5. Render

16

1. Create Canvas

It all starts with an HTML canvas

Say goodbye to EGL (yippie!)

17

Basic Canvas 2D Context Creation

<canvas id=“canvasId" width=“512” height=“512”>

</canvas>

<script type=“text/javascript”>

var canvas = document.getElementById(“canvasId”);

gl = canvas.getContext(“experimental-webgl”);

</script>

2. Setup Render Loop

setTimeout(“render();”, 1000/60);

setInterval(“render();”, 1000/60);

window.requestAnimationFrame(“render();”, canvas);

18

3. Buffer Geometry – Array Buffers

19

(-1, -1,0) (1, -1,0)

(0, 1,0)

x

y

Uploading Vertex Position Data to the GPU

var positionBuffer = gl.createBuffer();

gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);

var positions = [ -1, -1, 0, // Vertex 0

0, 1, 0, // Vertex 1

1, -1, 0 ]; // Vertex 2

gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(positions), gl.STATIC_DRAW);

positionBuffer.itemSize = 3; // (x, y, z)

positionBuffer.numItems = 3; // 3 vertices

3. Buffer Geometry – Array Buffers (continued)

20

(-1, -1,0) (1, -1,0)

(0, 1,0)

x

y

Uploading Vertex Color Data to the GPU

colorBuffer = gl.createBuffer();

gl.bindBuffer(gl.ARRAY_BUFFER, colorBuffer);

var colors = [ 1, 0, 0, 1, // Color 1 (Red)

0, 0, 1, 1, // Color 2 (Blue)

0, 1, 0, 1 ]; // Color 3 (Green)

gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(colors), gl.STATIC_DRAW);

colorBuffer.itemSize = 4; // (r, g, b, a)

colorBuffer.numItems = 3; // 3 vertices

4. Setup Shaders – Ins and Outs

21

Attributes Uniforms Uniforms

gl_Position

Varyings

gl_FragColor

4. Setup Shaders – Example

22

Vertex Shader

attribute vec4 aColor;

attribute vec4 aPosition;

uniform mat4 uTransformMatrix;

varying vec4 vColor;

void main() {

gl_Position = uTransformMatrix * aPosition;

vColor = aColor;

}

Fragment Shader

precision highp float;

varying vec4 vColor;

void main() {

gl_FragColor = vColor;

}

4. Setup Shaders – Create Programs

23

Vertex Shader

Fragment Shader

Shader Program

gl.linkProgram

gl.useProgram

gl.attachShader

gl.attachShader

gl.createProgram

gl.createShader

gl.shaderSource

gl.compileShader

gl.createShader

gl.shaderSource

gl.compileShader

4. Setup Shaders – Locations

24

uTransformMatrix

Shader Program

WebGL Program

aColor

aPosition

GPU

Connect Buffered Data to Shader Inputs

// Where to send our position attribute data

var posLoc = gl.getAttribLocation(shaderProgram, "aPosition");

// Where to send our transformation matrix uniform data

var transLoc = gl.getUniformLocation(shaderProgram, “uTransformMatrix");

5. Render

25

Render Frame

gl.clearColor(0, 0, 0, 1);

gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

gl.useProgram(shaderProgram);

// Setup Uniforms

gl.uniformMatrix4fv(transLoc, false, transMatrix);

// Setup Attributes

…

// Draw our primitives

…

5. Render – With Array Buffers

26

Set Up Attributes and Draw Primitives

gl.enableVertexAttribArray(posLoc);

gl.enableVertexAttribArray(colorLoc);

gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);

gl.vertexAttribPointer(posLoc, positionBuffer.itemSize, gl.FLOAT, false, 0, 0);

gl.bindBuffer(gl.ARRAY_BUFFER, colorBuffer);

gl.vertexAttribPointer(colorLoc, colorBuffer.itemSize, gl.FLOAT, false, 0, 0);

gl.drawArrays(gl.TRIANGLES, 0, positionBuffer.numItems);

gl.disableVertexAttribArray(colorLoc);

gl.disableVertexAttribArray(posLoc);

Render – Primitive Types

27

WebGL Frameworks

WebGL Frameworks

J3D by Bartek Drozdz (Open source, MIT)

Leverage Unity3D scenes in your WebGL applications.

https://github.com/drojdjou/J3D

Inka3D (Closed source)

Leverage Maya scenes in your WebGL applications.

http://www.inka3d.com

CubicVR.js by Charles Cliffe (Open source, MIT)

Full-fledged WebGL framework, JSON/Collada imports, physics.

https://github.com/cjcliffe/CubicVR.js

Three.js

Created by Ricardo Cabello as a 3D JavaScript library.

One of the most popular WebGL frameworks in use.

Large community involvement.

Under active development.

Distributed under MIT license.

https://github.com/mrdoob/three.js

Images from: http://mrdoob.github.com/three.js/ 30

Pure WebGL vs. Three.js

Exercise: Drawing a square in 6 steps.

31

Pure WebGL Three.js

Create a Canvas. Import Three.js.

Get a WebGL context. Setup scene.

Setup a GLSL program. Setup camera.

Setup geometry. Add geometry with material.

Setup shaders.

Render. Render

32

Step 1 of 6

Pure WebGL (Lines: 1) Create canvas.

<canvas id=“myCanvas” width=“1024” height=“512”></canvas>

Three.js (Lines: 1) Import Three.js.

<script src=“js/Three.js”></script>

33

Step 2 of 6

Pure WebGL (Lines: 3) Get WebGL context.

var canvas = document.querySelector(“#myCanvas”); var gl = canvas.getContext(“experimental-webgl”);

Three.js (Lines: 2) Setup scene.

scene = new THREE.Scene();

34

Step 3 of 6

Pure WebGL (Lines: 8) Setup GLSL program.

var vertexShader = createShaderFromScriptElement(gl, “2d-vertex-shader”); var fragmentShader = createShaderFromScriptElement(gl, “2d-fragment-shader”); var program = createProgram(gl, [vertexShader, fragmentShader]); gl.useProgram(program); var positionLocation = gl.getAttribLocation(program, “a_position”);

Three.js (Lines: 5) Setup camera.

camera = new THREE.PerspectiveCamera(FOV, ASPECT_RATIO, NEAR_PLANE, FAR_PLANE); camera.position.z = 1000.0; scene.add(camera);

35

Step 4 of 6

Pure WebGL (Lines: 13) Buffer geometry.

var buffer = gl.createBuffer(); gl.bindBuffer(gl.ARRAY_BUFFER, buffer); gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(

[-1.0, -1.0, 1.0, -1.0, -1.0, 1.0, -1.0, 1.0, 1.0, -1.0, 1.0, 1.0] ), gl.STATIC_DRAW);

gl.enableVertexAttribArray(positionLocation); gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);

Three.js (Lines: 9) Add geometry with material.

geometry = new THREE.CubeGeometry(1024, 600, 0); material = new THREE.MeshBasicMaterial({color: 0x00FF00, wireframe: false}); mesh = new THREE.Mesh(geometry, material); scene.add(mesh);

36

Step 5 of 6

Pure WebGL (Lines: 24) Setup shaders.

<script id=“2d-vertex-shader” type=“x-shader/x-vertex”> attribute vec2 a_position; void main() {

gl_Position = vec4(a_position, 0.0, 1.0); }

</script> <script id=“2d-fragment-shader” type=“x-shader/x-fragment”>

void main() { gl_FragColor = vec4(0.0, 1.0, 0.0, 1.0); // Green.

} </script>

37

Step 6 of 6

Pure WebGL (Lines: 25) Render.

gl.drawArrays(gl.TRIANGLES, 0, 6);

Three.js (Lines: 13) Render.

renderer = new THREE.CanvasRenderer(); renderer.setSize(window.innerWidth, window.innerHeight); document.body.appendChild(renderer.domElement); renderer.render(scene, camera);

Result:

TunnelTilt

38

TunnelTilt - Goals

Showcase our WebGL platform

Compelling piece of content

Leverage WebWorks + HTML5

Audio

Accelerometer

Package into .bar file

39

TunnelTilt – SlipStream was born!

Model loading with JSON + XMLHttpRequest

Vertex positions, texture coords, normals, indices

Model from: http://www.psionic3d.co.uk

40

TunnelTilt – A New Look

Curved tunnels

Light at the end of the tunnel

Higher frame rate

41

Simple level.json File

{ “data” : [

{“angles” : [90, 270], “position” : 100}

]}

Loaded Level With XMLHttpRequest

var JSONobj = {};

var request = new XMLHttpRequest();

request.open(“GET”, “level.json”);

request.onreadystatechange = function(){

if(request.readyState == 4) {

JSONobj = JSON.parse(request.responseText);

}

};

TunnelTilt – SlipStream (continued)

42

90°

270°

TunnelTilt - Texturing

43

Load Image

Texture = function(filename) {

this.texId = gl.createTexture();

var image = new Image();

image.onload = function() {

gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true);

gl.bindTexture(gl.TEXTURE_2D, this.texId);

gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, img);

};

image.src = filename;

};

TunnelTilt - Demo

Useful Debugging Technique

1. Create glDebug to hold our debug context

2. Get WebGL Context

3. Iterate over WebGL context attributes

4. If the attribute is a function add a new function to glDebug

context that wraps the function and then calls getError()

5. If the attribute is not a function, copy it to glDebug

TunnelTilt - Debugging

45

Useful Debugging Technique

glDebug = {};

glDebug.gl = canvas.getContext(“experimental-webgl”);

for (var attr in glDebug.gl) {

var method = eval(“glDebug.gl.” + attr);

if (typeof(attr) == ‘function’) {

glDebug[attr] = function() {

var result = arguments.callee.realFunc.apply(this.gl, arguments);

var error = this.gl.getError();

if (error != 0) {

console.debug(arguments.callee.realFunc);

debugger;

}

return result;

}

} else {

glDebug[attr] = gl[attr];

}

} 29

TunnelTilt - Geometry

Use triangle strips

Better still use degenerate triangles

46

(with anti-aliasing)

Triangles

Triangle Strips

Degenerate Triangles

2k 20k Number of Triangles

32

32

34

26

27

29

Triangles: 3 * number of triangles worth of vertices

Triangle strips: 2 + 2 * number of triangles worth of vertices

TunnelTilt – Geometry (continued)

47

A

B

C D

E F

A

B

C

D E G

F H

DE

Triangles Triangle Strips (Degenerate Triangles)

TunnelTilt - Precision

48

Reduce precision where possible

Samplers default to lowp

Query GL_FRAGMENT_PRECISION_HIGH for highp

TunnelTilt – Precision (continued)

49

PlayBook Desktop

Precision Problem

uniform sampler2D uSampler;

…

void main(void) {

…

mediump float t = 255.0;

shininess = texture2D(uSampler, vTexCoords) * t;

}

Precision Problem

uniform mediump sampler2D uSampler;

…

void main(void) {

…

shininess = texture2D(uSampler, vTexCoords) * 255.0;

}

Precision Problem

uniform sampler2D uSampler;

…

void main(void) {

…

shininess = texture2D(uSampler, vTexCoords) * 255.0;

}

TunnelTilt – Quality vs. Performance

50

Use Smaller Canvases

<canvas id=“canvasId" width=“512px” height=“300px” style=“width: 1024px;

height: 600px”>

</canvas>

Turn Off Anti-aliasing

<script type=“text/javascript”>

var canvas = document.getElementById(“canvasId”);

gl = canvas.getContext(“experimental-webgl”, {antialias:false});

</script>

Point Sprites

Require less data

Beware: max size!

Move work to the GPU

Use the vertex shader

TunnelTilt - Particles

51

TunnelTilt – WebWorks Tips

52

To Prevent Scrolling

function handleTouchMove(event) {

… // Do stuff

event.preventDefault();

}

document.ontouchmove = handleTouchMove;

To Prevent Zooming

<meta name="viewport" content="width=device-width,user-scalable=no"/>

To Lock the Orientation Add to config.xml

<rim:orientation mode=“landscape”/>

TunnelTilt – Audio + Accelerometer

53

Audio HTML5 or Flash Audio

function playSoundHTML5(file, loops, volume) {

var audio = new Audio(file);

if (loops)

audio.loop = “true”;

audio.volume = volume;

audio.preload = “auto”;

audio.play();

}

Accelerometer DeviceMotion API

var accelX = 0;

window.addEventListener(“devicemotion”, function(event){

accelX = event.accelerationIncludingGravity.x;

}, true);

TunneltTilt - Online

Play

Online at http://blackberry.github.com/WebGL-Samples/tunneltilt/

Free from BlackBerry App World (~73k downloads)

Download the source

https://github.com/blackberry/WebGL-Samples/

54

Peaks and Valleys

What Is Peaks and Valleys?

WebGL learning sample.

Combines a number of concepts:

Terrain generation

Skybox

Colour and texture blending

Ambient and directional lighting

Dual virtual joysticks for controls

Camera movement and rotation

www.github.com/blackberry/WebGL-Samples

Diamond-Square Fractal Algorithm

http://www.gameprogrammer.com/fractal.html

Fault Algorithm

http://www.lighthouse3d.com/opengl/terrain/index.php3?fault

Raised Zones

Generate a square.

Raise all included vertices.

Repeat.

Average with neighbours.

Generating the Landscape

57

Generating the Skybox

Two easy steps:

Paint the inside of a box.

Place box on head/camera.

Size limitations: 1024 x 1024

The magic happens with:

gl.disable(gl.DEPTH_TEST);

Don’t forget to turn this back on. 58

Blending Colours and Textures

Fragment shader blends colours and textures.

vec4 diffuseSand = vec4(0.8, 1.0, 0.0, 1.0);

vec4 diffuseGrass = texture2D(texture1, vec2(vTextureCoord.s, vTextureCoord.t));

vec4 diffuseRock = vec4(0.5, 0.5, 0.5, 1.0);

vec4 diffuseSnow = vec4(1.0, 1.0, 1.0, 1.0);

vec4 color = vec4(1.0, 1.0, 1.0, 1.0);

color = mix(diffuseSand, color, min(abs( 400.0 - vPosition.y) / 500.0, 1.0));

color = mix(diffuseGrass, color, min(abs( 800.0 - vPosition.y) / 200.0, 1.0));

color = mix(diffuseRock, color, min(abs(1000.0 - vPosition.y) / 300.0, 1.0));

color = mix(diffuseSnow, color, min(abs(1200.0 - vPosition.y) / 300.0, 1.0));

gl_FragColor = vec4(color.rgb * vLightWeighting, color.a);

59

Blending Colours and Textures

Special thanks to Chandler Prall. http://chandler.prallfamily.com/2011/06/blending-webgl-textures

60

Traversing Peaks and Valleys

Expanded on virtualjoystick.js by Jerome Etienne for multi-touch controls.

Minimize triangles via gl.TRIANGLE_STRIP.

61

Let’s Talk More

Join the roundtable for this session at 6:00pm in Exhibit Hall 1

Continue the conversation on Twitter using the hashtag #BBJam and the session ID, #JAM40

62

Don’t Forget

Download the BlackBerry Jam Americas Mobile Conference Guide from BlackBerry App World

Give us your Reasons to Believe at the 10k Reasons to Believe booth in the Jam Space

63

THANK YOU

JAM40

Erik Oros & Jonathan Feldstein

September 25-27, 2012