Web 3D [ semantic standards, webgl, hci ]

passionate about low-level 3D coding

embedded devices

enthusiast

graphics programmer

@victorporof  GSoC  student  at  Mozilla  h8p://github.com/victorporof  

Web 3D

Web 3.0D

Web 3.0?

“computer is generating new information, rather than humans”

Conrad Wolfram

Web 3.0?

Definitions vary greatly

¤ Sematic web ¤  “web of data” that enables machines to understand the

semantics, or meaning of information

¤  machine-readable metadata about pages and how they are related to each other

¤ Personalization

Web 3.0?

“first generation Metaverse”

John Smart

Metaverse

¤  web development layer

¤  extension of Web 2.0's technologies (and social networks)

¤  open video

¤  3D simulations

¤  augmented reality

¤  human-constructed semantic standards

3.0D?

© pl4n3

3.0D?

The wrong way

¤  X3D/ VRML (slow, XML)

¤  Java 3D (slower)

¤  QuickTime VR (fake panoramas)

¤  3DMLW (similar to X3D, scripting in Lua)

¤  Flash (proprietary, limited, and of course - slow)

3.0D?

¤  software library

¤  extends the capability of JavaScript

¤  generate interactive 3D graphics

¤  any compatible web browser

¤  context of the canvas HTML element

3.0D?

¤  software library

¤  extends the capability of JavaScript

¤  generate interactive 3D graphics

¤  any compatible web browser

¤  context of the canvas HTML element

3.0D?

¤  software library

¤  extends the capability of JavaScript

¤  generate interactive 3D graphics

¤  any compatible web browser

¤  context of the canvas HTML element

WebGL

3D computer graphics API

without the use of plug-ins

WebGL

3D computer graphics API

without the use of plug-ins

WebGL

3D computer graphics API

without the use of plug-ins

WebGL

3D computer graphics API

without the use of plug-ins

Big players

The WebGL working group includes

Apple, Google, Mozilla, and Opera

Big players

The WebGL working group includes

Apple, Google, Mozilla, and Opera J

their browser doesn’t support WebGL

When?

¤  Canvas 3D experiments started by Vladimir Vukićević ¤  at Mozilla

¤  prototype in 2006

¤  end of 2007: both Mozilla and Opera had made their own separate implementations

¤  early 2009: Mozilla and Khronos started the WebGL Working Group

Libraries

¤  GLGE

¤  C3DL

¤  Copperlicht

¤  SpiderGL

¤  SceneJS

¤  Processing.js

¤  Three.js

Libraries

¤  GLGE

¤  C3DL

¤  Copperlicht

¤  SpiderGL

¤  SceneJS

¤  Processing.js

¤  Three.js

Libraries

¤  GLGE

¤  C3DL

¤  Copperlicht

¤  SpiderGL

¤  SceneJS

¤  Processing.js

¤  Three.js

We’ll see some three.js demos

Hot on the internet a while ago… http://www.chromeexperiments.com/webgl

Yup, it’s made using WebGL http://chrome.angrybirds.com/

Hot on the internet right now… http://www.pixelnerve.com/webgl/sss/sss_singlelight.html

How does it work?

How does it work?

The three laws of a GPU:

1.  It’s not a CPU

2.  loves geometry & pixel algebra

3.  thinks like a SIMD*

The three laws of a GPU:

1.  It’s not a CPU

2.  loves geometry & pixel algebra

3.  thinks like a SIMD*

*SIMD = is a class of parallel computers: single instruction, multiple data

Programs

¤  Vertex shader

¤  Fragment shader

¤  Geometry shader ¤  Hot out of the oven

¤  Not very used

¤  Not quite supported

¤  Not really useful (arguable)

GLSL

¤  GLslang J

¤  high level shading language

¤  based on the C programming language ¤  (yes, it’s high level!)

¤  looks a lot better than first shader languages

Horror

DEFINE LUMINANCE = {0.299, 0.587, 0.114, 0.0}; TEX H0, f[TEX0], TEX4, 2D; TEX H1, f[TEX2], TEX5, CUBE; DP3X H1.xyz, H1, LUMINANCE; MULX H0.w, H0.w, LUMINANCE.w; MULX H1.w, H1.x, H1.x; MOVH H2, f[TEX3].wxyz; MULX H1.w, H1.x, H1.w; DP3X H0.xyz, H2.xzyw, H0; MULX H0.xyz, H0, H1.w; TEX H1, f[TEX0], TEX1, 2D; TEX H3, f[TEX0], TEX3, 2D; MULX H0.xyz, H0, H3; MADX H1.w, H1.w, 0.5, 0.5;

Better

#version 140 uniform Transformation { mat4 projection; mat4 modelview; }; in vec3 vertex; void main() { gl_Position = projection * modelview * vec4(vertex, 1.0); }

Best

Make the triangle pretty Color it red Kthxbye

Best

Make the triangle pretty Color it red Kthxbye

Shader toy

Interaction

¤  Display/ computer screen is 2D

¤  Most input devices are 2D ¤  Mouse, trackpad etc.

¤  3D environments are 3D

Interaction

¤  Display/ computer screen is 2D

¤  Most input devices are 2D ¤  Mouse, trackpad etc.

¤  3D environments are 3D

Interaction

¤  Display/ computer screen is 2D

¤  Most input devices are 2D ¤  Mouse, trackpad etc.

¤  3D environments are 3D

3D interaction

¤  a form of human-computer interaction

¤  users are should be able to move and perform interaction in 3D space

¤  the 3D space used for interaction can be ¤  the real physical space

¤  virtual space representation simulated in the computer

¤  a combination of both

3D interaction

¤  1962 – Sensorama simulator (Morton Heilig) ¤  3D video feedback

¤  motion

¤  audio

¤  haptic (tactile)

3D interaction

¤  1962 – Sensorama simulator (Morton Heilig) ¤  3D video feedback

¤  motion

¤  audio

¤  haptic (tactile)

3D interaction

¤  should be intuitive ¤  after all, humans interact in three dimensions in the real world

¤  interaction devices – 3D interfaces ¤  used in applications that feature virtual environments, and

augmented and mixed realities

3D interaction

¤  should be intuitive ¤  after all, humans interact in three dimensions in the real world

¤  interaction devices – 3D interfaces ¤  used in applications that feature virtual environments, and

augmented and mixed realities

3D interaction - Output

“Currently, users still have difficulty in interpreting 3D space visuals and

understanding how interaction occurs”

3D interaction - Output

¤  visual displays ¤  Head-mounted displays and CAVEs

Cave Automatic Virtual Environment

¤  semi-immersive displays allow users to see both

¤  auditory displays ¤  specially useful when supplying location and spatial

information to the users

¤  haptic displays ¤  tactile feedback or feeling

3D interaction - Output

¤  natural way for humans to move around in a three-dimensional world

¤  many sensory cues present in real environments are missing from virtual environments

¤  sensory cue = “a statistic or signal that can be extracted from the sensory input by a perceiver, that indicates the state of some property of the world that the perceiver is interested in perceiving”

¤  2D surface = inconsistencies in depth perception

3D interaction - Input

“Using 3D representations is not enough to create 3D interaction.

The users must have a way of performing actions in 3D as well”

3D interaction - Input

¤  special/ modified input devices ¤  ex: 3D mouse

¤  trackers ¤  detect or monitor head, hand or body movements

¤  important for presenting the correct viewpoint

¤  coordinate the spatial and sound information

¤  ex: motion trackers, eye trackers, data gloves, bodysuits

3D interaction input techniques

¤  selection and manipulation ¤  selecting, rotating and moving an object

¤  direct-hand manipulation is the most natural technique

¤  navigation ¤  wayfinding

¤  travel

¤  system control (menus, gestures, voice commands)

¤  symbolic input (add or edit text, ex: annotation)

Edusim

¤  free and open source

¤  3D virtual worlds on an interactive whiteboard

¤  direct manipulation ¤  3D virtual learning models

¤  Constructionist Learning

Open Cobalt Project

¤  free and open source software platform (Smalltalk)

¤  constructing, accessing, and sharing virtual world ¤  on local area networks

¤  across the Internet

¤  without any requirement for centralized servers

¤  create deeply collaborative and hyperlinked multi-user virtual workspaces

Mozilla - Tilt Project

¤  3D visualization of a webpage ¤  WebGL

¤  visualization tool

¤  a developer-friendly environment for debugging

¤  information displayed on request: ¤  document’s structure

¤  nesting of the DOM tree

¤  each node’s type, class, id, and other attributes if available

Mozilla – Tilt Project

¤  https://github.com/victorporof/Tilt

¤  http://blog.mozilla.com/Tilt

¤  Shedule: April 25th – August 22nd

¤  Deliverables: ¤  Firefox extension + export to compatible browsers

¤  WebGL javascript library designed to facilitate creating web page DOM visualizations

Questions? O_o

© pl4n3