Layar, the next mass medium

Creating Augmented Reality Experience with Layar Xuan WangDeveloper Support, Layar

Content• General Intro:

• What is Layar ?

• What does it offer ?

• How to create a layer ?

• Q&A, Discussion

• 3D modeling in Layar:

• What’s possible now?

• What are the best practices ?

• How to get 3d models in a layer ?

• Q&A, Discussion

GPSWhere am I?

CameraWhat do I see in the real world?

Compass In which direction am I looking?

AccelerometerHow is my phone oriented in the world?

Internet connectivityTo connect to the digital world

Gyroscope (optional)To provide a smoother experience

Information Service

Earthquake information

Experience - Berlin wall

WTC

Future - UAR

Brands/Agents: Audi A1, Moscow

Events

Social Service -Tweets Around

Foursquare

Games - ScavengAR Hunt

Uninvited Art Exhibition, MoMA

Layer Features Overview

• Support for icon/images/3D models

• Support for animated POIs

• Sharing to facebook, Twitter and Developer’s server

• Actions: call, email, share, web page, audio, video, etc

• Gaming elements: autotriggers, relative POIs, point to point

Open Platform

Brands, publishers Developers Layar End userContent Layers

Distribution

Search & Discovery

The Layar ecosystem

Layar Platform Components

Browser Player Publishing

Layar Connect - Layer Management System

• Publishing API for Third party layer management system

• End-to-end and one-stop-shop

Layar Platform Architecture

Layar server

Fixeddata

Layar publishing

environment

Layer service provider

Layer content sources

Get layersGet POIs

Layer definitio

ns

Layer Developer API

Get POIs

Create layer

Get POIsView POI information

View POI information

GPS data

Layar environment

Third-party environment

Legend

How to create layers ?

• Layers can be built using simple web technology

• You can use programming languages like PHP, MySQL, Java etc as long as the response is in JSON format

• 2 step process:

1. Create a layer definition in the Layar publishing environment (or using layar connect calls)

2. Create a layer service for delivering the POI content to the Layar app (Based on Developer API )

Step 1: Publishing environment

• http://layar.com/publishing

• Add all the static content and metadata of a layer

• Listing details

• Look & feel

• Service provider URL (connect to your web service)

Step 2: Layer service

• Create a layer service for delivering the POI content to the Layar app

• The following elements are needed to set up a layer service:

• A public web server

• A database

• The response should be in JSON format

Third party tools

• Instead of creating your own layer service, you can use one of the third party tools developed by the community, e.g.:

• PorpoiseOpen source server software, requires programming skills, more control

• Hoppala AugmentationHosted solution, easy to use, limited funtionality

• LMS using Layar Connect, such as BuildAR, Poistr, Poiz, VISAR, etc.

Developer Support

• The following support resources provide a wealth of information for developers:

• Layar developer wiki

• Layar developer support environment

• Discussion forums

• Support tickets

Launched in June 2009

Based in Amsterdam

VC funded

55 people

About Layar

3D modeling in Layar

• 3D capability in Layar

• Creating 3d models

• Converting 3D model (Layar3D Model Converter)

• Rendering 3d models

Layar Augmented Office

• Multiple materials with coloring and shininess

• Static/Animated texture

• Texture transparency

• Build 3D model on the fly

• Animation API (apply animation programmatically)

What’s possible now ?

General Flow

3D modeling Software, Blender, Google

Sketchup, 3ds Max, etc

wavefront file format(.obj/.mtl)

Layar3D Model Converter

.L3D file format

save as input output

• Coordinate system:

X → West to East

Y → South to North

Z → Ground to Sky

• Vertex: point in 3D space

• Face/polygons: area connecting 3 or more vertices.

Creating 3D models - Terminology (1)

• Normals:

• Indicate the direction of a face.

• Only front facing faces are rendered.

• have effect on the light shading

• If not present in your model, the vertex order is important. (front face is counter clock wise)

Creating 3D models - Terminology (2)

• Materials:

• Diffuse color: The main color of the material that is used when light is reflected on it.

• Ambient color: The color of the material that is used for environment light.

• Specular color: The color of the highlight in the model (often white)

• Shininess: controls the size of the highlight. High shininess gives a small highlight, low shininess makes the highlight area larger.

Creating 3D models - Terminology (3)

Limitations - Mobile devices

• Limited bandwidth (phone operator) - takes too long to load large files.

• Limited Memory/ processing power (low-end devices)

• The screen is small (usually 800X480), subtle details can not be seen.

• Limited GPU (Graphics processor) power

Limitations - Mobile devices

Keep the balance between complexity and efficiency !

• Limited bandwidth (phone operator) - takes too long to load large files.

• Limited Memory/ processing power (low-end devices)

• The screen is small (usually 800X480), subtle details can not be seen.

• Limited GPU (Graphics processor) power

• Polygon counts: recommended max. 10000 (after triangulation)

• File size: max 1mb

• Only support mesh based objects (polygonal modeling)

• The unit for the vertex coordinates is set to 1 meter.

• Keep the object centered on the grid (recommended)

• No transparency on the normal material colors (no alpha blending)

3D Model Requirements

• Supported Texture formats: PNG, JPEG, GIF (static/animated)

• Resized to be nearest power of 2 (width & height), 16x16, 32x64, etc.

• Use UV mapping

• Texture transparency is supported (PNG only)

• No blending (partial transparency not supported)

• No multiple textures supported for one material, e.g. bump mapping.

Texture Requirements

Texture UV Mapping

• Alpha values <0.1 are not rendered at all, causing the models behind the face to show. (cutout effect)

• Semi transparent color values (0.1 <alpha<1.0) are blended with the camera image, the models behind it are not shown.

Texture Transparency (1)

Texture Transparency (2)

Animated Texture

• Add frames:

• Using animated GIF texture, or,

• Add frames in Layar3D model converter

• Use good compression in image files (PNG, reduce number of colors)

• Use small images (recommended smaller than 256x256)

• Recommend to limit the number of frames to 10

Animated Texture

• Add frames:

• Using animated GIF texture, or,

• Add frames in Layar3D model converter

• Use good compression in image files (PNG, reduce number of colors)

• Use small images (recommended smaller than 256x256)

• Recommend to limit the number of frames to 10

Keep the balance between texture size and file size!

• Start with the basic model and add details later

• Keep the model complexity as low as possible

• use texture to add details, such as windows, doors, etc.

• prepare a simple version of the model ( 2 level of details supported in Layar )

• Keep the model as clean as possible

• no floating vertex points

• no overlapping faces/polygons

• remove invisible faces/polygons

• Check if the normals are facing out.

Guidelines & Best Practices - Model

• Keep the texture images as small as possible

• Use good compression in image files

• Reduce the amount of images used and combine them into a single texture file (UV-mapping).

• Create “see-through” parts using texture transparency.

• Optimize material group and organize faces based on material.

Guidelines & Best Practices - Textures

• Keep the texture images as small as possible

• Use good compression in image files

• Reduce the amount of images used and combine them into a single texture file (UV-mapping).

• Create “see-through” parts using texture transparency.

• Optimize material group and organize faces based on material.

Guidelines & Best Practices - Textures

Make sure your model is ready before export to .obj file!

• Convert Wavefront format to Layar3D (.l3d) format, which is based on Wavefront (.obj) but optimized for mobile phone devices.

• written in Java, requires Java v6.0

• Command line version is available

Layar 3D Model Converter

• Imported files

• .obj - contains the vertex and face data

• .mtl - contains the materials and references to texture files

• .png/.jpg/.gif - texture images

• Open existing .l3d model

Layar 3D Model Converter - Importing

Layar 3D Model Converter -Overview

• Model details

• Model dimensions (in meters)

• File size (bytes)

Layar 3D Model Converter -Materials

• Review materials and textures

• Edit colors and shininess values

• Replace texture

Layar 3D Model Converter - Animated Texture

• Change texture type:

• no texture

• static

• animated

• Add animated frames

Layar 3D Model Converter - Preview

• Imitating the Layar reality browser behavior in terms of 3D rendering

• The same look and feel can be expected in Layar reality Browser (v5.0 and above)

Layar 3D Model Converter - Edit model

• Basic functions for quick fix

• Will take effect after saving the model to .l3d format

• Still recommend to correct these while preparing the model in the modeling software

Layar 3D Model Converter - Placement

• Place the 3D model directly on the map ( bird view )

• Help find the accurate position of the model

• Save placement details in JSON format

Layar 3D Model Converter - Saving

• Save as .l3d model

• .l3d file embeds all textures and materials

• Approach 1:

• Create .obj file based on .obj file format

• Convert to .l3d model using command line version of Layar3D model converter.

• Approach 2:

• Generate .l3d file directly (in binary format, file format is available upon request)

Creating 3d model on the fly

• OpenGL ES API

• Simulated camera view in OpenGL

• position

• view

• up

• Object is rendered with perspective

Rendering 3D models in Layar

• Given: a 3d cube which is 500m away from user’s position and it is 50m in width, 40m in length and 30m in height.

• Question: How is it rendered in Layar ?

Rendering 3D models in Layar

{

"dimension": 3,

"object": {

"baseURL": “http://layar.example.com/”,

“full”: “full.l3d”,

“reduced”: “reduced.l3d”,

“icon”: “icon.png”,

“size”: 50

}

“transform”: {

“angle”: 45,

“rel”: false,

“scale”: 2

}

Define 3d Parameters in JSON Response

• 2 levels of detail for the 3d model : full, reduced

• “size” : determines which model to pick up to render. The length of the edge of the smallest cube in which the object can fit.

• “angle”: rotates the object around the z-axis (right hand rule)

• “rel”: if true, the rotation is calculated relative to the position of the user.

• “scale”: can be applied to alter the model size.

Which model to pick up ? (1)

• The rendered size in Layar is determined by:

• Distance: between the 3d model and the user, e.g. 500m

• “size”: the size parameter in “object”

• Scale factor: the “scale” parameter in “transform”.

{

"dimension": 3,

"object": {

"baseURL": “http://layar.example.com/”,

“full”: “full.l3d”,

“reduced”: “reduced.l3d”,

“icon”: “icon.png”,

“size”: 50

}

“transform”: {

“angle”: 45,

“rel”: false,

“scale”: 2

}

Which model to pick up ? (1)

In Layar, in the distance of 500m, there is a 3d object which is 100m (50m x 2) in width, length and height.

• The rendered size in Layar is determined by:

• Distance: between the 3d model and the user, e.g. 500m

• “size”: the size parameter in “object”

• Scale factor: the “scale” parameter in “transform”.

{

"dimension": 3,

"object": {

"baseURL": “http://layar.example.com/”,

“full”: “full.l3d”,

“reduced”: “reduced.l3d”,

“icon”: “icon.png”,

“size”: 50

}

“transform”: {

“angle”: 45,

“rel”: false,

“scale”: 2

}

• rendered size > 100 px, “full” model is picked up.

• 20 px < rendered size <100 px, “reduced” model is picked up.

• rendered size < 20 px, “icon” image is used and scaled down to 10 px threshold.

Which model to pick up ? (2){

"dimension": 3,

"object": {

"baseURL": “http://layar.example.com/”,

“full”: “full.l3d”,

“reduced”: “reduced.l3d”,

“icon”: “icon.png”,

“size”: 50

}

“transform”: {

“angle”: 45,

“rel”: false,

“scale”: 2

}

Based on the rendered size, layar client will determine which model (full/reduced/icon) should be downloaded and rendered.

• rendered size > 100 px, “full” model is picked up.

• 20 px < rendered size <100 px, “reduced” model is picked up.

• rendered size < 20 px, “icon” image is used and scaled down to 10 px threshold.

Which model to pick up ? (2){

"dimension": 3,

"object": {

"baseURL": “http://layar.example.com/”,

“full”: “full.l3d”,

“reduced”: “reduced.l3d”,

“icon”: “icon.png”,

“size”: 50

}

“transform”: {

“angle”: 45,

“rel”: false,

“scale”: 2

}

In Layar, in the distance of 500m, there is a 3d object which is 100m (50m x 2) in width, 80m in length and 60m height.

• The real size embedded in the 3d model will be used, instead of the “size” parameter.

• The “scale” factor will be applied to the real size of the model

The Actual Rendering{

"dimension": 3,

"object": {

"baseURL": “http://layar.example.com/”,

“full”: “full.l3d”,

“reduced”: “reduced.l3d”,

“icon”: “icon.png”,

“size”: 50

}

“transform”: {

“angle”: 45,

“rel”: false,

“scale”: 2

}

Animation API

• A collection of pre-defined animations on POIs.

• Simple appearance animation (drop, grow, spin)

• Full customizable animation: onClick, onUpdate, onFocus, etc

• Layer level/POI level

Animation Video

Useful links

• Creating 3D models (http://layar.pbworks.com/w/page/7783224/Creating-the-3D-objects)

• Download Layar3D model converter (http://public.layar.com/downloads/Layar3DModelConverter.jnlp) 

• Using Layar3D model converter (http://layar.pbworks.com/w/page/32586555/3D-Model-Converter)

• Animation API (http://layar.pbworks.com/w/page/35910564/animation-for-API-5)

• Need Help ? Devsupport environment (http://devsupport.layar.com)

Discussion

Thank youxuan@layar.com