Mobile Based Augmented Reality

MOBILE BASED AUGMENTED REALITY

WHITE PAPER

Author: Arvind Chikkalavalasa

WHITE PAPER MOBILE BASED AUGMENTED REALITY

Wipro Technologies Page 2 of 11

Abstract

Augmented reality adds information and meaning to a real object or place. Unlike virtual reality, augmented reality does not create a simulated reality. This paper surveys the field of Mobile based Augmented Reality. It describes the manufacturing applications that have been explored. Different types of augmented reality applications are also covered. The paper also describes the characteristics of Mobile based Augmented Reality systems. Development platform for developing such applications is discussed. The challenges in developing such applications are also explained.



Table of Contents

1. INTRODUCTION ........................................................................................................................................... 4 2. HISTORY .................................................................................................................................................... 4 3. DEFINITION ................................................................................................................................................ 4 4. TYPES OF AUGMENTED REALITY...................................................................................................................... 4 5. APPLICATIONS OF AR IN MANUFACTURING ...................................................................................................... 5 6. MOBILE BASED AUGMENTED REALITY .............................................................................................................. 6 7. DEVELOPMENT PLATFORM ............................................................................................................................ 6 8. JUNAIO : APPLICATION FRAMEWORK ............................................................................................................... 7 9. CHALLENGES OF MOBILE BASED AR ................................................................................................................ 9 10. APPENDIX............................................................................................................................................ 9 11. REFERENCES ..........................................................................................................................................10



1. INTRODUCTION

As computers increase in power and decrease in size, new mobile, wearable, and pervasive computing applications are rapidly becoming feasible, providing people access to online resources always and everywhere. This new flexibility makes possible new kind of applications that exploit the person's surrounding context. Augmented reality (AR) presents a particularly powerful user interface (UI) to context-aware computing environments. AR systems integrate virtual information into a person's physical environment so that he or she will perceive that information as existing in their surroundings. Mobile augmented reality systems (MARS) provide this service without constraining the individual’s whereabouts to a specially equipped area. Ideally, they work virtually anywhere, adding a palpable layer of information to any environment whenever desired.

2. HISTORY

Augmented Reality has evolved from Virtual Reality. Virtual Reality is a completely computer generated environment. Augmented Reality in contrast adds computer generated information to the real world.

3. DEFINITION

Augmented reality is related to the concept of virtual reality (VR). VR attempts to create an artificial world that a person can experience and explore interactively, predominantly through his or her sense of vision, but also via audio, tactile, and other forms of feedback. AR also brings about an interactive experience, but aims to supplement the real world, rather than creating an entirely artificial environment.

4. TYPES OF AUGMENTED REALITY

a. Surface: The most understandable form of reality that is augmented. The input would be screen, floors, walls, etc that respond to the touch providing virtual real time information or collaboration.

b. Pattern: The AR system performs simple pattern recognition on a shape, marker or face and replaces it with a static or moving element.

c. Outline: It is where hand, body or eye outline is picked up and seamlessly merged with the virtual object e.g, pick up a 3D object which doesn’t exist because the system is tracking the hand outline.

d. Location: Based on detailed GPS or triangulation location and position/view of the camera/device, the AR system overlays



information precisely over buildings and people as we move through real space.

We will discuss about Pattern based Augmented Reality in this paper.

5. APPLICATIONS OF AR IN MANUFACTURING

a. Design and Prototyping Checking of assembly space and possible collisions between different

components. Packaging analysis of a power train at the front wheel. Design, Aesthetics and usability validation. Examination of quantity and positions of welding points and bolts.

b. Production and Shop Floor Management

AR can highlight a dangerous area around a virtual fire. AR based Assembly Line management and machinery space allocation. Visualization of virtual robot and machinery in plant-environment. Viewing maintenance records for a machine.

c. Supply Chain and Distribution

AR based order picking for material and spare parts. AR based drop ship route planning, distribution tracking and performance. Trucker’s route assistance and Delivery details (Item details, Quantity, etc)

d. Usage and Training BMW’s AR glasses, which assist mechanics in performing maintenance. User training for assembly in case of DIY products, machinery setup guide,

trouble shooting help using AR

e. EOL Management Layar based AR mapping of Recycling facilities. Information related to pickup timings.



6. MOBILE BASED AUGMENTED REALITY

With the advent of IPhone and Android operating system, Augmented Reality applications can be created on mobile devices having these platforms. Design aspects of Mobile based AR:

a. Tracking the location of user device: For tracking solutions like Hybrid Tracking and Sensor Net Tracking have been used.

b. Deciding where the calculations need to be done for combining the virtual

and real worlds: Computations were earlier done on a remote server. However with the increased capability of mobile devices, most of the computation is done on the mobile device itself.

Mobile based AR systems can be classified on the basis of tracking mechanism as follows:

a. Location based: The tracking of the input happens on the basis of latitude and longitude of the location. Layar is one of the popular applications supporting Location based AR on mobile devices.

b. Image based: The input would be tracked with an image. The virtual content mapped to the input image would be rendered on the mobile AR browser.

Fig 1: Mobile Augmented Reality

7. DEVELOPMENT PLATFORM

We will be discussing the development of image based AR applications in the following topics. The development platform consists of the following: Hardware:

1. A smart phone with the below configuration is required: a. IPhone: It should at least have iOS4. b. Android based: HTC Desire or Motorola Tablet version.



c. Built in Camera with at least 3MP. d. Touch screen. e. Connected to network with a good bandwidth.

2. A server machine a. The server machine should be accessible on internet.

Software:

1. Junaio 1.5 browser should be installed on the mobile device. 2. Apache server should be installed on the server machine. 3. Zend Framework should be installed on the server machine.

8. JUNAIO : APPLICATION FRAMEWORK

Junaio is one of the development platforms for creating mobile based AR applications. Junaio provides an AR browser which needs to be installed on IPhone or Android mobile. Junaio framework supports both location-based and image-based AR. The image based AR is referred to as Junaio Glue. Junaio framework helps in achieving the following objectives with an image based tracking: 1. The input being an image, the output can be rendered as a 2D image. 2. The input being an image, the output can be a 3D content. 3. The input being an image, the output can be an audio or video file.



Fig 2: Junaio Architecture

How Junaio Works:

1. Channel: As a first step, a channel needs to be created on Junaio server. 2. Point of Interest (poi): Every input is considered as a point of interest. For

such POI, virtual content could be a 2D, 3D or audio/video file. 3. Callback URL: The channel created above will contain the URL of the server

where the virtual content resides. The request for POI is sent from mobile device to the Junaio server which is in turn forwarded to the external server. The virtual content is routed back to the mobile device via the Junaio server. Steps to develop an AR application: 1. The application can be developed using PHP programming language. 2. An account needs to be created at junaio.com. An API key is provided. This

key will be used for authentication. 3. Register a channel: When creating a channel, channel type needs to be

selected either as location based or image based (junaio glue). 4. Create a Tracking file: A tracking XML file has to be created for the input

image. 5. Provide the API key in the application configuration file. 6. Create Callback API: A POI file needs to be created which contains the

tracking file path and poi. The POI is identified with it’s id and contains the reference to virtual content.



7. Publish the POI on the external server. 8. Provide the site url as callback URL in the channel created above. 9. Validate the channel.

Run the Application:

1. Open Junaio browser on the mobile device. 2. Login to your Junaio account. 3. Click on Browse Channels. 4. Select your channel. 5. Point your mobile towards the input image for which the tracking XML was

created. 6. Virtual content mapped to this image will be rendered on the mobile

browser.

9. CHALLENGES OF MOBILE BASED AR

a. High network bandwidth on mobile devices is required. This is possible with 3G technology which is still not so easily available in India.

b. The virtual content needs to be always prepared and updated on the server.

10. APPENDIX

<?php

echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>

<results

trackingurl=\"http://www.junaio.com/publisherDownload/tutorial/

tracking_tutorial.xml_enc\">

<poi id=\"1\"

interactionfeedback=\"none\"><name><![CDATA[metaio

Man]]></name>

<author><![CDATA[metaio]]></author>

<translation>0.0,0.0,0.0</translation>

<o>0.0,0.0,0.0</o>

<mime-type>model/md2</mime-type>

<mainresource><![CDATA[http://www.junaio.com/publisherDownload/

tutorial/metaioman.md2_enc]]></mainresource>

<thumbnail>http://www.junaio.com/publisherDownload/tutorial/ic

on.jpg</thumbnail>

<icon>http://www.junaio.com/publisherDownload/tutorial/icon.jp

g</icon>

<route>false</route>

<force3d>true</force3d>



<s>1</s>

<behaviours>

<behaviour

type=\"click\"><length>6</length><node_id>close_up</nod

e_id>

</behaviour>

<behaviour

type=\"idle\"><length>0</length><node_id>idle</node_id>

</behaviour>

</behaviours>

<customizations/>

<resources><resource>http://www.junaio.com/publisherDownload/t

utorial/metaioman.png</resource></resources>

</poi>

</results>";

?>

11. REFERENCES

1. Mobile Based Augmented Reality

http://www1.cse.wustl.edu/~jain/cse574-10/ftp/reality/index.html 2. Mobile Based Augmented Reality

http://graphics.cs.columbia.edu/projects/mars/mars.html 3. Junaio

http://www.junaio.com

http://www1.cse.wustl.edu/~jain/cse574-10/ftp/reality/index.html

http://graphics.cs.columbia.edu/projects/mars/mars.html

http://www.junaio.com/



About the Authors Arvind Chikkalavalasa is a Module Lead working for JSSTK project in Boeing Account.

About Wipro Technologies

Wipro is the first PCMM Level 5 and SEI CMMi Level 5 certified IT Services Company globally. Wipro provides comprehensive IT solutions and services (including systems integration, IS outsourcing, package implementation, software application development and maintenance) and Research & Development services (hardware and software design, development and implementation) to corporations globally. Wipro's unique value proposition is further delivered through our pioneering Offshore Outsourcing Model and stringent Quality Processes of SEI and Six Sigma.

© Copyright 2002. Wipro Technologies. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without express written permission from Wipro Technologies. Specifications subject to change without notice. All other trademarks mentioned herein are the property of their respective owners. Specifications subject to change without notice.

Documents

Mobile Based Augmented Reality