iDevice Augmented Reality

Kyle A. Hoover

ITGM 230 Information Design for Dynamic MediaSavannah College of Art and DesignProfessor David C. Meyers

04/25/10

Summary:This document details my upcoming project, exploring augmented reality on common interactive devices.

(Subject Devices will include iPhone/iPod/iPad) ~iDevice

Basic Concept:By acquiring real-time data,(most importantly the angles and positions of the iDevice), and pushing or computing visual data; a small common mobile device can be used as a window into the universe of augmented reality .

Multi Phase Breakdown:Because of the scope and advanced goals of this project I have broken it down into 3 sub projects or phases.

The success of each one will act as a foundation for the next.

Phase1: Device/Dongle-ComputerWi-Fi GPS

Phase2: Device/Dongle-Additional iDeviceNative GPS

Phase3:Device/Dongle-OnChip controllerModified Camera Stream

Relevant Technology use and Rational:

iDeviceApple's mobile devices provide the ideal environment for this project.Their accessibility, mobility, battery life, display, GPS, Camera, and other features make this the obvious choice for this exploration. Also, the development framework and forward compatibility will hopefully continue to make this project available and relevant for at least several product cycles.Android development will be included in future revisions.

iDevice 3G ARM ProcessorThe same 600 MHz CPU powering the Palm Pre (that can actually run at 833 MHz).1GHz processors are available and dual cores are rumored to be available in the iPhone 4G. I want to market toward the commonality of the iPod/iPhone 3G.It is nice to know that these speeds are around the corner as they will be necessary for realtime 3d and advanced imaging augmented reality.

BluetoothBluetooth Chips are very versatile and cost around 2$. They are common and have low power consumption and a suitable range. Many data profiles have been developed including those that push image and video data. They may be used in conjunction with Wi-Fi to steer a high and reliable data rate. This will not be necessary for the phase 1 application which will use mostly absolute position data.

Wi-FiWiFi will be used for program and data file updates. In Phase 2 video and data streams from high speed external and networked processors may be used. This will be useful in simulations, medical, and CSI like applications.

Wiimote IR CameraCore to the project, this camera will detect the position of the device based on 4 IR LEDs residing in a common accessory case holding the iDevice. The camera will be located on the Dongle and powered by USB in Phase 1. In Phase 3 the portable tracking device may be placed and powered in different locations.

iDevice Pin-OutIn Phase 1 the case will plug into the iDevice and this will power the IR LEDs.In Phase 3 the case will include a pin through plug at the bottom as not to interfere with nomal charging and data transfer, or an imbedded solar cell and battery since little power is required for the IR LEDs. Or the camera could be placed on the device with the dongle used only for IR LED reference unless a high processing connection is required.

Relevant Technology use and Rational: Continued...

Arduino CAn Arduino is a single-board microcontroller and a software suite for programming it. The hardware consists of a simple open hardware design for the controller with an Atmel AVR processor and on-board I/O support. The software consists of a standard programming language and the boot loader that runs on the board. Arduino can be used to interact with the Wiimote controller hardware and is reasonably well documented. The Wiimote may also be used as a bluetooth transciever.

Objective CObjective-C is a reflective, object-oriented programming language which adds Smalltalk-style messaging to the C programming language.Today, it is used primarily on Apple's Mac OS X and iPhone OS: two environments based on the OpenStep standard, though not compliant with it.[2] Objective-C is the primary language used for Apple's Cocoa API.

Java / HardcodeOnce the program is stable and tested it will be converted to “hardcode” with a system on chip structure. May be used in Phase 2 to extend host computer accessibility and in phase 3 embedded into the dongle chip or part of the iPhone chip.

GPSUsed to locate your general location for applications that use global absolute positioning. For Phase 1 this will be done through Wi-Fi. In Phase 3 native iDevice GPS will be used.

Camera feedIn Phase 3 the iDevice cameras can pull the data from the camera augment it and push it back to the view-port. In Phase 3, for the first application “Star Gazer”, a picture may be used to get an absolute global position.

Google EarthFor the first application “Star Gazer” star data will be pulled from Google Earth or NASA's Hubble data. If Google is chosen this application may be submitted to the Google code directory.

Open MAR (Open Mobile Augmented Reality) I hope to develop a standard framework for mobile Augmented Reality Applications.

Test Application #1 || AR StarGazer:This application will be the primary testing application and proof of concept for my “MAR”iDevice.

Basic Concept:Applying the “window” concept to the sky allowing one to view stars and other objects in the sky on a cloudy night, in the city, or even from indoors.

Visual Study:

`Connect to WUB

If Web Available `Check for StarGazer & NASA Updates

`Load Program Assets`Get Location`Load Star Package`Begin GUI Push Loop:

`Check Local Pos`Check User Input`Push View

AI Chart for example application #1:AR StarGazer

Open Application

1st TimeMenu/TrackerSelect

Eventually, there may be more than one way to track this device. As a default the WUB tracking dongle will be used.

RMF: http://letsmakerobots.com/node/7752

RMF =Resource to Move forward || WUB= Wii-USB-Bluetooth || AR = Augmented Reality

1st TimeMenu/TrackerSelect

`1st TimeMenu

`Tracker Select

Default=MyWiiUSBBluetoothTrackingDongle

C iPad Init()

WUB-iPad Information flow:

Phase 1: Computer: JAVA Processing

Phase 2: On WUB Linux Program

Phase 3: WUB On-Chip Program

Test Application #1 || AR StarGazer:

Ad Style Concept