1

Augmented Reality

Ruth Aylett

Overview

What is AR? Combining real and virtual worlds

– Markers and registration– Components and performance

Display technologies Applications

What is Augmented Reality?

A combination of a realscene viewed by a userand a virtual scenegenerated by acomputer thataugments the scenewith additionalinformation.

What is the Goal of AR?

To enhance a person’s performance andperception of the world

But, what is the ultimate goal????

2

The Ultimate Goal of AR

Create a system such that no userCANNOT tell the difference betweenthe real world and the virtualaugmentation of it.

Augmented Reality vs.Virtual Reality

Augmented Reality System augments

the real world scene User maintains a

sense of presence inreal world

Needs a mechanismto combine virtualand real worlds

Virtual Reality: Totally immersive

environment Visual senses are

under control ofsystem (sometimesaural andproprioceptivesenses too)

Miligram’s Reality-VirtualityContinuum

Mixed Reality (MR)

RealEnvironment

VirtualEnvironment

AugmentedReality (AR)

AugmentedVirtuality (AV)

Miligram coined the term “Augmented Virtuality” to identifysystems which are mostly synthetic with some real worldimagery added such as texture mapping video onto virtualobjects.

Miligram’s Taxonomy forMixed Reality Displays

Reproduction Fidelity – quality ofcomputer generated imagery

Extent of Presence Metaphor – levelof immersion of the user within thedisplayed scene

Extent of World Knowledge –knowledge of relationship betweenframes of reference for the realworld, the camera viewing it, and theuser

ReproductionFidelity

Extent ofPresenceMetaphor

Extent of WorldKnowledge

3

Combining the Real andVirtual Worlds

We need: Precise models Locations and optical properties of the viewer

(or camera) and the display Calibration of all devices To combine all local coordinate systems

centered on the devices and the objects inthe scene in a global coordinate system

Combining the Real andVirtual Worlds (cont)

Register models of all 3D objects ofinterest with their counterparts in thescene

Track the objects over time when the usermoves and interacts with the scene

Popular 2D Markers Illustration of possible SpecialPattern Markers for 3D Object

4

Same Object Two BrandNames

Realistic Merging

Requires: Objects to behave in physically plausible

manners when manipulated Occlusion Collision detection Shadows

**All of this requires a very detailed descriptionof the physical scene

Components of an AugmentedReality System

Research Activities

Develop methods to register the twodistinct sets of images and keep themregistered in real-time– New work in this area has started to use

computer vision techniques

Develop new display technologies formerging the two images

5

Performance IssuesAugmented Reality systems are expected: To run in real-time so that the user can move around

freely in the environment Show a properly rendered augmented image

Therefore, two performance criteria are placed on thesystem:

Update rate for generating the augmenting image Accuracy of the registration of the real and virtual image

Limitations for Updating theGenerated Images

Must be at 10 times/second More photorealistic graphics rendering Current technology does not support

fully lit, shaded and ray-traced imagesof complex scenes

Failures in Registration

Failures in registration due to:– Noise

• Position and pose of camera with respect to the realscene

• Fluctuations of values while the system is running

– Time delays• In calculating the camera position• In calculating the correct alignment of the graphics

camera

Display Technologies

Monitor Based Head Mounted Displays:

– Video see-through– Optical see-through

6

Monitor Based AugmentedReality

Simplest available Little feeling of being immersed in

environment

Optical see-through HMD

Video see-through HMD Video Compositionfor Video see-through HMD

Chroma-keying– Used for special effects– Background of computer graphics images is set to

a specific color– Combining step replaces all colored areas with

corresponding parts from video

Depth Information– Combine real and virtual images by a pixel-by-

pixel depth comparison

7

Advantages ofVideo see-through HMD

Flexibility in composition strategies Wide field of view Real and virtual view delays can be

matched

Advantages ofOptical see-through HMD

Simplicity Resolution No eye offset

Different Inputs

Capacitive sensing on a table Sensors on a PDA

Applications

Medical Entertainment Military Training Engineering

Design Robotics and

Telerobotics

Manufacturing,Maintenance, andRepair

Consumer Design Hazard Detection Audio

8

Medical AR The proposed extension

Archeoguide

Visitor enters personal preferences,then proceeds to walk through atour the system customizes forthem.

Visitor arrives at a cultural heritagesite;

System Administrator provides ahigh-tech “wearable computer”

– lightweight portable computer– Head Mounted Display (glasses,

earphone, speaker, camera)– batteries etc.

Visitor “sees” reconstructed ancientmonument placed where the ruinslie.

Visitor listens to the information thesystem provides, and may interactwith the system: request more infoor interrupt the info coming fromthe system.

ambient wood

real wood! … filled with electronics light and moisture meters

– recorded with GPRS location– drawn on map later

‘periscope’– shows invisible things– uses RFID

triggered sound

9

City - shared experience

visitors to Mackintosh Interpretation Centre– some on web, some use VR, some really there

interacting– talk via microphones– ‘see’ each other virtually

different places different modalities shared experience

Collaborative work

Overcomes some of the problems ofcollaborative VR systems

Taking games off the desk

Two linked technologies make itpossible– Mobile computing

• Cell phones, PDAs

– Augmented reality• Projecting virtual images into the real world

Pervasive gaming

Games that happen in every-day life– Actively draw you into play

Using many modalities– They email you or phone you– Use websites– And the real world

10

The Nokia Game

Ran in 2002 for 20 days– 600,000 players– 28 countries in Europe and Middle East– Used Internet, press, radio, email, telephone, TV– System phoned players to tell them to watch a TV intro– Adventure format - series of puzzles to solve

Botfighters

Construct a bot on the webin the BotLab

Locate other players viaSMS– Start shooting when

physically close using phone– Killing a bot ups your score

depending on its strength– Earn Robucks with which to

upgrade bot

Uncle Roy is all around you

Run in UK cities 2003-2004– Location-based players with PDAs on the

streets looking for Uncle Roy– Internet -based players could help or

hinder using a 3D model of area– Street players phoned in location in

exchange for hints– 15 operators and actors ran the game

Uncle Roy gameplay

One hour time-limited play Had to locate an office in the real city

– Enter and answer a question– Met outside by limousine to unknown destination– If took this then conversation with driver led to story

conclusion

Misleading clues blurred boundaries between playersand world– ‘look for a (non-existent) tourist approaching you’

11

But where am I really?

Taking an imaginary game world into the realone– Children do this when they play– Share the illusion with other players– A kind of ‘magic’

Does it have potential dangers?– Confused bystanders?– Traffic accidents?– For some, confusion between the real and the

story? LINEAGE for real???