Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
1
User Interfaces for Mobile and User Interfaces for Mobile and Wearable ComputingWearable Computing
Augmented Reality 2Augmented Reality 2
COMS E6176Prof. FeinerColumbia UniversityApril 22, 2004
2
The Ultimate DisplayThe Ultimate Display
“The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal. With appropriate programming, such a display could literally be the Wonderland into which Alice walked.”
— I. Sutherland, The Ultimate Display, Proc. IFIP 65, 506–508, 1965
3
Sutherland’s SeeSutherland’s See--Through Through HeadHead--Worn Display Worn Display Late 60’sLate 60’s
4
SizeWeightDurabilityRun-timeComfortAppearance
Research Challenges:Research Challenges:Ubiquity/Ubiquity/WearabilityWearability
Columbia “Touring Machine”
5
Research Challenges:Research Challenges:DisplaysDisplays
Minolta “forgettable display”
MicroOptical eyeglass display
Eyeglass displays
[www.microopticalcorp.com]
6
Technology:Technology:Spatial TrackersSpatial Trackers
7
Spatial Tracking BasicsSpatial Tracking Basics
Usually a sensor determines position/orientation relative to a signal it detects from a sourceSource or sensor is fixed in a known position/orientationTechnologies:
electromagnetic, ultrasonic, mechanical, video/optical, earth-relative, inertial, satellite, hybridcoarse approaches: radio, RFID, IR
8
Electromagnetic (EM) TrackingElectromagnetic (EM) TrackingThree orthogonal source coils emit a radio frequency electromagnetic signal in sequenceThree sensor coils simultaneously measure the signal from each of the three source coilsNine measurements provide position and orientation
To host computer
Sensor
SourceE.g., Polhemus [www.polhemus.com],Ascension [www.ascension-tech.com]
9
EM Tracking Advantages and EM Tracking Advantages and DisadvantagesDisadvantages
Advantagesrelatively unencumberingdo not need line-of-sight from source to sensor
Disadvantageslarge error in tracking, particularly dynamic errorshort usable range (< 20’ for long-range xmtr)highly sensitive to electromagnetic radiation and metal in environment
10
Ultrasonic TrackingUltrasonic TrackingAn emitter emits an ultrasonic burst, and three receivers pick up the burstThe time between emission and reception by each receiver gives emitter–receiver distanceCan reconstruct 3D position of emitter if positions of receivers are knownThree emitters allow full orientation measurement Sensor (receiver)
Source (emitter)E.g., Logitech “Red Baron”
11
Ultrasonic Tracking Advantages Ultrasonic Tracking Advantages and Disadvantagesand Disadvantages
Advantageslow costeasy to build
Disadvantagesrequires clear line-of-sight from emitters to receiversvery noisy, high dynamic distortionvery short rangeSimple designs are susceptible to echos and environmental noise (e.g., jangling keys)
12
Mechanical Linkage TrackingMechanical Linkage TrackingTypically a jointed structure that measures the angle at each jointUsing known angles and lengths of arms, compute relative positions/orientations of end
Base
trans(L1)
trans(L2)trans(L3)
rot(J1)
rot(J2)
rot(J3)
13
Mechanical Linkage Tracking Mechanical Linkage Tracking Advantages and DisadvantagesAdvantages and Disadvantages
Advantagesvery high quality tracking with low dynamic distortioncan support heavy displays (e.g., CRTs for head tracking)mature off-the-shelf technology
Disadvantagesrestricted motion
14
Video/Optical TrackingVideo/Optical Tracking
Video cameras monitor environmenttrack active LEDs or passive retroreflectivemarkers
“Outside-in”: tracked user/object wears targets“Inside-out”: tracked user/object wears cameras
E.g., ARToolkit [www.hitl.washington.edu/shared_space], Northern Digital [www.ndigital.com], Phoenix Technologies [www.ptiphoenix.com], 3rdTech [www.3rdtech.com]
15
Video/Optical Tracking Video/Optical Tracking Advantages and DisadvantagesAdvantages and Disadvantages
Advantagesvery high qualitycan use head-worn cameras for video see-through displaypotentially supports object recognition
Disadvantagesrequires clear line-of-sightmulti-camera configurations can be difficult to calibrate
16
““SourcelessSourceless” Tracking” Tracking
17
EarthEarth--Relative TrackingRelative Tracking
magnetometer determines yaw from earth’s magnetic fieldinclinometers determine roll and pitch from earth’s gravitational field
E.g., Precision Navigation [www.precisionnav.com]
18
EarthEarth--Relative Tracking Relative Tracking Advantages and DisadvantagesAdvantages and Disadvantages
Advantagesno source needed (besides the earth!)lightweight (< 2 oz.)inexpensive
Disadvantagessusceptible to magnetic field anomalies (large metal objects) and changes over timeangle limits caused by inclinometer and/or magnetometer gimbals
19
Inertial TrackingInertial TrackingLinear accelerometersAngular rate gyroscopesAdvantages
unlimited range
Disadvantagescurrently very difficult/expensive to make accurate systemserror accumulates over time
20
Satellite Position TrackingSatellite Position TrackingConstellations of US (GPS)/Russian (GLONASS) satellitesRadio receiver(s) determine distance from multiple satellites (ca. 10m accuracy)Differential GPS uses correction signal broadcast from receiver at preciselyknown location (ca. submeter accuracy)RTK carrier-phase measurement GPS(ca. cm accuracy)
E.g., Trimble [www.trimble.com], Ashtech [www.ashtech.com]
21
Satellite Position Tracking Satellite Position Tracking Advantages and DisadvantagesAdvantages and Disadvantages
AdvantagesVery long range
Disadvantagesmust be within line-of-sight of satellites (not indoors or in “urban canyons”)poor altitude trackinggovernments own satellites (encryption, “selective availability”, politics)low update rate
22
Hybrid TrackersHybrid Trackers
Combine different technologiesExamples
Inertial tracker (fast, but drifts) andultrasonic tracker (–slower, +ground truth, →position and orientation): InterSense IS 900magnetometer (–slower, +ground truth, +sourceless, → orientation): InterSense IS 300www.isense.com
Orientation tracker, GPS, and dead reckoning: Point Research Systems
23
Research Challenges:Research Challenges:UI DesignUI Design
What should we see, hear, feel, . . .?How should we interact with it?
24
Windows on the World Windows on the World 19931993
X11 serverTrack head/body/hand and selected objectsUser “wears” virtual information surroundWindows fixed to display, surround, worldHypermedia links between physical and virtual objects
25
Architectural Anatomy Architectural Anatomy 19951995Webster, Krueger, MacIntyre, & KellerWebster, Krueger, MacIntyre, & Keller
User examines building support systems in situ3D graphics reveal hidden physical elements2D graphics overlay output of analysis tools
26
Field Service Field Service 19961996
“Crossbox” telecom junction panel connects clients to main switch
27
Field Service Field Service 19961996
3D graphics identify groups of connection posts2D windows describe connection status2D windows can be pulled in/out to avoid clutter
28
Field Service: Field Service: Telecom Telecom CrossboxCrossboxNormal viewNormal view
29
Field Service: Field Service: Telecom Telecom CrossboxCrossboxAugmented view: Blocks of postsAugmented view: Blocks of posts
30
Field Service: Field Service: Telecom Telecom CrossboxCrossboxAugmented view: Selected group connected to noteAugmented view: Selected group connected to note
31
Field Service: Field Service: Telecom Telecom CrossboxCrossboxAugmented view: Note “pulled in” for inspectionAugmented view: Note “pulled in” for inspection
32
Field Service: Field Service: Telecom Telecom CrossboxCrossboxAugmented view: Additional notesAugmented view: Additional notes
33
ARC: Augmented Reality for ARC: Augmented Reality for Construction Construction 19961996––9898(Joint with Building Technologies Lab)(Joint with Building Technologies Lab)
Assist worker in assembling spaceframe buildingGoal
Right pieceRight placeRight sequence
34
SpaceframeSpaceframe Construction Construction TaskTaskfor each spaceframe piece in sequence {
select piece and verify identityinstall pieceverify location and identity
}
35
User InterfaceUser Interface
See-through head-worn display
provides visual andaudio instructions
Display and reader are tracked
Bar code readerverifies part identity and placement
36
Real WorldReal World
37
Virtual WorldVirtual World
38
Real World +Real World +Virtual WorldVirtual World
39
Touring Machine Touring Machine 19961996––
Assist mobile user in exploring an unfamiliar environmentUser interface
See-through head-worn displayHand-held pen computer/tablet
Backpack containscomputer with 3D graphics carddifferential GPS system spread-spectrum radio modem
40
UI: Overlaid LabelsUI: Overlaid Labels
41
UI: Overlaid LabelsUI: Overlaid Labels
42
Touring Machine: Columbia CampusTouring Machine: Columbia Campus
43
Touring Machine: Columbia CampusTouring Machine: Columbia Campus
44
Touring Machine: Columbia CampusTouring Machine: Columbia Campus
45
Touring Machine: Columbia CampusTouring Machine: Columbia Campus
46
Touring Machine: Columbia CampusTouring Machine: Columbia Campus
47
Touring Machine: Columbia CampusTouring Machine: Columbia Campus
48
Research Challenges:Research Challenges:Environment ManagementEnvironment Management
Task of managinglarge numbers of objects onlarge numbers of displays forlarge numbers of users
Develop methods toorganize, filter, modify, navigate
interactive information spaces that arelarge, shared, 3D, multimedia
49
Environment Management Environment Management ArchitectureArchitecture
World,User,Hdw,
Situation,Tasks
ARUI
InformationFiltering
Choose data to display
UI ComponentDesign
Map data to UI components
ViewManagement
Lay out UI components
Dynamically redesign UI
50
Information Filtering Information Filtering 20002000(Imaged with Naval Research Lab BARS)(Imaged with Naval Research Lab BARS)
Unfiltered: All objects shown Filtered: Only objects relevant to route-finding task are shown
51
UI Component Design UI Component Design 20012001
Precise position tracking: labels registered with real world
Imprecise position tracking: scaled world model displayed
52
Arbitrary viewpointArbitrary scene configurationDynamic environment
ProblemProblem
→Annotation soup
53
View Management View Management 20012001
Dynamic layout of 3D information to avoid occlusion, overlap:
LabelsAnnotationsUI components
54
View ManagementView ManagementNaïve AnnotationNaïve Annotation
Annotation based on centroidsMisplaced, overlapped, ambiguous
55
View ManagementView ManagementAutomated AnnotationAutomated Annotation
Annotation based on centroidsMisplaced, overlapped, ambiguous
Annotation using new approachCorrectly placed, overlap and ambiguity avoided
56
View ManagementView ManagementCampus OverviewCampus Overview