3D User Interface Design for Virtual Environments
Doug A. BowmanEdited by C. Song
(C) 2005 Doug Bowman, Virginia Tech 2
Background: Human-computer interaction HCI studies communication Users and computers communicate via the
interface 2D HCI issues:
Direct manipulation vs. commands Interface layout Interaction techniques Cognitive directness
3D
(C) 2005 Doug Bowman, Virginia Tech 3
Why 3D interaction?
3D / VE apps. should be useful ImmersionNatural skills Immediacy of visualization
But, current VE apps eitherare not complex interactively, orhave serious usability problems
(C) 2005 Doug Bowman, Virginia Tech 4
What makes 3D interaction difficult? Spatial input Lack of constraints Lack of standards Lack of tools
Lack of precision Fatigue Layout more
complex Perception
(C) 2005 Doug Bowman, Virginia Tech 5
Isn’t the 3D interface obvious?
Naturalism: make VE work exactly like real world
Magic: give user new abilitiesPerceptualPhysicalCognitive
(C) 2005 Doug Bowman, Virginia Tech 6
Goals of interaction design
Performance (efficiency, accuracy, productivity)
Usability (ease of use, ease of learning, user comfort)
Usefulness (users focus on tasks, interaction helps users meet system goals)
(C) 2005 Doug Bowman, Virginia Tech 7
Outline
System controlSymbolic input2D interaction in VEsConstraintsPassive haptic feedbackTwo-handed interaction
(C) 2005 Doug Bowman, Virginia Tech 8
8. System control
Issuing a command toChange the system modeChange the system state
Often composed of other tasksSometimes seen as a “catch-all” for 3D
interaction techniques other than travel, selection, & manipulation
(C) 2005 Doug Bowman, Virginia Tech 9
Classification of system control techniques
(C) 2005 Doug Bowman, Virginia Tech 10
Floating menus
Can occlude environment
Familiar interaction Using 3D selection for a
1D task Attach the menu
User’s headTracked physical surface (a tablet)
(C) 2005 Doug Bowman, Virginia Tech 11
1 DOF menu
Ring menu & selection baseket
Correct number of DOFs for the task
Can be put awayOnly one menu level at a
time
(C) 2005 Doug Bowman, Virginia Tech 12
TULIP menus
Goal: display all options but retain efficiency and comfort
Pinch Gloves: Three-up, Labels In Palm (TULIP) only three items available for
selection other items appear in sets of
three on the palm “more” item linked to next set
(C) 2005 Doug Bowman, Virginia Tech 13
Command and control cube (C3)
(C) 2005 Doug Bowman, Virginia Tech 14
Gestural commands
Can be “natural” Limited vocabulary Lack of affordances Fuzzy recognition issues Gesture as command - doesn’t mimic our use of
gestures in the real world Pen-based gestures can be powerful Perhaps more appropriate in multimodal interfaces
(C) 2005 Doug Bowman, Virginia Tech 15
System control design guidelines
Don’t disturb flow of actionUse consistent spatial reference Allow multimodal inputStructure available functionsPrevent mode errors by giving feedback
(C) 2005 Doug Bowman, Virginia Tech 16
9. Symbolic input
Communication of symbols (text, numbers, and other symbols/marks) to the system
Is this an important task for 3D UIs?Why is symbolic input in 3D UIs different
from symbolic input elsewhere?
(C) 2005 Doug Bowman, Virginia Tech 17
Symbolic input scenarios
Which technique(s) would you choose when… … architects will be making annotations to various
parts of a design? … engineers need to enter file names to save their
work? … an AR user needs to add hundreds of labels to
an environment?
(C) 2005 Doug Bowman, Virginia Tech 18
Symbolic input techniquesKeyboard-based
Miniature keyboards Low key-count keyboards Chord keyboards Pinch Keyboard Soft keyboards
Pen-based Pen-stroke gesture
recognition Digital Ink
(C) 2005 Doug Bowman, Virginia Tech 19
Symbolic input techniques 2
Gesture-based Sign language gestures Numeric gestures Instantaneous gestures
Speech-based Single-character speech recognition Whole-word speech recognition Unrecognized speech input
(C) 2005 Doug Bowman, Virginia Tech 20
Pinch keyboard video
(C) 2005 Doug Bowman, Virginia Tech 21
10. Usability Engineering
Methodology for constructing 3D UIsIterative process
Requirement gatheringDesign (Chapter 10)Prototype buildingEvaluation (Chapter 11)
(C) 2005 Doug Bowman, Virginia Tech 22
2D interaction in 3D VEs
Quite useful for appropriate tasks (match task and input DOFs)
Can integrate seamlessly with 3D If presence is important, the 2D interface should be embedded, not overlaid
Examples: Interaction on the projection surface or viewplane Using a PDA for VE input
(C) 2005 Doug Bowman, Virginia Tech 23
Constraints
Artificial limitations designed to help users interact more precisely or efficiently
Examples:Snap-to grid Intelligent objectsSingle DOF controls
(C) 2005 Doug Bowman, Virginia Tech 24
Passive haptic feedback
Props or “near-field” hapticsExamples:
Flight simulator controlsPirates’ steering wheel, cannonsElevator railing
Increase presence, improve interaction
(C) 2005 Doug Bowman, Virginia Tech 25
Two-handed interaction
Symmetric vs. AsymmetricDominant vs. non-dominant handGuiard’s principles
ND hand provides frame of referenceND hand used for coarse tasks, D hand for
fine-grained tasksManipulation initiated by ND hand
(C) 2005 Doug Bowman, Virginia Tech 26
Two-handed interaction examples
(C) 2005 Doug Bowman, Virginia Tech 27
Pen & tablet interaction
Involves 2D interaction, two-handed interaction, constraints, and props
(C) 2005 Doug Bowman, Virginia Tech 28
Pen & tablet interaction
Can put away Constrained surface
for input Combine 2D/3D
interaction
Handwriting input? Use any type of 2D
interface, not just menus
(C) 2005 Doug Bowman, Virginia Tech 29
QuickTime™ and a MPEG-4 Video decompressor are needed to see this picture.
Usability one of the most crucial issues facing VE applications
Implementation details critical to ensure usability
Ease of coding not equal to ease of useSimply adapting 2D interfaces is not
sufficient
Conclusions
(C) 2005 Doug Bowman, Virginia Tech 30
More work needed on…
System control performance Symbolic input and markup Usability evaluation methods Mapping interaction techniques to devices Integrating interaction techniques into
complete UIs
(C) 2005 Doug Bowman, Virginia Tech 31
Resources
3DUI groupMailing listAnnotated bibliographywww.mic.atr.co.jp/~poup/3dui.html
3DI research at VT research.cs.vt.edu/3di/
(C) 2005 Doug Bowman, Virginia Tech 32
Acknowledgments
Joe LaViola, Brown UniversityErnst Kruijff, GMDIvan Poupyrev, Sony CSL