Upload
francis-stafford
View
215
Download
2
Tags:
Embed Size (px)
Citation preview
1
User Interfaces at Microsoft Research
Intelligent Information Access using Intelligent Information Access using
Animated 2 and 3D Information Animated 2 and 3D Information
VisualizationVisualization
Intelligent Information Access using Intelligent Information Access using
Animated 2 and 3D Information Animated 2 and 3D Information
VisualizationVisualization
Mary Czerwinski
2
A Sweet Spot? Use 3D VisualsUse 3D Visuals
Apply 3D cuesApply 3D cues
Occlusion, Relative Size, ShadowsOcclusion, Relative Size, Shadows
Spatialized AudioSpatialized Audio
2D Interaction technique2D Interaction technique
Little or no egocentric navigationLittle or no egocentric navigation
Objects come to the userObjects come to the user
Use 3D VisualsUse 3D Visuals
Apply 3D cuesApply 3D cues
Occlusion, Relative Size, ShadowsOcclusion, Relative Size, Shadows
Spatialized AudioSpatialized Audio
2D Interaction technique2D Interaction technique
Little or no egocentric navigationLittle or no egocentric navigation
Objects come to the userObjects come to the user
3
Data Mountain
Subject Layout of 100 Pages
“Strongest cue ...relative size”
4
Video
Data MountainData MountainData MountainData Mountain
5
Data Mountain Usability
Study #1 (Compare with IE4 Favorites)Study #1 (Compare with IE4 Favorites)
Reliably faster (26%)Reliably faster (26%)
Study #2 (Longevity and Thumbnails)Study #2 (Longevity and Thumbnails)
After 6 months, no performance changeAfter 6 months, no performance change
Images help, but are not requiredImages help, but are not required
Study #1 (Compare with IE4 Favorites)Study #1 (Compare with IE4 Favorites)
Reliably faster (26%)Reliably faster (26%)
Study #2 (Longevity and Thumbnails)Study #2 (Longevity and Thumbnails)
After 6 months, no performance changeAfter 6 months, no performance change
Images help, but are not requiredImages help, but are not required
6
“Thumbless” Data Mountain
Figure 2: Same Data Mountain, without thumbnails.
7
Figure 3. Average retrieval times to find 100 web pages.
1st vs 2nd visit vs 2nd visit-no images:RT
0
5
10
15
20
25
30
Thumbnail Title Summary All
Retrieval Cue
Re
acti
on
Tim
e (
se
c) 1st visit
2nd visit
2nd visit, no images
Study #2 Results
8
Figure 6. Average retrieval time by thumbnailcondition and trial block (1 block=10 trials).
Thumbnail Condition by Block
0
5
10
15
20
25
30
35
1 2 3 4 5
Block
Avg.
RT
(sec
onds
)
Thumbnails
No Thumbnails
Study #2 Results
9
Visualizing Implicit Queries 2 studies:2 studies:
on managementon management
on retrievalon retrieval
Visualized with DMVisualized with DM
3D, spatial layout 3D, spatial layout of web pagesof web pages
Pages similar to Pages similar to selected page are selected page are highlightedhighlighted
2 studies:2 studies:
on managementon management
on retrievalon retrieval
Visualized with DMVisualized with DM
3D, spatial layout 3D, spatial layout of web pagesof web pages
Pages similar to Pages similar to selected page are selected page are highlightedhighlighted
10
Implicit Queries (IQ): Performance Benefits
IQ users (IQ1 & 2) IQ users (IQ1 & 2)
built more categoriesbuilt more categories
took longer organizingtook longer organizing
faster finding pagesfaster finding pages
Retrieval far more Retrieval far more frequent implies frequent implies IQ faster overallIQ faster overall
IQ users (IQ1 & 2) IQ users (IQ1 & 2)
built more categoriesbuilt more categories
took longer organizingtook longer organizing
faster finding pagesfaster finding pages
Retrieval far more Retrieval far more frequent implies frequent implies IQ faster overallIQ faster overall
Web Page Retrieval Time
0
2
4
6
8
10
12
14
Implicit Query Condition
Ave
rag
e R
T (
sec
on
ds
)
IQ 0
IQ 1
IQ 2
Figure 3. Average retrieval times (includingstandard error of the mean) for the 3 IQ conditions.IQ1,2 use different similarity metrics
11
Glances
Novel 3D navigation techniqueNovel 3D navigation technique
Lightweight, ephemeralLightweight, ephemeral
Two-handedTwo-handed
Non-dominant hand used for glance gestureNon-dominant hand used for glance gesture
Low-cost awareness of surrounding areaLow-cost awareness of surrounding area
Novel 3D navigation techniqueNovel 3D navigation technique
Lightweight, ephemeralLightweight, ephemeral
Two-handedTwo-handed
Non-dominant hand used for glance gestureNon-dominant hand used for glance gesture
Low-cost awareness of surrounding areaLow-cost awareness of surrounding area
12
Toolspaces Objects attached to Objects attached to
virtual bodyvirtual body
Glances used to get Glances used to get to toolspacesto toolspaces
Multiple usesMultiple uses
3D widgets3D widgets
Information storeInformation store
Object transportObject transport
Objects attached to Objects attached to virtual bodyvirtual body
Glances used to get Glances used to get to toolspacesto toolspaces
Multiple usesMultiple uses
3D widgets3D widgets
Information storeInformation store
Object transportObject transport
13
Task Gallery
3D Shell for Windows 3D Shell for Windows
Task ManagementTask Management
Simple, forward-back Simple, forward-back navigationnavigation
Tasks laid out spatially Tasks laid out spatially on floor, ceiling, wallson floor, ceiling, walls
Simple task switchSimple task switch
3D Shell for Windows 3D Shell for Windows
Task ManagementTask Management
Simple, forward-back Simple, forward-back navigationnavigation
Tasks laid out spatially Tasks laid out spatially on floor, ceiling, wallson floor, ceiling, walls
Simple task switchSimple task switch
14
Task Gallery
Simultaneous Simultaneous viewing of viewing of multiple windowsmultiple windows
Simple shift selectSimple shift select
Smart arrangementSmart arrangement
Use 3D to provide Use 3D to provide uniform scalinguniform scaling
Simultaneous Simultaneous viewing of viewing of multiple windowsmultiple windows
Simple shift selectSimple shift select
Smart arrangementSmart arrangement
Use 3D to provide Use 3D to provide uniform scalinguniform scaling
15
Task Gallery Tools Glance Left for Start Glance Left for Start
PalettePalette
Glance Right for Glance Right for Windows ExplorerWindows Explorer
Glance Up or Down Glance Up or Down for status and for status and notificationnotification
Glance Left for Start Glance Left for Start PalettePalette
Glance Right for Glance Right for Windows ExplorerWindows Explorer
Glance Up or Down Glance Up or Down for status and for status and notificationnotification
16
Task Gallery Windows stacked on Windows stacked on
podium or in space until podium or in space until selected for use via selected for use via “flicking”“flicking”
User studies improved User studies improved performanceperformance
Exploring other Exploring other spaces/metaphorsspaces/metaphors
Windows stacked on Windows stacked on podium or in space until podium or in space until selected for use via selected for use via “flicking”“flicking”
User studies improved User studies improved performanceperformance
Exploring other Exploring other spaces/metaphorsspaces/metaphors
17
Video
Task GalleryTask GalleryTask GalleryTask Gallery
18
Hierarchy VisualizationHierarchy Visualization
2D and 3D Research2D and 3D Research
Most frequently requested visualization Most frequently requested visualization solution from MS product teams solution from MS product teams
Hierarchy VisualizationHierarchy Visualization
2D and 3D Research2D and 3D Research
Most frequently requested visualization Most frequently requested visualization solution from MS product teams solution from MS product teams
19
Current Approaches
Windows Tree ControlWindows Tree Control
Many observed problems (from customers and usability Many observed problems (from customers and usability
testing)testing)
Many 2D and 3D hierarchy visualizationsMany 2D and 3D hierarchy visualizations
Each works for some tasks and some scalesEach works for some tasks and some scales
Windows Tree ControlWindows Tree Control
Many observed problems (from customers and usability Many observed problems (from customers and usability
testing)testing)
Many 2D and 3D hierarchy visualizationsMany 2D and 3D hierarchy visualizations
Each works for some tasks and some scalesEach works for some tasks and some scales
20
Problems: Fitting Data
Extreme aspect ratio (broad and shallow)Extreme aspect ratio (broad and shallow)
Balanced tree visualization not informativeBalanced tree visualization not informative
May be multiple hierarchiesMay be multiple hierarchies
Scaling issuesScaling issues
Extreme aspect ratio (broad and shallow)Extreme aspect ratio (broad and shallow)
Balanced tree visualization not informativeBalanced tree visualization not informative
May be multiple hierarchiesMay be multiple hierarchies
Scaling issuesScaling issues
21
Problems: Cognitive Overhead
Loss of context when looking at detailLoss of context when looking at detail
Loss of detail when looking at overviewLoss of detail when looking at overview
Separate detail/overview – extra overheadSeparate detail/overview – extra overhead
Which item is open?Which item is open?
Poor use of display spacePoor use of display space
Loss of context when looking at detailLoss of context when looking at detail
Loss of detail when looking at overviewLoss of detail when looking at overview
Separate detail/overview – extra overheadSeparate detail/overview – extra overhead
Which item is open?Which item is open?
Poor use of display spacePoor use of display space
22
Problems: Fitting User Task
Multiple foci of interest is difficultMultiple foci of interest is difficult
No integration of tree view with searchNo integration of tree view with search
Tree structure may not reflect or support Tree structure may not reflect or support
user task at handuser task at hand
Multiple foci of interest is difficultMultiple foci of interest is difficult
No integration of tree view with searchNo integration of tree view with search
Tree structure may not reflect or support Tree structure may not reflect or support
user task at handuser task at hand
23
Hierarchical Tree Control Problem
24
Basic View Strategies
Two view (separate detail/overview views)Two view (separate detail/overview views)
Distorted viewDistorted view
Distorted data: fisheyeDistorted data: fisheye
Distorted space: 3D, hyperbolicDistorted space: 3D, hyperbolic
Focus in Context (integrated view)Focus in Context (integrated view)
Two view (separate detail/overview views)Two view (separate detail/overview views)
Distorted viewDistorted view
Distorted data: fisheyeDistorted data: fisheye
Distorted space: 3D, hyperbolicDistorted space: 3D, hyperbolic
Focus in Context (integrated view)Focus in Context (integrated view)
25
Basic Visualization Approaches
IndentationIndentation
Tree controlTree control
FisheyeFisheye
ContainmentContainment
TreemapsTreemaps
Pad++Pad++
IndentationIndentation
Tree controlTree control
FisheyeFisheye
ContainmentContainment
TreemapsTreemaps
Pad++Pad++
ClusteringClustering
• Galaxy of NewsGalaxy of News
• ThemeScapeThemeScape
• Hot SauceHot Sauce
GeographicGeographic
• Floor plansFloor plans
• Street mapsStreet maps
26
Basic Visualization ApproachesNode-link diagramsNode-link diagrams
SemNetSemNet
2D diagrams2D diagrams
Cone TreeCone Tree
Fisheye Cone TreeFisheye Cone Tree
Hyperbolic viewerHyperbolic viewer
FSNFSN
XML3DXML3D
Node-link diagramsNode-link diagrams
SemNetSemNet
2D diagrams2D diagrams
Cone TreeCone Tree
Fisheye Cone TreeFisheye Cone Tree
Hyperbolic viewerHyperbolic viewer
FSNFSN
XML3DXML3D
27
Perceptualization GoalsEffective Use of Visual PerceptionEffective Use of Visual Perception
Utilize Visual Perception Characteristics Utilize Visual Perception Characteristics
Encode Documents along Multi-DimensionsEncode Documents along Multi-Dimensions
Add Shading/Transparency Cues to Increase Pre-attentive Add Shading/Transparency Cues to Increase Pre-attentive 3D Perception 3D Perception
Audio PerceptionAudio Perception
Haptic PerceptionHaptic Perception
Effective Use of Visual PerceptionEffective Use of Visual Perception
Utilize Visual Perception Characteristics Utilize Visual Perception Characteristics
Encode Documents along Multi-DimensionsEncode Documents along Multi-Dimensions
Add Shading/Transparency Cues to Increase Pre-attentive Add Shading/Transparency Cues to Increase Pre-attentive 3D Perception 3D Perception
Audio PerceptionAudio Perception
Haptic PerceptionHaptic Perception
28
Perceptive Senses AvailableReady to UseReady to Use
Visual Visual
3D Spatialization 3D Spatialization
Volume / Size Volume / Size
Color Color
Shape (curvature) Shape (curvature)
Opacity Opacity
Texture Texture
Haptic Haptic
Ready to UseReady to Use
Visual Visual
3D Spatialization 3D Spatialization
Volume / Size Volume / Size
Color Color
Shape (curvature) Shape (curvature)
Opacity Opacity
Texture Texture
Haptic Haptic
• Experimental – Proprioception
– Haptic Haptic
– Auditory
– Olfactory
29
XML3D
30
User Study-XML3D•Used Snap.com content; compared XML3D Used Snap.com content; compared XML3D to to www.snap.comwww.snap.com and 2D collapsible tree and 2D collapsible tree control UIcontrol UI
•Target end users were developers and Target end users were developers and maintainers of large web info spacesmaintainers of large web info spaces
•Used search tasksUsed search tasks
•Single or multiple parentsSingle or multiple parents
•New of existing categoriesNew of existing categories
•Used Snap.com content; compared XML3D Used Snap.com content; compared XML3D to to www.snap.comwww.snap.com and 2D collapsible tree and 2D collapsible tree control UIcontrol UI
•Target end users were developers and Target end users were developers and maintainers of large web info spacesmaintainers of large web info spaces
•Used search tasksUsed search tasks
•Single or multiple parentsSingle or multiple parents
•New of existing categoriesNew of existing categories
31
www.snap.com
32
Hierarchical Tree Control UI
33
Task Time Results: XML3D v. 2D UI’s
0
50
100
150
200
250
Existing/Single Existing/Multiple New /Single New /Multiple
Category X Parent Task Condition
Seco
nd
s XML3D
2D
34
Consistency, Tool Usage ResultsFor existing For existing categories, categories, XML3D benefits XML3D benefits do not sacrifice do not sacrifice quality of searchquality of search
Users preferred Users preferred 2D lists for new 2D lists for new category tasks. category tasks. Why?Why?
For existing For existing categories, categories, XML3D benefits XML3D benefits do not sacrifice do not sacrifice quality of searchquality of search
Users preferred Users preferred 2D lists for new 2D lists for new category tasks. category tasks. Why?Why?
XML3D Feature Usage
02468
1012
3D graph List 3D graph List
Existing New
Category
Avg
. # T
imes
Use
d
35
Direction…Using perceptual cues to provide important Using perceptual cues to provide important visual pattern information to uservisual pattern information to user
User can search or browseUser can search or browse
3D final arrangement can emphasize 3D final arrangement can emphasize multiple/user defined attributesmultiple/user defined attributes
Ready to drop in audio, haptic, clustering, Ready to drop in audio, haptic, clustering, implicit queryimplicit query
Using perceptual cues to provide important Using perceptual cues to provide important visual pattern information to uservisual pattern information to user
User can search or browseUser can search or browse
3D final arrangement can emphasize 3D final arrangement can emphasize multiple/user defined attributesmultiple/user defined attributes
Ready to drop in audio, haptic, clustering, Ready to drop in audio, haptic, clustering, implicit queryimplicit query
36
3D UI Conclusions Exploration of Desktop 3D “sweet spots” Exploration of Desktop 3D “sweet spots”
Perform well against existing GUIsPerform well against existing GUIs
Usability test informs design evolutionUsability test informs design evolution
Audio in 3D environment is usefulAudio in 3D environment is useful
Let users choose organizationLet users choose organization
Spatial memory and perceptual cues critical in Spatial memory and perceptual cues critical in Desktop 3DDesktop 3D
Exploration of Desktop 3D “sweet spots” Exploration of Desktop 3D “sweet spots”
Perform well against existing GUIsPerform well against existing GUIs
Usability test informs design evolutionUsability test informs design evolution
Audio in 3D environment is usefulAudio in 3D environment is useful
Let users choose organizationLet users choose organization
Spatial memory and perceptual cues critical in Spatial memory and perceptual cues critical in Desktop 3DDesktop 3D
37
3D User Interfaces Future Work
High value non-game apps:High value non-game apps:
Shell/Web/Mail/File BrowserShell/Web/Mail/File Browser
New 3D interaction techniquesNew 3D interaction techniques
Navigation and manipulationNavigation and manipulation
Input devices/techniquesInput devices/techniques
New visualization techniquesNew visualization techniques
Prove value of 3D and interactive animationProve value of 3D and interactive animation
High value non-game apps:High value non-game apps:
Shell/Web/Mail/File BrowserShell/Web/Mail/File Browser
New 3D interaction techniquesNew 3D interaction techniques
Navigation and manipulationNavigation and manipulation
Input devices/techniquesInput devices/techniques
New visualization techniquesNew visualization techniques
Prove value of 3D and interactive animationProve value of 3D and interactive animation