View
2
Download
0
Category
Preview:
Citation preview
1
A Toolkit for Managing User Attention in Peripheral DisplaysTara Matthews, Anind K. Dey*٭, Jennifer Mankoff٭, Scott Carter, Tye Rattenbury
EECS Department, UC Berkeley*Intel-Berkeley Research
currently of HCII, Carnegie Mellon٭
10/27/2004 Peripheral Display Toolkit 2
Problematic Scenario• Bus schedule
– Requires a task switch to use– Doesn’t afford continuous monitoring
RTE LAWR GRIZ EUCL CNTRNUM HALL SR. GRIZ SHAT65 605a 616a 630a 65 635a 646a 700a 65 703a 716a 730a 65 735a 746a 800a 65 803a 816a 830a 65 835a 846a 900a 65 903a 916a 930a 65 935a 946a 1000a 65 1003a 1016a 1030a 65 1035a 1046a 1100a 65 1103a 1116a 1130a 65 1135a 1146a 1200p 65 1203p 1216p 1230p 65 1235p 1246p 100p 65 103p 116p 130p 65 135p 146p 200p 65 203p 216p 230p 65 235p 246p 300p65 303p 316p 330p 65 335p 346p 400p
10/27/2004 Peripheral Display Toolkit 3
Solution: Peripheral Displays• Provide awareness with min attention• Separate from primary task• Bus LED Display
+ Get bus info at a glance+ Dynamically manages attention it attracts– Hard to accomplish
10/27/2004 Peripheral Display Toolkit 4
Why is creating PDs hard?• Need to abstract info to be glance-able• Need mechanisms for dynamically
managing attention PDs attract:– Deciding attention levels to attract
(notification levels)– Displaying info appropriately (transitions)
• Our contribution:– Peripheral Display Toolkit (PTK) supports
these key issues in PD creation
2
10/27/2004 Peripheral Display Toolkit 5
Example PTK Applications• Remote Activity
– Social Guitar– Audio Monitor– Motion Monitor– Remote Awareness Display
• Bus Displays– Bus Mobile– Bus LED
• Instant Messenger Status
IM Picture FrameSocial Guitar Bus LED BusMobile
Orb showing remote activity
10/27/2004 Peripheral Display Toolkit 6
Example PTK Applications• Remote Activity
– Social Guitar– Audio Monitor– Motion Monitor– Remote Awareness Display
• Bus Displays– Bus Mobile– Bus LED
• Instant Messenger Status
IM Picture FrameSocial Guitar Bus LED BusMobile
Orb showing remote activity
10/27/2004 Peripheral Display Toolkit 7
Example PTK Applications• Remote Activity
– Social Guitar– Audio Monitor– Motion Monitor– Remote Awareness Display
• Bus Displays– Bus Mobile– Bus LED
• Instant Messenger Status
IM Picture FrameSocial Guitar Bus LED BusMobile
Orb showing remote activity
10/27/2004 Peripheral Display Toolkit 8
Example PTK Applications• Remote Activity
– Social Guitar– Audio Monitor– Motion Monitor– Remote Awareness Display
• Bus Displays– Bus Mobile– Bus LED
• Instant Messenger Status
IM Picture FrameSocial Guitar Bus LED BusMobile
Orb showing remote activity
3
10/27/2004 Peripheral Display Toolkit 9
Example PTK Applications+ Manage attention with
abstraction, notification levels, & transitions
+ Glance-able+ Few lines of code+ Simplified code design
& code re-use
IM Picture FrameSocial Guitar Bus LED BusMobile
Orb showing remote activity
10/27/2004 Peripheral Display Toolkit 10
Simple Example: Motion Monitor
no/low motion
medium motion
high motion
10/27/2004 Peripheral Display Toolkit 11
Outline• PTK: Motivation & Examples• Managing attention
– Abstraction– Notification levels– Transitions
• Toolkit Architecture & Library• Conclusion
10/27/2004 Peripheral Display Toolkit 12
Managing User Attention• Attention is a crucial issue in design
– Significant impact on success– More info can be monitored w/o overburdening
• Attention management is difficult– Used to designing apps that attract full attention– Attention attracted should match info urgency
• Literature surveys led to key characteristics of attention management
info urgency user attention
Attentionnone focal
4
10/27/2004 Peripheral Display Toolkit 13
Attention Management Characteristics
info urgency user attention
From survey of peripheral displays:1. Abstraction
• enables lower-attention info consumption2. Notification levels
• help determine how much attention to attract based on urgency of info
3. Transitions• enable display of info to match urgency
10/27/2004 Peripheral Display Toolkit 14
Abstraction• Enables lower-attention consumption of info• Extracting features or reducing fidelity of info• Motion Monitor: replaces video images with
general motion amount (high, medium, none)
video abstraction amount ofimages motion
• high motion
• medium motion
• no/low motion
10/27/2004 Peripheral Display Toolkit 15
Notification Levels• Indicate the urgency of info
– Low levels = non-urgent info– High levels = more urgent info
• Motion Monitor: doesn’t show urgent info, so uses low notification levels
10/27/2004 Peripheral Display Toolkit 16
More on Notification Levels
• Attention is (roughly) scalar – PD designers must decide where on the scale displayed info should fall
0% 100%attention
awareness
unconscious
5
10/27/2004 Peripheral Display Toolkit 17
More on Notification Levels
Attention Type1. None
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 18
More on Notification Levels
Attention Type1. None
Notification LevelIgnore
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 19
More on Notification Levels
Attention Type1. None2. Inattention
Notification LevelIgnore
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 20
More on Notification Levels
Attention Type1. None2. Inattention
Notification LevelIgnoreChange blind
0% 100%attention
awareness
unconscious
6
10/27/2004 Peripheral Display Toolkit 21
More on Notification Levels
Attention Type1. None2. Inattention3. Divided
Notification LevelIgnoreChange blind
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 22
More on Notification Levels
Attention Type1. None2. Inattention3. Divided
Notification LevelIgnoreChange blindMake aware
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 23
More on Notification Levels
Attention Type1. None2. Inattention3. Divided4. Focused
Notification LevelIgnoreChange blindMake aware
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 24
More on Notification Levels
Notification LevelIgnoreChange blindMake awareInterrupt
Attention Type1. None2. Inattention3. Divided4. Focused
0% 100%attention
awareness
unconscious
7
10/27/2004 Peripheral Display Toolkit 25
More on Notification Levels
Notification LevelIgnoreChange blindMake awareInterruptDemand action
Attention Type1. None2. Inattention3. Divided4. Focused5. Focused + interaction
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 26
Transitions• Enable display of info to match
urgency• Depend on notification level
– Ignore: no changes– Change blind: fading, slow motion, tiny updates– Make aware: discrete, abrupt updates– Interrupt: flashing, beeping, vibration– Demand action: interrupt until user does action
• Motion Monitor: varies pulse speeds for different notification levels
10/27/2004 Peripheral Display Toolkit 27
Outline• PTK: Motivation & Examples• Managing attention
– Abstraction– Notification levels– Transitions
• Toolkit Architecture & Library• Conclusion
10/27/2004 Peripheral Display Toolkit 28
PTK Architecture1. Support for managing impact on
human attention using abstraction, notification levels, and transitions
2. Simplified code design and code re-use
3. Library of common PD components
Output
Transition
Input
NotificationMap
Abstractor
8
10/27/2004 Peripheral Display Toolkit 29
Motion Monitor Architecture
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidgetTrans
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidgetTrans OrbWidgetTrans
10/27/2004 Peripheral Display Toolkit 30
Motion Monitor Architecture
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidgetTrans
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidgetTrans OrbWidgetTrans
10/27/2004 Peripheral Display Toolkit 31
Motion Monitor Architecture
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidgetTrans
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidgetTrans OrbWidgetTrans
motion = medium
10/27/2004 Peripheral Display Toolkit 32
Motion Monitor Architecture
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidgetTrans
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidgetTrans OrbWidgetTrans
motion = medium
notification level = change blind
9
10/27/2004 Peripheral Display Toolkit 33
Motion Monitor Architecture
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidgetTrans
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidgetTrans OrbWidgetTrans
motion = medium
notification level = change blind
10/27/2004 Peripheral Display Toolkit 34
Motion Monitor Architecture
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidgetTrans
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidgetTrans OrbWidgetTrans
5 lines of logic code
10/27/2004 Peripheral Display Toolkit 35
Remote Awareness Display
Count # people,How much motion?
Phone ring?Voices?
Add notification level
10/27/2004 Peripheral Display Toolkit 36
Motion Monitor Remote Awareness Display
Input-Side Output-Side
Motion Monitor
DiscoveryServer
CameraInput PTK
DiscoveryServer
MotionAbstractor
Exact Match(Motion) Orb
WidgetTrans
Input-Side Output-Side
Motion Monitor
DiscoveryServer
CameraInput PTK
DiscoveryServer
MotionAbstractor
Exact Match(Motion) Orb
WidgetTrans
10
10/27/2004 Peripheral Display Toolkit 37
Motion Monitor Remote Awareness Display
Input-Side Output-SideDiscoveryServer
CameraInput PTK
DiscoveryServer
MotionAbstractor
Exact Match(Motion) Orb
WidgetTrans
Motion Monitor
Input-Side Output-SideDiscoveryServer
CameraInput PTK
DiscoveryServer
MotionAbstractor
Exact Match(Motion) Orb
WidgetTrans OrbWidgetTrans
Motion Monitor
10/27/2004 Peripheral Display Toolkit 38
Motion Monitor Remote Awareness Display
Input-Side Output-SideDiscoveryServer
CameraInput PTK
DiscoveryServer
MotionAbstractor
# PeopleAbstractor
Exact Match(Motion) Orb
WidgetTrans
Motion Monitor
Input-Side Output-SideDiscoveryServer
CameraInput PTK
DiscoveryServer
MotionAbstractor
# PeopleAbstractor
Exact Match(Motion) Orb
WidgetTrans OrbWidgetTrans
Motion Monitor
10/27/2004 Peripheral Display Toolkit 39
Motion Monitor Remote Awareness Display
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion) Orb
WidgetTrans
Motion Monitor
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion) Orb
WidgetTrans OrbWidgetTrans
Motion Monitor
10/27/2004 Peripheral Display Toolkit 40
Motion Monitor Remote Awareness Display
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion)
Exact Match(Audio)
Threshold(# People)
OrbWidgetTrans
Motion Monitor
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion)
Exact Match(Audio)
Threshold(# People)
OrbWidgetTrans OrbWidgetTrans
Motion Monitor
11
10/27/2004 Peripheral Display Toolkit 41
Motion Monitor Remote Awareness Display
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion)
Exact Match(Audio)
Threshold(# People)
OrbWidgetTrans
Orb Wrapper
Motion Monitor
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion)
Exact Match(Audio)
Threshold(# People)
OrbWidgetTrans OrbWidgetTrans
Orb Wrapper
Motion Monitor
10/27/2004 Peripheral Display Toolkit 42
Remote Awareness Display
54 – total lines of code
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion)
Exact Match(Audio)
Threshold(# People)
OrbWidgetTrans
Orb Wrapper
TickerWidgetTrans
Tikr. Wrapper
PTKDiscovery
Server
Remote Awareness Display
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion)
Exact Match(Audio)
Threshold(# People)
OrbWidgetTrans OrbWidgetTrans
Orb Wrapper
TickerWidgetTrans TickerWidgetTrans
Tikr. Wrapper
PTKDiscovery
Server
Remote Awareness Display
10/27/2004 Peripheral Display Toolkit 43
General Architecture DiagramInput-Side Discovery
ServerOutput-Side
Peripheral Display 1(Abstractors) (Notification
Maps)(Output Widgets w/ optional Transition)
Peripheral Display 2
Input 1
Input 2
Input N
Abs. 1
Abs. N PTKDiscovery
Server
Abs. 2
Abs. N
Abs. 1 N.M. 1
N.M. 2
N.M. N
Out 1Trans
Out 2
Out N
... …
Abs. 2
Abs. N
Abs. 1 N.M. 1
N.M. 2
N.M. N
Out 2Trans
Out 1
Out N
…
Input-Side DiscoveryServer
Output-Side
Peripheral Display 1(Abstractors) (Notification
Maps)(Output Widgets w/ optional Transition)
Peripheral Display 2
Input 1
Input 2
Input N
Abs. 1
Abs. N PTKDiscovery
Server
Abs. 2
Abs. N
Abs. 1 N.M. 1
N.M. 2
N.M. N
Out 1Trans Out 1Trans
Out 2
Out N
... …
Abs. 2
Abs. N
Abs. 1 N.M. 1
N.M. 2
N.M. N
Out 2Trans Out 2Trans
Out 1
Out N
…
10/27/2004 Peripheral Display Toolkit 44
Library Components• Input
– audio, camera, Phidgets, Context Toolkit, online calendars, news, stocks, weather,Web page parser, serial port communication
12
10/27/2004 Peripheral Display Toolkit 45
Library Components• Input
– audio, camera, Phidgets, Context Toolkit, online calendars, news, stocks, weather,Web page parser, serial port communication
• Output– ticker text, Ambient Orb, Phidgets
10/27/2004 Peripheral Display Toolkit 46
Library Components• Input
– audio, camera, Phidgets, Context Toolkit, online calendars, news, stocks, weather,Web page parser, serial port communication
• Output– ticker text, Ambient Orb, Phidgets
• Abstractors– motion, people counting, voices, phone ringing
10/27/2004 Peripheral Display Toolkit 47
Library Components• Input
– audio, camera, Phidgets, Context Toolkit, online calendars, news, stocks, weather,Web page parser, serial port communication
• Output– ticker text, Ambient Orb, Phidgets
• Abstractors– motion, people counting, voices, phone ringing
• Notification– exact match, threshold, contains,
degree of change
10/27/2004 Peripheral Display Toolkit 48
Transitions• PTK provides architectural support
for implementing as animations• Timing• Number of steps• Flashing (single, repeated)
• Tied to output– Depend on output modality– Example transitions: color changing and
pulsing (orb); fading, motion, & flashing (ticker text)
13
10/27/2004 Peripheral Display Toolkit 49
Future Work & Conclusion• PTK supports the creation of PDs• Focuses on managing user attention• To do this, uses abstraction,
notification levels, transitions• Validated with 7 example applications
– Supports attention management– Facilitates code re-use
• Future work– Incorporate interruptibility and
local context sensing– Cross-modality animations– PD evaluation support
10/27/2004 Peripheral Display Toolkit 50
Questions?
• For more information & downloads:
www.cs.berkeley.edu/projects/io/ptk/
tmatthew@cs.berkeley.edu
Thanks to…NSF, Ed De Guzman, Jeff Heer,Scott Klemmer, Berkeley & CMU
10/27/2004 Peripheral Display Toolkit 51
What about user context?• PDs must 2 sets of attention issues:
– User context• interruptibility, primary activity, focus of attention
– Attention management• balancing info urgency and user attention
• PTK focuses on attention management– User context invasive to sense, not practical to
assume PDs have access– Management issues are independent of user
context • Context input can be considered on a
library level (in notification maps)
10/27/2004 Peripheral Display Toolkit 52
Related Work• AROMA: architectural support for abstraction
• Context Toolkit: infrastructure for context input and some abstraction
• Phidgets: library of hardware widgets for input and output
• Real World Interfaces Toolkit: library of X10 output widgets
• iStuff: infrastructure for distributed inputs and outputs
Recommended