Matching Attentional Draw with Utility in Interruption

1Imager Laboratoryfor Graphics, Visualization and HCI

Matching Attentional Draw with Utility in Interruption

Jennifer Gluck, Andrea Bunt, Joanna McGrenereUniversity of British Columbia

CHI 2007 April 30, 2007

2 2Imager Laboratoryfor Graphics, Visualization and HCI

Matching Attentional Draw with Utility in Interruption?

• Attentional Draw (AD): How quickly attention is drawn to a notification

• Utility: Usefulness of the interruption content

• Matching: Low AD ↔ Low Utility … High AD ↔ High Utility


Interruption

When something causes a break in our actions, activities, or concentration.


Negative Effects

Lowered primary task performance Cutrell, Czerwinski, & Horvitz 2001 Gillie & Broadbent 1989

Increased anxiety, annoyance, perceived difficulty in completing primary task Bailey, Konstan, & Carlis, 2001


Positive Aspects

Technologies that interrupt are popular

suggests there is value to interruption

Other systems also have the potential to promote beneficial interruption…


Assist users by making suggestions that can help users complete a task

FlexExcel (Thomas & Krogseoter, 1993)

Adaptive Bar (Debevc, Meyer, Donlagic & Svecko 1996)

Mixed-Initiative Systems

How can we design interruption to emphasize positive aspects?


Design Guideline

Make the amount of attention attracted by a notification relative to the usefulness of the interruption content

Obermayer & Nugent, 2000McFarlane & Latorella,

2002


What did we do?Empirical investigation of the effectsof matching attentional draw (AD) of notification and interruption utility


Attentional Draw (AD)Empirical investigation of the effectsof matching attentional draw (AD) of notification and interruption utility

How quickly your attention is drawn to the notification signal

attentional draw


UtilityEmpirical investigation of the effectsof matching attentional draw (AD) of notification and interruption utility

How useful, important, or urgent the interruption content is to the recipient

“How helpful the interruption is in terms of performing a primary task”

utility


Findings

• Reduces negative effects • Facilitates positive perception of

interruption

Empirical investigation of the effectsof matching attentional draw (AD) of notification and interruption utility


Research Overview

Study 1: Find some signalsEstablished a set of significantly different notification signals in terms of their attentional draw (AD)

Study 2: Match the signals to utilityInvestigated matching AD and utility to see if this strategy can promote positive interruption


Primary Task








Goal: Establish a set of visual notification signals with significantly different detection times

Approach: Concurrent detection task Play Memory game Detect notification signals

Study 1


Notification Signals

10 signals: Transformations applied to a base icon

Properties explored Colour Motion Location


Signals

Time (seconds)


Study 1 Design

• Within-subjects• 12 subjects• 10 signals• 120 trials per subject


Study 1 Results: Detection Times

High AD signal: Follow

Med AD signal: Slow Zoom

Low AD signal: Flag


Goal:Investigate the effects of matching attentional draw and utility

In terms of: Annoyance Perceived benefit Performance

Study 2


Interrupting Task

Context-sensitive hints• 3 hint utility levels:

very helpful somewhat helpful not helpful

• Emulated a mixed-initiative system


QuickTime™ and aTechSmith EnSharpen decompressor

are needed to see this picture.


3 Game Conditions

1. Match• Match AD to utility

2. Static • Same AD for all utilities

3. Control• No interruption

Low utility hint ↔ Low AD signal

Med utility hint ↔ Med AD signal

High utility hint ↔ High AD signal

Low utility hint

Med utility hint ↔ Med AD signal

High utility hint


Study 2 Design

• Conditions: Match, Static, Control

• Within-subjects design

• 24 subjects

• 15 interruptions per condition


Study 2 Measures & Procedure

Procedure: Training, {Condition, Survey} x 3, Interview

Measures: Annoyance Perceived benefit Fatigue Performance (# of matches) Preference


Study 2 HypothesesH1: Interruption annoyance is lower in the

Matchcondition than in the Static condition

H2: Perceived benefit is higher in the Match condition than in the Static condition

H3: Performance is higher in the Match

conditionthan in all other conditions


Results: Annoyance & Benefit Annoyance lower in Match (p = .034) Benefit higher in Match (p = .037)

0

20

40

60

80

100

InterruptionAnnoyance

PerceivedBenefit

MatchStatic


Condition Preference

75%

15%10%

0%

20%

40%

60%

80%

100%

Match Static Control

% o

f Par

ticip

ants


Performance Results (# of matches)• No main effect of condition (p = .454)

• Other effects present Condition x Presentation, Fatigue, Learning,

Performance results unclear


Study 2: Summary of Results

H1: Interruption annoyance is lower in the Match condition than in the Static condition

H2: Perceived benefit is higher in the Match condition than in the Static condition

H3: Performance is higher in the Match conditionthan in all other conditions


Future Research

• Computational appraisal of utility• Notification signals• Scope of utility

Primary task relevance: • Re-examine performance results

Personal relevance: • Confirm annoyance and benefit results


Take Home MessageMatching AD and utility

Reduces annoyance and Increases perception of benefit

• UI design implications?


Acknowledgements

Many thanks to:• Natural Science and Engineering Research Council

of Canada (NSERC)• Lyn Bartram


Matching AD and utility Reduces annoyance and Increases perception of benefit

Take Home Message

Questions?

40Imager Laboratoryfor Graphics, Visualization and HCI

Additional Information


Interview Results

Comprehension of AD-utility relationshipLow 70%Med 40%High 45%All 40%None 25%

40% utilized this knowledge to ignore low-utility hints


Related Work

Coordination McFarlane 2002

Timing of Onset Fogarty et al. 2004 Bailey et al. 2001

Peripheral Awareness

Maglio & Campbell 2000 McCrickard et al. 2003

Notification Bartram, Ware, Calvert 2003 Horvitz, Kadie, Paek, Hovel 2003

Relevance Czerwinski, Cutrell, Horvitz 2000

Utility Horvitz, Koch, Apacible 2004 Horvitz, Jacobs, Hovel 1999 Avrahami & Hudson 2005 Gievska & Sibert 2005


Study 1: Simple Editor Task


Study 1 DesignWithin-subjects: 2 (task) x 10 (signal) x 3 (block)

Block Design 2 replications per block 3 blocks per primary task (6 total) 120 trials per subject

Measures Detection time

12 subjects, 90 minute experiment


Study 1: Structure of a Trial


Study 1 Results• Significant main effect of signal (p < .001)• 3-way significant comparison between Flag,

Slow Zoom, and Follow

4.83 4.57

6.02

1.96 1.88 1.74

3.39

9.19

1.491.19

0

2

4

6

8

10

Flag Colour Grow Oscillate SlowZoom

SlowBlink

Bounce FastZoom

FastBlink

Follow

Tim

e (s

econ

ds)

4.83 4.57

6.02

1.96 1.88 1.74

3.39

9.19

1.491.19

0

2

4

6

8

10

Flag Colour Grow Oscillate SlowZoom

SlowBlink

Bounce FastZoom

FastBlink

Follow

Tim

e (s

econ

ds)


Study 1: Pairwise Comparisons


Study 1: TimeoutsMain effect of signal(F(2.542,27.961) = 3.630, p = .031, η2 = .248)


Study 2 Design• Conditions: Match, Static, Control

• Within-subjects design

• 5 replications in 17-min block (i.e. 15 interruptions per condition)

• 65-second average interruption frequency

• 24 subjects

• 1 hour experiment


Study 2: Interruption Structure


Missed-Hints Dialog


Study 2 General AnnoyanceNo main effect of condition(F(2,24) = 2.788, p = .079, η2 = .166, power = .429)


Study 2: Presentation Order Effect

F(5,14) = 2.720, p = .064, η2 = .493


Study 2 Interaction Effect

F(10,28) = 2..35, p = .068, η2 = .421


Study 2 Hint DurationMain effect of utility(F(2,36) = 6.839, p = .003, η2 = .275)


Study 2 WorkloadNASA-TLX Factor F(2,28) p η2

Mental Demand .057 .945 .004

Physical Demand 2.335 .115 .143

Temporal Demand 1.069 .357 .071

Effort .118 .889 .008

Perceived Performance 1.347 .276 .088

Frustration .381 .687 .027


NASA Task Load Index

Mental DemandHow much mental and perceptual activity was required to play the game and attend to the hints? (e.g., thinking, remembering, looking, searching, deciding, etc.)?

Physical DemandHow much physical activity was required to play the game? (e.g. moving the mouse, clicking the mouse button, etc.)

Temporal DemandHow much time pressure did you feel due to the rate or pace at which the tasks or task elements occurred?


NASA Task Load Index (2)

EffortHow hard did you have to work (mentally and physically) to accomplish your level of performance?

Perceived PerformanceHow successful do you think you were in accomplishing the goals of the task set by the experimenter (or yourself)?

FrustrationHow insecure, discouraged, irritated, stressed and annoyed versus secure, gratified, content, relaxed and complacent did you feel during the task?


Performance Results

• No main effect of condition (p = .454)

• Main effect: presentation order (p = .064, η2 = .493) Interaction effect: condition & presentation order

(p = .068, η2 = .421)

• Learning effect (p = .053)

• Fatigue effect (p = .009)

Impossible to interpret performance results


Education

Matching Attentional Draw with Utility in Interruption