Wearable ESMDifferences in Experience Sampling Across Wearable DevicesJavier Hernandez (javierhr@mit.edu)Daniel J. McDuffChristian InfantePattie Maes Karen Quigley *Rosalind W. Picard

*

BackgroundWearable ESM ToolReal-life ExperimentResultsConclusions

Content

BackgroundWearable ESM ToolReal-life ExperimentResultsConclusions

Content

Experience Sampling Method

(Csikszentmihalyi et al. 1977)(Larson and Csikszentmihalyi, 1983)(EMA, Shiffman et al., 2008; Bolger et al., 2003)

Minimize biases

Uncontrolled

Minimize interactions

Natural settings

4

Social Interactions

Time ProductivityEmotions

Food Intake

Conner et al, 2009Ayzenberg et al, 2012Burgin et al, 2012Scholl et al 2014Adams et al, 2014

Berkman et al, 2014Church et al 2014Timmermann et al, 2015Hu et al, 2015

ESM: Applications

5

ESM: Devices

Can we use wearable devices to minimize some

of the problems?

Disruptive MissedInterruptions

Response Biases

6

Timmermann et al, 2015

Considered Locations

Stress and mood measurement

Pocket Head-worn

IntimateVery Accessible

CumbersomeVery Different

FamiliarWidespread

DisruptiveBiases

ConcealableAccessible

Small DisplayDifferent

Wrist-worn

Hu et al, 2015Scholl et al 2014

Adams et al, 2014Ayzenberg et al, 2012Berkman et al, 2014Burgin et al, 2012Church et al 2014

7

BackgroundWearable ESM ToolReal-life ExperimentResultsConclusions

Content

Experience Sampling Tool

2D Grid

5-Likert Scale

9

VERY PLEASANT

VERY

UN

PLEA

SAN

T

VERY ENERGETIC

LOW ENERGY

How were you feeling during the previous 5 minutes?

HIGH

RESOU

RCESLOW

RES

OU

RCES

HIGH DEMANDS

LOW DEMANDS

What situation best reflects your previous 5 minutes?

How stressed were you feeling during the previous

5 minutes?

Was this prompt disruptive?

Prompt Triggering

1. Time Distribution. ~1 prompt every 45 minutes

2. Time Variability. Distributed over the day

3. Device Variability. No more than two consecutive prompts on the same device

16

BackgroundWearable ESM ToolReal-life ExperimentResultsConclusions

Content

18

19

Samsung Galaxy S41920x1080 pixels

Samsung Gear Live320x320 pixels

Google Glass640x360 pixels

20

Starting Phase• Tutorial• Surveys

Ending Phase• Usability of devices• Free-form Interview

1st day $15, 2nd day $25, 3rd day $35, 4th day $45, 5th day $55

+ $25 for successfully completing the whole study

$200 for the whole study

Data Collection

21

Data Overview

15 participants (7 females and 8 males) from MITGraduate students and one administrator29.66 years (STD: 6.42) 5 (mostly) consecutive work days 22

BackgroundWearable ESM ToolReal-life ExperimentResultsConclusions

Content

Average #Prompts Issued by Device

Response Rate per Person over 5 Days

Total: 627 prompts

* Statistically significant based on ANOVA and Bonferroni correction (p<0.007)

*

24

Missed: 111 prompts (~18%)

Average #Prompts Issued by Device

Average (%) Submitted

Total: 627 prompts

Response Rate per Person over 5 Days

* Statistically significant based on ANOVA and Bonferroni correction (p<0.007)

*

25

Response Times

Time (seconds)

FinalSubmission

Prompt Triggered

Total Time

26

Response Times

Time (seconds)

FinalSubmission

Prompt Triggered

1st interactionwith the application

Total Time Starting Time

*

27

*

Response Times

Time (seconds)

FinalSubmission

Prompt Triggered

1st interactionwith the application

Total Time Starting Time Answering Time

*

28

Response Distribution

“I found it difficult to point out things [on the Gear] because my finger was on top of it”

“The watch was the hardest [to point] because my finger may be too fat” 29

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

Qua ty o epo ts

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

Very

Challenging

Very

Easy

vs1 2 3 4 5

0

2

4

6

8

10

Very

Inaccurate

Very

Accurate

vs1 2 3 4 5

0

2

4

6

8

10

Very

Uncomfortable

Very

Comfortable

vs1 2 3 4 5

0

2

4

6

8

10

Never All the timevs1 2 3 4 5

0

2

4

6

8

10

Not at all SignificantlyMore

vs1 2 3 4 5

0

2

4

6

8

10

Never All the timevs

Usability of DevicesDifficulty

of UseDevice

ComfortElicited Stress

Reports Quality

Affected Social Interactions

Potential Future Use

30

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

Qua ty o epo ts

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

Very

Challenging

Very

Easy

vs1 2 3 4 5

0

2

4

6

8

10

Very

Inaccurate

Very

Accurate

vs1 2 3 4 5

0

2

4

6

8

10

Very

Uncomfortable

Very

Comfortable

vs1 2 3 4 5

0

2

4

6

8

10

Never All the timevs1 2 3 4 5

0

2

4

6

8

10

Not at all SignificantlyMore

vs1 2 3 4 5

0

2

4

6

8

10

Never All the timevs

Usability of DevicesDifficulty

of UseDevice

ComfortElicited Stress

Reports Quality

Affected Social Interactions

Potential Future Use

“[The Glass] is painful, I wear glasses sometimes and they're not that uncomfortable...”“Surprisingly [the Glass] was not uncomfortable” 31

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

Qua ty o epo ts

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

Very

Challenging

Very

Easy

vs1 2 3 4 5

0

2

4

6

8

10

Very

Inaccurate

Very

Accurate

vs1 2 3 4 5

0

2

4

6

8

10

Very

Uncomfortable

Very

Comfortable

vs1 2 3 4 5

0

2

4

6

8

10

Never All the timevs1 2 3 4 5

0

2

4

6

8

10

Not at all SignificantlyMore

vs1 2 3 4 5

0

2

4

6

8

10

Never All the timevs

Usability of DevicesDifficulty

of UseDevice

ComfortElicited Stress

Reports Quality

Affected Social Interactions

Potential Future Use

“People would feel I was taking pictures of them and did not enjoy the conversation”“[The Glass] was a nice ice breaker” 32

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

Qua ty o epo ts

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

1 2 3 4 50

2

4

6

8

10

Very

Challenging

Very

Easy

vs1 2 3 4 5

0

2

4

6

8

10

Very

Inaccurate

Very

Accurate

vs1 2 3 4 5

0

2

4

6

8

10

Very

Uncomfortable

Very

Comfortable

vs1 2 3 4 5

0

2

4

6

8

10

Never All the timevs1 2 3 4 5

0

2

4

6

8

10

Not at all SignificantlyMore

vs1 2 3 4 5

0

2

4

6

8

10

Never All the timevs

Usability of DevicesDifficulty

of UseDevice

ComfortElicited Stress

Reports Quality

Affected Social Interactions

Potential Future Use

“I found it very annoying receiving notification through Glass when speaking with people because it was so noticeable” 33

BackgroundWearable ESM ToolReal-life ExperimentResultsConclusions

Content

Overview

ResponseTime

ResponseDistribution

ResponseRates

Usability35

• Application improvementsModify submissionSelf-report on demand

• Influencing factorsNegative press of GlassUsability of devices

• Other interactionsModalities (e.g., voice, camera)Input (e.g., text, numbers)

Limitations & Future Work

36

Wearable Experience Sampling Tool

Qualitative and Quantitative Analysis

Summary

37

Wearable ESMDifferences in Experience Sampling Across Wearable DevicesJavier Hernandez (javierhr@mit.edu)Daniel J. McDuffChristian InfantePattie Maes Karen Quigley *Rosalind W. Picard

*