30
how can sensor data be used to infer how people feel? @neal_lathia university of cambridge

Using Sensors to Infer how People Feel

Embed Size (px)

DESCRIPTION

Overview of the current research around emotionsense.org

Citation preview

Page 1: Using Sensors to Infer how People Feel

how can sensor data be used toinfer how people feel?

@neal_lathiauniversity of cambridge

Page 2: Using Sensors to Infer how People Feel

how many smartphones do you have?

Page 3: Using Sensors to Infer how People Feel

how often do you carry your phone with you (%) ?

Page 4: Using Sensors to Infer how People Feel

how often do you carry your phone with you (%) ?

research: 90% of the time it is in the same room as you

making it a powerful, personal device that can sense, monitor, and interact with you

Page 5: Using Sensors to Infer how People Feel

sense (infer): activities, social interactions, mobility, ambient sound

in the moment

Page 6: Using Sensors to Infer how People Feel

monitor: green transportation habits and modes, fitness levels

individuals over time

Page 7: Using Sensors to Infer how People Feel

interact: social network broadcasts, ubiquitous technology evaluation, persuasive interfaces

understand & persuade

Page 8: Using Sensors to Infer how People Feel

now: a tool for psychology research“ubiquitous, intimate, sensor-rich, computationally

powerful, remotely accessible”

Page 9: Using Sensors to Infer how People Feel

sense

interact

mental processes

infer

Page 10: Using Sensors to Infer how People Feel
Page 11: Using Sensors to Infer how People Feel

sensed context

reported mood

“in the moment”

infers

(responds)

Page 12: Using Sensors to Infer how People Feel

system → challenges

user → challenges

inference → challenges

Page 13: Using Sensors to Infer how People Feel

system → challenges

(kiran rachuri)

Page 14: Using Sensors to Infer how People Feel

system → challenges

battery

processing

memory

network

smartphone sensors weren't built for continuous monitoring

Page 15: Using Sensors to Infer how People Feel

system → challenges:: adaptive sensor sampling

sense sleep

continuous interval (50%)adaptive

-45% battery -5 % battery-2% accuracy - 40% accuracyco-location

-44% battery -8 % battery-1% accuracy - 34% accuracyaccelerometer

Page 16: Using Sensors to Infer how People Feel

inference → challenges

(ubicomp 2010)

Page 17: Using Sensors to Infer how People Feel

inference → challenges:: machine learning {sensed context} → emotion

microphone sample

emotional prosody-based model

daily diary inferred emotion

+ 71% accuracy

MAP likelihood

5 broad emotions

Page 18: Using Sensors to Infer how People Feel

user → challenges

Page 19: Using Sensors to Infer how People Feel

user → challenges:: protocols for experience sampling

signal respond

compliance, bias, delay

experience sampling

Page 20: Using Sensors to Infer how People Feel

system → challenges

user → challenges

putting it together

inference → challenges

Page 21: Using Sensors to Infer how People Feel

adaptive sampling

experience sampling

putting it together?

emotion inference learning

conflict?

conflict?

conflict?

Page 22: Using Sensors to Infer how People Feel

{sensed context} → {current emotion}

in the moment

Page 23: Using Sensors to Infer how People Feel

{sensed context} → {current emotion}

{ } → {current emotion}

{sensed context} → { }

Page 24: Using Sensors to Infer how People Feel

{sensed context} → {current emotion}

design for participant compliance

design for sensor data

Page 25: Using Sensors to Infer how People Feel

{sensed context} → {current emotion}

design for participant compliance

design for sensor data

just finishing: a/b test with a variety of triggers. results?

Page 26: Using Sensors to Infer how People Feel

so, to conclude..

Page 27: Using Sensors to Infer how People Feel

who canmisinterpret

your emotions?

you

your social circlesa psychologist

Page 28: Using Sensors to Infer how People Feel

who canmisinterpret

your emotions?

you

your social circlesa psychologist

your phone

Page 29: Using Sensors to Infer how People Feel

sensed context

reported mood

“in the moment”

infers

responds

Page 30: Using Sensors to Infer how People Feel

emotionsense.org

@neal_lathiauniversity of cambridge