Contextual Dissonance: Design Bias in Sensor-Based Experience Sampling Methods

Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods

@neal_lathia, k. rachuri, c. mascolo (@cecim), j. rentfrowcomputer laboratory, university of cambridge#ubicomp13

Contextual Dissonance?

Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods

You are tasked with researching X (e.g., X = emotions) in daily life using ubiquitous tech; so you decide to build a system that will:

●Ask participants for assessments of the X they experience

●Collect sensor data to 'objectively' measure participants' contexts and quantify their behaviour

Research Scenario

why would you do this?

●… to explore whether machine learning approaches could infer people's subjective responses/complex behaviours

●… to understand the extent that the broad set of sensor data reflects self-reported behaviour

“...automated tracing is widely used to provide insight into what and when; however, it does not provide the why...”

Froehlich et al.

Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods

Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods

“...researchers are faced with concrete decisions regarding design [...] studies have often been classifed into the three categories of interval-, signal-, and event-contingent protocols...”

Bolger et. al

ESM design: how should I ask questions?

“...sampling to capture data from the sensors of the phone cannot be performed continuously, as this will drain the battery rapidly. However, conservative sampling leads to the loss of valuable behavioural data...”

K. Rachuri

sensor design: how should I sample from sensors?

Both of these design protocols will affect the quantity and quality of data that you receive from participants.

● Shouldn't sense everything all the time: triggers a survey based on a particular sensor

●Ask for subjective responses and, while doing so, sample data from other sensors to gather behavioural signals

Research Scenario

We built a system like this. It includes: sensor data collection, ESM interfaces, etc., and remote reconfguration.

Open Source Smartphone Libraries for Computational Social ScienceN. Lathia, K. Rachuri, C. Mascolo, G. Roussos. 2nd ACM Workshop on Mobile Systems for Computational Social Science.

as an aside...

22 users; 1-month; questions about mood & current context (location, sociability); background sensing from many sensors; triggers remotely reconfgured weekly.

Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods

… by asking the “what if” question:

Your ESM protocol is driven by the accelerometer's state: questionnaires will be triggered based on when the participant is moving.

Example Research Scenario

...skews sampling towards the later hours of the day

But all sensors have their own distribution...

… so how have I skewed my other results?

P( state(sensor1) = b | state(accelerometer) = a) ~ P( state(sensor1)) = b

Data that I would have received by continuous sampling

Data received by triggering on one sensor's state.

Bias?

Accelerometer ~ Non-Stationary10.61% of the data is non-stationary.

When it is, participants are:95.23% non-silent; 39.24% at home; 14.43% communicating with others.

Full Sample vs. Accelerometer Trigger

Non-silent?37.78% | 95.23%

Communicating with others?4.60% | 14.43%

More Examples?

Microphone ~ Silent/Non-SilentAccelerometer ~ Moving/Not-Moving

Location ~ Home/AwayScreen ~ Using the device

SMS/Calls ~ Communicating with othersProximity ~ Near the phone

Microphone ~ Non-Silent37.78% of the data is non-silent.

When it is, participants are:26.75% non-stationary; 47.12% at home; 9.48% communicating with others.

Full Sample vs. Microphone Trigger

Moving?10.61% | 26.75%

Communicating with others?4.60% | 9.48%

Dissonance; a tension or clash resulting from the combination of two disharmonious elements

Dissonance; between using sensor states to trigger ESM surveys while using sensor data to quantify context and behaviour.

Ok; so replace the accelerometer trigger with sampling uniformly across time.

Example Research Scenario

temporal sampling is more likely to fnd your participant in “dominant” contexts, e.g., at home.

But the response data I get back from participants will not be affected by the choices that I make... right?

Research Scenario

1-month; 4 groups with random weekly trigger orders: (a) screen, (b) communication events, (c) immediately during non-silence, (d) some time after non-silence

“4 of the 6 tests found that the negative affect ratings (and 2 out of 6 for the positive ratings) were signifcantly different from one another with at least 90% confdence.”

who are you with?

alone 33.33% of the time (screen trigger) to 60.77% of the time (microphone trigger)

Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods

Where do we go from here?Opportunities for more research...

Generalise sensor-enhanced experience sampling tool. Currently in alpha testing.

Working with Android sensors?Try out library!

One of the goals is to enable easy and quick access to sensor data in 2 lines of code.https://github.com/nlathia/SensorManager

Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods

@neal_lathia, k. rachuri, @cecim, j. rentfrowACM Ubicomp 2013

References

● Smyth and Stone. “Ecological Momentary Assessment Research in Behavioral Medicine.” Journal of Happiness Studies 2003.

● Froehlich et al. “MyExperience: A System for In Situ Tracing and Capturing User Feedback on Mobile Phones.” ACM MobiSys 2007.

● Froehlich et al. “UbiGreen: Investigating a Mobile Tool for Tracking and Supporting Green Transportation Habits” ACM CHI 2009.

● Rachuri. “Smartphones Based Social Sensing: Adaptive Sampling, Sensing and Computation Offloading.” PhD Thesis 2013.

● Bolger et. al. “Diary Methods: Capturing Life as it is Lived” Ann. Rev. Psychology 2003.