Conducting longitudinal research with online diary studies

Beverly Freeman, PayPal

User Research Friday!November 2010

1

2

(“The best worst 1st date ever”)

3

What causes change over time?

• Learning curve• System reliability, scalability• Change in personal situation• Interaction with co-users• Variance in quality of service encounters

4

Benefits of online longitudinal studies

• Geographic diversity• Efficient data collection and analysis• Ability to capture infrequent moments of truth• Capture changes over time

5Old-school paper diaries. Rich in data, but a logistical pain in the rear.

6

eBay homepage diary study

eBay homepage diary study

8

9

10

“I only used the homepage as a way to get to other pages”

Change over time for both test and control, but even more so for the new homepage!

(Beware the influence of asking the question)

11

12

PayPal Black Friday mobile diary study

13

Mobile makes data collection easier

14

PayPal Black Friday mobile diary study

15

Moderator view

Participant view (web)

16

17

#1: Online and offline are friends– Online longitudinal feed into face-to-face research

• Decide whom to invest time in to visit• Gather context, establish rapport before showing up at doorstep

#2: Plan to be surprised in exploratory studies– Both time and the unknown will surface interesting stuff

• Take advantage of the medium, and plan for adjustments

#3: Make it interesting for participants in quant studies too!– Repetitive surveys boredom drop-off sad researcher

• Spice it up a little• Opportunity to gather exploratory data for another topic?

Thank you!

Questions? eHarmony stories? bfreeman@paypal.com

18Diary photo credit: http://www.flickr.com/photos/hippie/2475835909/