Upload
amanda-nance
View
1.102
Download
0
Tags:
Embed Size (px)
Citation preview
Conducting Longitudinal Diary Studies
Amanda Nance
Sage
June 10, 2009
Early diary study
Goal of this presentation
• How to run a better diary study
– Create effective template
– Understand the data
– Keep users engaged
• See benefits of the method
Topics for this presentation
• Reason for the project
• About the diary study
• Diary study tips
• Unique contributions of the diary method
Reasons for the project
• Assess “getting started” experience of our desktop accounting software
• Focus on tasks after initial set up
• Create requirements for next release
Used multiple methods to get truer picture
Site visits Support calls Survey analysis
Researcher calls Diary studyDiary from: http://www.flickr.com/photos/bdorfman/15846725/
About the diary study
• 5 participants: small business owners, bookkeepers
• Diaries kept for up to 8 weeks beginning at install
• Diaries submitted weekly by email
• Compensation – at least $40/week
About the diary study
• Provided template in Word to record
– Daily Work: tools they used, other tasks
– Issues and Comments: problems with our product
Tips
Recruit extra participants
• Recruited 12
• 5 completed
• Reasons for drop outs were interesting findings, too
Use a recruiter to save time
• Pros and cons
• They can recruit and more– Email documents– Remind of deadlines
– Collect diaries
Communicate deadlines
• Communicate due date for first submission
• Disqualify participants who don’t respond
• Close the loop
Template tips
Make it easy for participants
• Keep it short
– This study: 3 questions daily, 6-7 per incident
• Separate questions from answers
Separate questions from answers
Show sample entries
What did you do for your business today?
• I answered several customer phone calls.
• Called automated bank system to make sure we have enough money for payroll.
• Called in payroll to ADP (we outsource payroll).
• Ordered parts for one of our jobs by phone.
Ask for screenshots
“Looking at Aged Receivables…. They do not match. Why are they different?”
Tips for screenshots
• If they are important, require them
• Include instructions
Structure template for easy analysis
• Separate daily entries from critical incidents
• Include participant name and week number in footer and file name
Iterate on the template
• Prior to launching
• As you receive user feedback
Process tips
Require daily entries
• Keeps participants engaged
• Gives you a reason to pay them
• Opportunity to learn about the broader work context
Talk to the participants yourself
• Intro call to understand business and product expectations
• Follow up on unclear entries
• Provide support
Research tips
• Read entries as soon as possible
• Leave time to research
Data analysis
• No different from other methods
• Evaluate data– Excel to categorize theme, second theme, week #
• Share and act on data– Summarize key findings, provide recommendation
for each
– Use customer quotes
– Share complete diaries
Unique contributions of diary study
• Infrequent tasks
• Help use
• Users resolving issues
• Tasks over time
Reasons to run a diary study
• No travel required
• Non-intrusive, in context
• Cheaper than site visits
Engagement and meaningful data
• Pay for more complete responses
• Encourage useful responses through feedback on recent entries
• Continue participants who are useful
Key Points
• Daily entries
• Explain what you expect
• Sample entries
• Reasonable number of questions
• Personal contact with researcher
• Questions?
Contact
More tips in conference proceedings
Twitter: @amandaux