Upload
elizabeth-snowdon
View
1.975
Download
2
Tags:
Embed Size (px)
DESCRIPTION
It is important to learn to be flexible in your work and to learn tools that will empower usability analysts to be more effective in their roles. Also, in an age of diminishing corporate spending and rapid prototyping and testing, remote usability can save companies a lot of time and money.
Citation preview
11
Remote Usability Testing
December 5, 2009
Elizabeth SnowdonUsability Analyst, Sage Software
22
About meSenior Usability Analyst at Sage Software
Previously worked in usability at high-tech company
Conducting usability studies since 2003
3
Principles of Scientific Management
4
Traditional Lab Testing
55
• Remote usability• Advantages / Disadvantages• Case study• Tools
What I will talk about today
6
What is Remote Testing?
• Testing sessions in which you and participant are not physically in the same place but are communicating via electronic technology
6
7
Types of Remote Testing
7
Remote User Research
Synchronous
Asynchronous
•One on one and real time•Shared technology
•No moderator•Self-paced •Special tools record activities & feedback
8
Current Usability Testing Trends
1. Movement away from lab testing
2. Increased interest in testing in users’ environments
3. Agile development methods
9
Advantages• Lab not required
9
University of Leeds Human Perception and Performance:Usability Laboratory
10
Advantages
• Expands sampling population
• Ease of recruitment
• Users in their own natural browsing environment
• Share their desktop with you
11
Advantages
• Save money– Travel
– Equipment
– Recruiting costs
– Compensation
11
12
Disadvantages
• Difficult to see participant’s screen
• Cannot see participants
• Problems downloading or installing desktop sharing
• Task scenarios disseminated in advance
• Participants interrupted or distracted
12
13
When not to use
• When security is important
• When reliable high speed internet connection is unavailable
• For eye tracking studies
• When it is important to attend to user’s physical movement
13
14
When to use
• Recruiting valuable individuals
• Minimal set-up
• For lone wolf consultants
• Rapid iterative development (Agile)
14
15
Recruiting Users
15
Invitation + incentive
Screener ScheduleConfirmation +
Consent
Session Thank YouTasks
16
Case Study
• Baseline study for Accpac
• Three personas – clerks, controllers, operations
• 24 participants – 20 remote
• US, South Africa, Australia, UK, Canada
• One moderator, one note taker, observers
17
Results
• Accpac study completed in 4 weeks• Cost savings
– Travel– Lab set-up
• Time savings– Recruiting – Scheduling – Less overtime
• Bottom line - $50,000 saved
18
Recruiting
• Internal recruitment, customer lists, Craigslist, email
• Interested parties must pass screener
19
Other issues to sort out
• Obtaining informed consent and confidentiality agreement
• What computing and communications equipment do candidates have?
• Are they willing and able to download and run collaboration software?
20
Typical Set-up
• Moderator / Note-taker – Screen sharing: WebEx– Recording: Morae – Speakerphone
• Participant– High speed internet access– Speakerphone or headset telephone
21
Meeting room set-up
22
What can go wrong?
• User forgets– Send reminder one day in advance
• No shows– Provide your cell # and get participant’s cell
and work #
• User cannot download plug in– Send participant WebEx compatibility test
check email in advance
23
What can go wrong?
• User takes a long time to log onto WebEx– Add extra 10 minutes for WebEx set-up– Log onto session 15 minutes in advance– Get participant to log onto WebEx 10 minutes
in advance
• User brings along his co-worker– Explain 1to1 test– Reschedule secondary participant
24
What can go wrong?
• Can’t see what the user is doing– Ask user to point with mouse
• Conversations– Mute your line
25
WebEx Limitations
• Test! Test! Test! – Dry run
• Delays– Explain that there is a slight delay between
what they are seeing and what you see
• Cannot view hover overs– Verbally tell users what they are hovering
over
26
Pre-test / Test checklist
Email scenarios Set-up system or web site Open up required documents Dial into conference bridge or participant Sign onto WebEx Facilitator script Start recording Hand WebEx control to participant
27
Incentives
• Challenging for international users
• Email local Amazon gift certificates
28
Resources
• IBM - Experience remote usability testing
www.ibm.com/developerworks/library/wa-rmusts1/• Remote Usability remoteusability.com/• User Experience magazine
ww.usabilityprofessionals.org/upa_publications/user_experience/past_issues/2008-3.html
• Moderating Usability Tests: Principles and Practices for Interacting by Loring and Dumas
• Usability Engineering by Jakob Neilsen
29
Remote tools
• Liz Bacon listing http://remoteusability.com/remote-testing-tools-round-up-by-liz-bacon/
• Bolt Peters http://remoteusability.com/tools/