Embed Size (px)
DESCRIPTION
Doing user research before and during development helps inform your choices about strategy (what to build) as well as tactics (how to build it)-- and it doesn't have to slow down your development process . In fact some rapidly executed research can speed up your time to market by reducing the need to refactor late in a project. This presentation includes practical information to help product owners and developers quickly get inside the heads of their users, validate product ideas and improve the usability of their software at warp speed. The talk included tips and techniques for recruiting research participants, shadowing and interviewing users effectively, getting valuable feedback on product concepts and information architecture, and rapidly iterating on the user interface to improve usability. They discussed remote testing tools that help teams evaluate if users can successfully achieve their goals with their designs, and reviewed best practices collecting feedback from users after launch.
Citation preview
Rapid User Research #rapidresearch
A part of the
HotHousing product kickoff & exploration framework
Aviva Rosenstein, PhD UX Consultant, Evolve Beyond
@avivaux
Gabrielle Benefield CEO, Evolve Beyond
@gbenefield
HotHousing.com 1
Why user research?
HotHousing.com 2
Why user research?
HotHousing.com 3
Customers don’t always share our knowledge, values, assumptions or interests.
Tech workers mostly here
User Research helps bridge the gaps between developers and users
End users mostly here
HotHousing.com 4
You might need user research if you hear or see these phrases:
“… I think they want to do this….”
“I’d want it to work like this…”
“"ey asked for this feature...”
“I assume they want…”
HotHousing.com 5
HOW DO YOU DO RAPID USER RESEARCH?
HotHousing.com 6
Methods to use depends on where you are in your product lifecycle
HotHousing.com 7
What do we build?
How do we build
it?
How did we
do?
And on what answers you need.
Understanding actual user behavior: What are users doing? When? Where? Understanding reasons for behavior Why are users doing that? Understanding user attitudes and opinions How do your users feel about doing it? HotHousing.com 8
HotHousing.com
Different methods provide different insights: mix ‘em up
Understand user’s experience, values, desires, environments
Measure or model behavior, predict outcomes
9
A few proven rapid methods
• Interviewing & Shadowing Users • Concept Tests
What do we build?
• Card Sorts, Tree Tests & Click Tests • RITE studies
How do we build it?
• Compare key metrics pre and post • Product Experience Feedback
How did we do?
HotHousing.com 10
Rapid research requirements
1. Executive and team buy-in 2. Feedback from the right people 3. Efficient data collection & analysis 4. Actionable, understandable insights 5. In-time reporting
HotHousing.com 11
• What roles do they play in relation to the product? User Roles
• How would you describe them? • Any relevant skills & knowledge? Characteristics
• What’s special about their situations? Context
What do we build?
Understanding users
HotHousing.com 12
• What are they trying to achieve? • How do they feel about these
tasks? Goals
• What do they need to accomplish those goals?
• What needs aren’t being met? Needs
• What are they doing now? • What can you improve? Pain Points
What do we build?
Understanding users
HotHousing.com 13
Empathizing with users’ pain and frustration
HotHousing.com 14
Pain scale (adapted from Hyperbole and a Half)
1 2 3 4 5
Being heard Cash or goodies
Knowing what’s coming
HotHousing.com 15
Finding the right people: push the right lever
Interviewing
Used to – Explore needs,
feelings, opinions – Obtain recollections
and rationales – Gather feedback
HotHousing.com 16
sweater and asked you to wear it.
• Image copyright Wil Wheaton CC BY-NC-SA 2.0
Hey, I
knitted this
sweater for you!
Gosh! Thank you!
(Ugh, it’s horrible.)
HotHousing.com 17
Behavior: Observing vs. Asking
What people say, what people do, and what they say they do are entirely different things.” HotHousing.com 18
19 Photo by Eric Allix Rogers, permission granted, and available under a Crea<ve Commons A=ribu<on-‐Noncommercial license. Source h=p://www.flickr.com/photos/reallyboring/6054538551/in/photostream HotHousing.com
Task Card: [task description] Performed by Role: [role name]
Context of Use: q Where and when is it performed?
q In what environment?
q What corporate culture?
q Where in development process?
q Direction of information flow?
q Device constraints/ media channels?
q Needs for q Auditability q Accuracy & Credibility q Confidentiality
Task Characteristics:
q Frequency q Regularity q Continuity q Intensity of use q Timeframe to act q Complexity q Predictability q Who controls the process?
q Legal/regulatory restrictions
q Operational/safety risks
q Other roles involved:
HotHousing.com 20
Shadowing Users
21 HotHousing.com
22 Photo by Jane Mejdahl, used under CC BY-‐SA 2.0. Source h=p://www.flickr.com/photos/gullig/5016829583/ HotHousing.com
ROLE: Business Owner TASK: Approve visual design direction CONTEXT: Waterfall dev process. Supervises multiple product managers. Frequently mobile; uses iPhone. CHARACTERISTICS: Short attention span. Under significant time pressure. Focuses on visuals and metrics. CONTENT CRITERIA: Brief, clear presentation in common formats consumable on mobile devices
ID
Dev Mgr VzD
PM BO
TASK:Approve visual design direction
Communicating insights from a shadowing session
HotHousing.com 23
What do we build?
Validating product and design ideas
How do they feel about our concept(s)?
Do they think our ideas make sense?
Will our concept work for them?
What features do users value?
HotHousing.com 24
Concept Interviews
Stimulate discussion with a narrative, storyboard, UI concept, prototype, demo, video or walkthrough
Used to: Explore needs, rationales,
and attitudes Gather feedback on ideas
HotHousing.com 25
Mackenzie is building a data-‐driven site and isn’t sure about the proposed schema.
HotHousing.com 26
Are these tables the right ones?
Mackenzie documents the schema but wants to get approval from her manager.
HotHousing.com 27
Mackenzie sends her product manager a link to the schema. Designcomics.org by Martin Hardee / Sun
Microsystems Illustrations by ISD Group, licensed under CC-BY
HotHousing.com 28
Collecting responses
Ques+on P1 P2 P3
Role or relevant characteris<c
Does concept work for them?
Features valued:
Posi<ve or Nega<ve Reac<on
Comments:
HotHousing.com 29
How to organize the site architecture
What labels to use on the navigation
What kind of navigation do we use?
What visual design approach to use
Do users understand how to use the site to accomplish their goals?
Does this product meet our quality standards (prior to launch)?
How do we make this usable?
How do we build
it?
HotHousing.com 30
Tools for Rapid Remote Testing
HotHousing.com 31
ZURB Apps
UX Punk
Optimal Workshop
Open online card sort interface
HotHousing.com 32
Analysis: Darker clusters, more associations
HotHousing.com 33
34
Data clustering results
HotHousing.com
RITE Studies
Stands for Rapid Iterative Testing and Evaluation.
USE IT TO: • Identify and resolve
usability issues in an interface, increasing levels of fidelity through the process.
• Improve and validate ease of use.
35 HotHousing.com
RITE Studies
36
1) Obvious Cause & Solution With Quick Fix
2) Obvious Cause, But Solution Needs Time to Design
3) Problem With No Obvious Solution (Keep Watching)
4) Issues Related To Test Script Or Study Protocol
Include participant #, issue, fix
Include # of participant, issue, fix
Include # of participant and issue
Include # of participant and issue
P#1 didn’t scroll down to see CTA- move up?
P#2 expected to download support information – create content library
P#2 didn’t know that link was clickable – add underline on hove?
P#3 unable to locate support link
P#1 test script set expectation for discounts, revise
P
HotHousing.com
RITE Studies
CONSIDERATIONS • Participants must represent and/or share key characteristics with target users • Conduct session in person or remotely using screen sharing applications. • Decision-makers must attend all sessions because decisions are made after each one. • Prototypes and task scripts may change during the study, so don’t collect success metrics
that depend on experimental rigor (i.e. time on task, error rate.) • Use either concurrent or stimulated retrospective think aloud technique to understand users’
expectations for and understanding of design elements. • Number of participants may vary depending on number of iterations needed. • Leave some time between sessions to allow for debriefing and making design changes. Try
scheduling a day between every three or four sessions to allow for design changes that require additional thought or time to implement.
ROLES 1 or 2 participants per session, 1 facilitator, stakeholder observers. MORE PRACTICAL INFORMATION Using the RITE Method to improve products: a definition and a case study. .Medlock, M. C., et al.
(2002). Usability Professionals Association, Orlando FL July 2002
37 HotHousing.com
How did we
do?
Evaluating success after launch
Do users understand how to accomplish their goals?
Are users satisfied with it?
Did we increase conversion/sales?
Are we keeping users engaged?...
HotHousing.com 38
SPLIT TESTING
Page 39
Surface
Skeleton
Structure
Scope
Strategy
• Interchangeable design elements
• Modules within a grid
• Complete features or versions
• Self-contained feature within an existing platform
• I am so, so sorry.
HotHousing.com
Efficient Cumulative Consistent Bias Resistant Sharable Retainable Contextual
Message Boards
Ad Hoc Surveys
Customer Calls
Focus Groups
PXF Survey
Collecting subjective product experience feedback
HotHousing.com 40
Product Experience Feedback
Requirements: q Provides actionable insights q Easy to share information with team members q Knowledge retained in the company q Doesn’t pollute the user experience q Easy to implement, uses resources efficiently
q Contextual to specific feature of interest q Consistent across product lines q Measurable, trackable progress q Construct validity, resistance to bias
HotHousing.com 41
Product Experience Feedback Survey
Includes: Open ended responses ü Problem reports ü Suggestions ü Praise ü Other comments
Product Usability Scale measuring perceptions of ü Efficiency ü Utility ü Performance ü Learnability ü Satisfaction ü Integration
HotHousing.com 42
Example template flow: Problem filter
HotHousing.com 43
Collect bugs first; then group feedback by type 1.Have you experienced any problems or errors when using [NAME OF FEATURE] in [PRODUCT NAME]? (Yes/No-randomized) 2. Please describe any problems or errors you've noticed while using [NAME OF FEATURE] 3. What, if anything, do you like most about [NAME OF FEATURE]? 4. Do you have any ideas or suggestions for improving [NAME OF FEATURE]? 5. If there is anything else you'd like us to know about the [NAME OF FEATURE] in [PRODUCT NAME], tell us here.
HotHousing.com 44
Product Experience Ratings: subjective experience metrics
6. Please rate how strongly you agree or disagree with each of the following statements:
HotHousing.com 45
1) Strongly Disagree 2) Disagree 3) Neither agree nor disagree 4) Agree 5) Strongly Agree)
a. I expect to use [NAME OF FEATURE] within [PRODUCT NAME] frequently.
b. [PERFORMING KEY USER STORY] with [NAME OF FEATURE] is easy and straighdorward.
c. I am sa<sfied with the [NAME OF FEATURE] in [PRODUCT NAME], d. I had to learn a lot of things before I could use the [NAME OF
FEATURE] effec<vely. e. The [NAME OF FEATURE] works seamlessly with the rest of the
[PRODUCT NAME] applica<on. f. When I use the [NAME OF FEATURE] it feels quick and responsive.
Utility Ease of Use Satisfaction Learnability Integration Performance
Rapid + Valuable = 6 steps
1. Engage stakeholders early and often 2. Keep plan focused 3. Get a representative sample 4. Ask questions and listen without bias 5. Collect data efficiently (but follow up
hunches and surprises) 6. Share actionable findings
HotHousing.com 46
