25
#UTwebinar @UserTesting Aiden Bordner Co-founder and Principal Designer Parade The agency’s guide to effective user research

The agency's guide to effective user research

Embed Size (px)

Citation preview

Page 1: The agency's guide to effective user research

#UTwebinar @UserTesting

Aiden Bordner Co-founder and Principal Designer

Parade

The agency’s guide to effective user research

Page 2: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

Who is this for?

•Large Agencies •Small Agencies •Anyone who works with Stakeholders

Page 3: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

Why do agencies struggle?

• Large Agencies: • Traditionally, testing was to be avoided • Hierarchical nature, focus on client relationship • Perception of high cost • Fear of “bad results”

Issues largely to do with Culture and Business Model

Page 4: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

Why do agencies struggle?

• Small Agencies: • No staff on hand to focus on research • Project-basis and of fixed-fee scopes • No experience doing research before

Issues largely to do with Culture and Business Model

Page 5: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

How do we improve?

1. Make Testing a Rule 2. Set Expectations Early, Often 3. Develop a Broad Toolkit to Right-Size Research 4. Reduce Costs by Learning to Moderate Yourself 5. Include the Client in Testing 6. Proactively Manage Results

Six tips for doing more effective user research and overcoming cultural and business impediments.

Page 6: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

1. Make Testing a Rule

• Best way to ensure testing is to change culture • Don’t let yourself take it out of proposals • Don’t let the client take it out either • Cite monetary or performance sales examples

Page 7: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

1. Make Testing a Rule

• How to schedule • Plan on testing for every ~3-4 weeks of design. • Testing should take 1-2 weeks depending on

methodology • You can keep designing while you’re testing • Include time for revisions

Page 8: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

2. Set Expectations Early, Often

• Work to diffuse feeling that testing is a “Big Deal” • Not market research • Qual vs. quant • Formalize expectations in a deliverable

• (e.g. Research Plan)

Page 9: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

Benefit Cosmetics • Brows Digital Experience • Outside Test Research Plan Page / 1 4

BENEFIT BROWS DIGITAL EXPERIENCE

Outside Research Planv1.0 December 8, 2015

Research Plan

• 1-3 page document • Sections:

• Overview

• Methodology

• Stimulus

• Area of Focus

• Areas of Non-Focus

• Known Limitations

• Outcomes/Reporting

• Schedule

• Share at beginning of each research effort

Solidifies Plan & Sets Expectations

Page 10: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

3. Right-Size the Research for the Assignment

• Learn to develop a broad kit of tools

Page 11: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

3. Right-Size the Research for the Assignment

• Recruiting:

• Services • Live Recruiting • Friends & Family • Building Occupants • Client Employees • Intercepts • Craigslist • Pro-Tip: Look to Client for Incentives

Develop a Broad Kit of Tools

Page 12: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

3. Right-Size the Research for the Assignment

• Usability Testing:

• DIY vs. Full-Service • In-Person, Remote • Moderated, Unmoderated • Free Exploration, Task Completion, Natural Task • “Hallway” Testing

Develop a Broad Kit of Tools

Page 13: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

3. Right-Size the Research for the Assignment

• User Research:

• Great way to focus before designing • Simple Observation • Interviews • Card Sorts, Tree Testing

Develop a Broad Kit of Tools

Page 14: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

3. Right-Size the Research for the Assignment

• Learn to develop a broad kit of tools • Discuss options, recommendations with client • Keep the recruit small • Downsize vs. eliminating

Page 15: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

4. Learn to Moderate Yourself

• Moderating internally saves time and money • Two key outputs:

• Discussion Guide • Moderation

Page 16: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

4. Learn to Moderate Yourself

• Discussion Guide:

• Script For Moderation • Sections:

• Orientation/Ease-Setting • Recording Permission • “Think Aloud” Conditioning • Self-Directed Exploration or Natural Task Repeat • Task Completion • Open-Ended Questions • Thank You/Incentive Information

Introduction (2 minutes)TOPIC MODERATOR

Introduction So, thanks again for participating. My name is _________, I’m going to be guiding you though this experiment today. This is ________, who’ll just be taking notes.

Before we get started I would like to tell you a little bit about what we are doing today. We’re collecting feedback on some work in progress designs for a future Benefit website. This website will be about eyebrows—both in general and about specific products for eyebrows, some of which you may be familiar with and some of which may be new. We’ll be testing just a [mobile or desktop] version of the website today, so in a few minutes I’m going to [hand you this mobile device or ask you to use this laptop] which is set up to record your actions and where you tap on the screen so we can review it later.

Makes sense?

Orientation / Relaxation

Your job is really easy, you just have to be yourself and act as you naturally would. Our goal today is to get honest feedback, so feel free to say whatever’s on your mind—you won’t offend us. As you interact with the site, you should know this is just a prototype—a work in progress—so some things may not be fully functioning or complete. If something doesn’t work, just tell me what you expected to happen instead.

[Mobile only: You should also know there’s one point where we may need to change apps for the test. If that’s necessary, I’ll let you know when, and you can just hand to phone to me and I’ll make the switch before we continue. We’ll have to switch back when we’re done, but it will only take a second.] Okay? Great!

Start Recording So I’m just going to start the recording on this device and we’ll be almost ready to go.

[Hand device to participant or position participant at laptop] Okay, and just for the record, could you please look at the screen and say your name for us.

… Okay great!

Brows Digital Experience • Hallway Test Moderator Guide Page / 2 6

Page 17: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

4. Learn to Moderate Yourself

• Moderating:

• Stop Being the Designer • Above All: Avoid Leading Questions • Answer Questions with More Questions • Give the Subject Time; Never Interrupt • When in Doubt, Don’t Talk • Don’t Get Frustrated • Be Encouraging and Thankful

Page 18: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

4. Learn to Moderate Yourself

• Moderating internally saves time and money • Two key outputs:

• Discussion Guide • Moderation

• Don’t worry: trust your team

Page 19: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

5. Include the Client

• Important to give clients real-time visibility • Use technology to create a virtual lab • Place someone with the client, if possible • Schedule discussion breaks • Provide recordings • Do a Tech Rehearsal

Page 20: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

6. Manage Results Proactively

• Proactively filter the “noise” • Focus only on the trends • Explain behavior vs. comprehension • Make design recommendations proactively • Dispel non-issues proactively, too • Keep it lean • Don’t bludgeon your client with research

Page 21: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

Recap

1. Make Testing a Rule 2. Set Expectations Early, Often 3. Develop a Broad Toolkit to Right-Size Research 4. Reduce Costs by Learning to Moderate Yourself 5. Include the Client in Testing 6. Proactively Manage Results

Six tips for doing more effective user research and overcoming cultural and business impediments.

Page 22: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

Our Approach

1. Plan to do a study for every 3-4 weeks of design

2. Do upfront user research for any project involving unknown user types, special interactions

3. Build research into the schedule at time of proposal 4. Three Deliverables:

• Research Plan

• Discussion Guide (if necessary)

• Report

Parade’s process for research scheduling

Page 23: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

Our Approach

5. Should be able to accomplish a moderated, one-day study in 7 working days:

• Day 1: Review, adjust and approve research plan

• Day 2: Review moderation Guide

• Day 3: Finalize guide

• Day 5: Testing

• Day 7: Present findings

Parade’s process for research scheduling

Page 24: The agency's guide to effective user research

The Agency’s Guide to Effective User Research #UTWebinar

Questions?

• Learn more about Parade and Aiden’s team at • parade-sf.com

• Questions about UserTesting? • Email [email protected]

Page 25: The agency's guide to effective user research