Lean UX NYC: Getting Buy-in For UX Research

GETTING BUY-IN FOR UX RESEARCH

CHRISTINE PERFETTI & EZRA GILDESGAME ACQUIA

THE LOGISTICS

Ask questions as they occur to you

The schedule

You’ll be working in groups of 2

Lean UX NYC Perfetti & Gildesgame, © 2014 1

EXERCISE

CONDUCT A USABILITY TEST

USABILITY TEST

A usability test involves putting your users in front of the product and observing what they do


One person is the user

‣ Think aloud as you work

One person is the observer

‣ Observe silently and take notes

After we perform the task, we’ll ask you to switch roles and do another

EXPLORATION: USABILITY TEST

‣ We’re evaluating the design, not you

‣ The answer is somewhere on the site

‣ If you experience problems, it’s not your fault

‣ You’re helping the observers learn more about the design problems

AS THE USER


Take notes as the user works‣ Was the user successful?

‣ How did the user go about performing the task?

‣ What obstacles did the user encounter?

‣ What confusion did they experience?

‣ Note things that worked well

When you complete a task, say, “Got it!”

AS THE OBSERVER

‣ How many users found the answer?

‣ What helped users succeed?

‣ What obstacles prevented them from finding what they wanted?

‣ What do you expect will happen?

DISCUSSION


Planning your research program‣ The first 90 days

‣ How to choose your first project

‣ Identifying your allies and champions

Involving stakeholders in planning‣ Planning and running usability tests

‣ Quick testing techniques for buy-in

WHAT WE’LL COVER

Analyzing the data‣ Rolling list of observations

‣ KJ technique

‣ Journey maps

Communicating results effectively‣ Weekly research days

‣ Formal and informal reporting strategies

‣ Effective communication techniques

DISCUSSION

What prevents you fromgetting buy-in for your research?


THE FIRST 90 DAYS


Text

THE FIRST 90 DAYS

Meet with all stakeholders

Know your audience

Write a study plan

Conduct your first usability test


‣ Meet with all of your product stakeholders to understand their priorities

‣ Treat these meetings just like any other research project‣ Who are your users?

‣ What are their main goals?

‣ What challenges do they face today?

‣ What do they currently know about their users?

‣ What do they know about the product? What are the areas of risk?

KNOW YOUR AUDIENCE

What are all of the components of your product?


PRODUCT COMPONENTS

Who is on the product team?


‣ Interview anyone who has knowledge of the product or the business

‣ The product team includes:‣ Engineering

‣ Product Management

‣ UX

‣ Marketing

‣ Sales

‣ Support

‣ Legal

THE PRODUCT TEAM

‣ What is your role with respect to the product?

‣ What did you do before this?

‣ Who is the product for?

‣ How would you describe the client onboarding process?

‣ Is there anything that concerns you about the product?

‣ What should the user research accomplish for the business?

‣ How will you define success for the research?

‣ How would you like to be involved in the research?

GENERAL QUESTIONS


QUESTIONS: SPECIFIC TO AUDIENCE

Sales‣ Why do customers buy the product?

‣ Why do you win/lose sales?

‣ Who are the users today?

‣ How should they be different in 5 years?

‣ What do customers complain about most?Who

Support‣ Who typically contacts you?

‣ Why do they contact?

‣ What are the top problems for customers?

‣ What areas of the product cause problems?

‣ How can we best reduce calls?

The research team is at your service


SELLING THE VALUE: FRAME BENEFITS IN THEIR TERMS

Sales‣ Increase win rates

‣ Make selling easier

Product‣ Research can inform debate about product strategy

‣ Come armed with great data & insights

Support‣ Decrease # of tickets

‣ Decrease ticket response time

Finance and leadership‣ Increase renewals

‣ Improve win/growth rate

‣ Improve margins

Identify your Executive Champion

‣ Who drives the product and business decisions?

‣ What leader most values and understands user research and design?

Conduct a research project to:

‣ Answer the Executive Champion’s questions

‣ Focus on their risks and concerns

‣ Accomplish a quick win

CHOOSE YOUR FIRST PROJECT


CREATE A STUDY PLAN

THE STUDY PLAN

Serves as a blueprint for the test

Provides a systematic approach to testing

Facilitates buy-in from stakeholders


‣ Study goals

‣ Research questions and issues to explore

‣ User characteristics

‣ Schedule

‣ Task list

‣ Data to collect

‣ Reporting strategy

THE TOPICS

THE PLANNING MEETING

At the beginning of the project, we hold a planning meeting with all product team members and stakeholders to discuss:‣ An overview of the testing process

‣ Project scope

‣ Issues (risks and concerns) with the design

‣ Number of users to test

‣ Testing schedule

GOAL: No surprises! Avoid the game of telephone


MEETING ATTENDEES

‣ Product and Design

‣ Marketing and Engineering

‣ Anyone involved in design decisions

PROJECT SCOPE

What are you testing?

‣ Complete working prototype

‣ Prototype with only some screens

‣ Paper mockup

‣ Just the navigation


‣ What do you want to learn?‣ Most studies aren’t purely exploratory

‣ The issues and research questions will drive the tasks and recruitment

‣ Work with the team to outline the research questions

‣ Vague statement: Will this work for users?

‣ Ask increasingly specific questions

RISKS AND CONCERNS

‣ What are you hoping to learn from the study?

‣ What areas of the design are you most concerned about?

‣ Why are you concerned?

‣ Are you worried about a specific group of users?

‣ What are the greatest risks for the organization?

QUESTIONS TO ASK


‣ What do you want to learn?

‣ What areas are you most concerned about?

‣ Is our mortgage application too long?

‣ Will users know all fields are required?

‣ Will they use account lookup? (It will be hard to build.)

‣ Do our users understand what “conforming” means?

EXAMPLE: MORTGAGE APPLICATION

TECHNIQUES FOR BUY-IN


THE EXCUSES

Not enough time

No resources and staff

No money No buy-in from management

Unsure where to start

Testing is too scientific

We already know the product

TECHNIQUES FOR STAKEHOLDER BUY-IN

First Click Test

5 Second Test

Comprehension Test

Ethnographic Interviews


Quick and dirty technique for measuring content pages

Takes less than 10 minutes to run

Measures if content pages quickly convey their purpose

5 SECOND PAGE TESTS

You want to post pictures online from your last vacation

You are concerned that the upload process will be difficult

How confident are you that you can upload photos quickly and easily?

SHARING PICTURES






SHARING PICTURES





SHARING PICTURES



Useful method to assess where users first click on your site’s home or entry page

Provide users with a specific task to complete when they arrive at the site

By observing where users first click, it’s a clear indicator whether they’ll succeed

FIRST CLICK TEST



Pages containing complex content

Such as policies or procedures

How your product works

User comprehension is imperative to their success

Questions determine if users understand content

COMPREHENSION TEST


Excellent technique for understanding users’ goals

ETHNOGRAPHIC INTERVIEWS

Context of use

Pain points

Workarounds

Motivations


With a field study is, we learn about:‣ How technology fits into users’ life

‣ Tasks that users haven’t talked about

‣ Pain points and user frustration

‣ Evidence of technology that isn’t working for the user

“If I had asked people what they wanted, they would have said faster horses.”- Henry Ford (maybe)

FIELD STUDIES

VISIT USERS IN THEIR NATURAL SETTING


‣ Typically takes at least two hours at the users’ site

‣ If it’s not possible to visit users onsite, we use a remote meeting tool, such as GoToMeeting or WebEx

‣ Equipment‣ Digital recorder or audio recorder

‣ When approaching the interview, take the mindset of an apprentice‣ The user is the expert

INTERVIEW LOGISTICS

‣ Purpose of interview or site visit

‣ Demographics and background

‣ Overview of work process‣ What is your typical workday?

‣ Can you give an overview of your work?

‣ Who do you interact with?

‣ What are the most important and frequent tasks?

RECOMMENDED TOPICS

‣ Current behavior relevant to project without focusing on technology‣ Overview of technology and software user

works with

‣ How does technology support your work?

‣ What are you trying to accomplish by performing the tasks? (Goals)

‣ Follow up on key user feedback

‣ Closing


Avoid leading questions‣ Users like to respond, “yes”

Avoid:‣ Do you like this feature?

‣ Are you confused by this content?

BEWARE: LEADING QUESTIONS

‣ What activities waste your time or drive you crazy?

‣ How did you learn about the product?

‣ What kind of training did you receive?

‣ What are you most important activities?

‣ Who do you provide information to? What information do you collect?

SAMPLE QUESTIONS

‣ What are your most important and frequent activities with the product?

‣ What are the two things you like best about the product?

‣ What are the two things you would like to see improved?

‣ Are there any activities you currently perform that you’d like to see automated?

‣ What shortcuts help you?


‣ Understanding of work environment

‣ Tasks that users haven’t talked about

‣ “Cheat sheets”

‣ Evidence of technology that isn’t working for the user

WHAT TO OBSERVE

‣ Product pain points that were previously unknown

‣ Customers successfully using product in unexpected ways

‣ Customer needs that were previously unknown

‣ Examples that help tell the story (supplementing data)

‣ Identify clients who want to continue to partner

WHAT STAKEHOLDERS LEARN


ANALYZING DATA WITH STAKEHOLDERS

ANALYZING DATA

The KJ Method

Rolling list of observations

Journey mapping


‣ Gather all observations

‣ Identify problems

‣ Prioritize problems based on importance and frequency

‣ Iterate on the design

ANALYZE DATA

ROLLING LIST: OBSERVATIONS


‣ Named after Kawakita Jiro

‣ Consensus method for grouping and prioritizing usability findings

‣ Quickly determines most important observations

‣ Captures everyone’s perspective and observation

THE KJ: METHOD FOR PRIORITIZING DATA

‣ Call together the group

‣ Only invite members of the team who observed a test session

‣ KJ takes one hour

STEP 1: ORGANIZE GROUP


‣ Every KJ has a focus question that drives the prioritization exercise:

‣ What are the biggest usability problems we observed in the test?

‣ What needs to be fixed in the product to improve the user experience?

STEP 2: DETERMINE FOCUS QUESTION

KJ EXERCISEWhat are the biggest problems

that need to be fixed with the airline experience?


STEP 3: LIST USABILITY PROBLEMS OBSERVED

STEP 4: PUT OBSERVATIONS ON WALL


STEP 5: GROUP NOTES

STEP 6: NAME EACH GROUP


STEP 7: CHOOSE AND RANK GROUPS

STEP 8: VOTING


STEP 9: ORDER GROUPS AND DISCUSS

‣ We order the groups based on number of votes, highest numbers at the top

‣ Participants combine groups and discuss identified priorities

‣ Everyone is involved

‣ Writing down observations lets all team members contribute

‣ Prevents too much influence from select team members

‣ Identifies top priorities

METHOD ADVANTAGES


‣ Up-front study planning‣ Focus question: What are your research questions?

‣ Brainstorming sessions with stakeholders?

‣ Focus question: What are the biggest user problems we should address?

‣ Usability test debriefs

‣ Focus question: What are the biggest problems we observed in the tests?

‣ Immediately following field studies

‣ Focus question: What are the biggest pain points you observed? What solutions could help?

WHEN TO USE THE KJ METHOD

JOURNEY MAPPING


‣ Excellent technique for identifying pain points in the user journey

‣ Clearly communicates the opportunities for improvements and innovation

‣ Prioritizes the areas of the product that need the most attention

‣ Go from abstract to actionable

“World-class product” “Delight users” “Leading user experience”

JOURNEY MAPPING


`







Carbonite.com Journey Map

Mapping out the current experience


Task profile: Well performing areas

Task profile: Areas to improve


Q1 Update: 11 Critical Tasks Tested

Q1 2013 compared to Q4 2012


CREATIVE BRIEF

Benefits

‣ Helps people to fully articulate their ideas

‣ Shared understanding of goals and risks (and even disagreement)

‣ Help establish a shared understanding before starting a project

Components:

‣ Product statement

‣ Business problem

‣ Success criteria

‣ Risks

‣ Personas

‣ Deadlines and milestones

‣ Inspiration

COMMUNICATING RESEARCH


COMMUNICATING RESULTS

Formal methods

Informal methods

Involve stakeholders

‣ Communicate the results quickly

‣ Debrief often

‣ Report to stakeholders and project team when they need it

‣ Don’t report too much information

‣ Determine best delivery method considering the needs of your different audiences

‣ No surprises!

SUCCESS FACTORS


Informal

‣ Emails or internet dashboard with short memos of findings and recommendations

‣ Working meetings with project teams

‣ Integration with existing problem/bug reporting strategy

Formal

‣ Presentations and workshops

‣ Usability reports

METHODS: COMMUNICATING RESULTS

‣ Effective communication tool for sharing test results immediately

‣ After each day of tests, send a short email to design team with major observations from the day

EMAIL / INTERNET DASHBOARD


INVOLVE STAKEHOLDERS

`


`

`


‣ When quick changes are required after a round of tests, schedule a one-hour meeting to share and prioritize the test results

‣ Review the usability test observations

‣ Determine next steps and short-term fixes

WORKING MEETING

‣ Most organizations already have a method for collecting problem or bug reports

‣ Merge usability problems with other bugs

‣ Prevents having two tracking systems

‣ Helps determine relative importance of a problem

INTEGRATE WITH BUG TRACKING


‣ 1-2 hour presentation with design team, engineers, marketing, and key stakeholders

‣ Share most important usability findings based on prioritization of problems

‣ Effective communication technique

‣ People often ignore written communications

‣ Active involvement with design team and stakeholders

PRESENTATIONS/ WORKSHOPS

EXECUTING ON UX IMPROVEMENTS: MORE THAN BUY-IN

‣ Nobody says, “Let’s have a poor UX”

‣ “Experience Rot” as a risk

‣ It’s about the ideas, not the people

‣ Encourage open discussion of data

‣ Avoid sacrificing long-term quality for short-term goals

‣ Document commitment to UX and get buy-in for when things deviate

‣ Eat vegetables, floss daily, follow through on UX goals



URTH AND SURF

WEEKLY RESEARCH





QUESTIONS?
