DSW '19 - Choosing a direction Learning how to test ideas and … · 2020-03-21 · usability...

SEP 16-20, 2019 DenverStartupWeek.org #DENStartupWeek

CHOOSING A DIRECTION: LEARNING HOW TO TEST IDEAS AND DESIGNS Lys Maitland and Ari Weissman

TITLE SPONSORS

TRACK SPONSORS

HEADLINE SPONSORS

PARTNER SPONSORS British Consulate Officescapes Charter / Spectrum Procare Checkr Quizlet Ridg The Commons on Champa Segment Cooley SimpleBooth CTRC Slalom Ford Smart Mobility Tendril / Uplight Fremont Economic Development Test Double General Assembly Two Parts / The Passport Hilton Garden Inn Program Nanno Nix Media

MEMBER SPONSORS CoastalCloud Formidable Granicus Hogan Lovells Iterable Name.com Plante Moran Slifer Frampton Starry Swiftpage Syntropy Partners Yoonit Wine

#DENStartupWeek

#UX

The Agenda

1. Intro 2. What is UX Research? 3. The Design Process 4. Why do we test? 5. How to plan 6. $$$ vs. $ 7. Testing Outputs 8. Engaging the whole team 9. What to do next

888.310.5327

• 15+ years experience

• Responsible for research from product inception to implementation

• Led enterprise-level research for clients like Transamerica, Davita, Scottrade, NextGear, Securian

Lys Maitland Experience Researcher Manager

888.310.5327

• 15+ years global experience

• Responsible for all things UX, from research through experience design

• Lead for clients including TimeWarner Cable, AMEX, PwC, FreemanCo, Securian, …

Ari Weissman Director of UX @ Crownpeak

What is UX research?

To many people, conducting user research is a bit like cleaning up the garage: they know it would be a good thing “ to do and would bring some real benefits, yet somehow they never quite get round to it.

” — Arin Bhowmick Vice President, Design at IBM

Mike Kuniaysky

AUTHOR OF OBSERVING THE USER EXPERIENCE

What is UX Research?

“User research focuses on understanding user behaviors, needs, and motivations through observation techniques, task analysis, and other feedback methodologies…the process of understanding the impact of design on an audience.”

The Product Design Process

Double Diamond: Problem Space

The Problem Space • This is where we test ideas

• Goal is to identify a strategy and develop a plan

a.k.a. “Design the right thing.”

• This is where great user experiences come from

Discovery Research (diverge)

Understanding the User andProblem • Desk Research: lit reviews, existing

research, domain research, google scholar

• Stakeholder Research: workshops, interviews, demonstrations

• User Research: interviews, observation, surveys, contextual inquiry

Definition Research (converge)

Aligning around understanding • Target audience

• Desired user outcomes

• Desired business outcomes

• Risks and assumptions

• Method to measure success

• A hypothesis on how to solve the problem

Vision without action is a“ daydream. Action without vision is a nightmare. — Japanese Proverb ”

Double Diamond: Solution Space

The Solution Space • This is where we test solutions

• Goal is to work through tradeoffs to deliver an optimal solution

a.k.a. “Design things right”

• This is also where great usability comes from

Development Research (diverge)

Exploring Solutions • Design: sketching, prototyping,

usability testing

• Information Architecture: prototyping, card sorting, tree tests, click tests

• Collaboration: design studio, co-design, feedback workshop

• Desk Research: comparative analysis, pattern review

Delivery Research (diverge)

Delivering an outcome • Test and iterate on design and

content

• Consolidate design rationale

• Document experience / design decisions

Don’t Forget to Measure!

Did you choose how to measure success before you began design? Did you benchmark the current state to know if it's improved? Is your measurement strategy based on user outcomes?

Why do we test? UX research helps:

• Validate assumptions • Understand the problem from the user’s perspective

• Learn about the ‘hidden’ things we didn’t know • Measure the delta

• Remove opinions from the design process • Create alignment

• Reduce the cost of delivering a successful product.

Some examples of testing

CLICK TEST

Testing an IA or design learnability Testing flows and content USABILITY TESTING

BENCHMARKING CARD SORTING

Testing and measuring current state Closed card sort to validate an IA for redesign

ITERATIVE DESIGN TESTING

Iterative testing of flows and concepts, some generative research

What is user testing

let's watch

Planning a test Regardless of fidelity

Planning

1. Define the goals

2. Define the audience

3. Choose what to test

4. Choose your tools

5. Analysis

888.310.5327

Some questions to ask:What are - Why are you testing?

your goals? - What will you do with the data?

- What do you need to learn? Why?

888.310.5327

Some questions to ask: - Who will use of the thing you’re

building?

- Are there different groups of users? If so, which have the highest priority?

- Where are they?

Who is your audience?

You can learn valuable things by asking the right people the wrong questions. If you are talking to the “wrong people it doesn't matter what you ask.

— Erika Hall, author of Just Enough Research ”

What can I test?

Analysis and Synthesis

1 2 3 4

Trends Look for trends across

all participants

5 6 7 8

Quotes Solutions Opportunities Identify quotes that Collect solutions Do the outcomes

capture the essence of identified by suggest good next your insights participants steps?

Good and Bad Document what

doesn't work AND what does work

Prioritize Choose the most

important findings to share and focus on

Think big picture Is the issue identified a symptom of a bigger

problem?

Centralize/ Share

Easy access and redundancy.

$$$ vs. $

Formal

1 2 3 4

Location Quiet room with a table

Separate location for observers

Quality phone if remote

Logistics Schedule a location and

time for two hours

Recruit participants and distribute incentive

Prep/share/revise discussion guide

Create data capture sheet

Audio/Video recording

Costs Location

Recruiting and incentives

Screen sharing/ Recording tool

Snacks and beverages

Transcriptions

Fidelity to test Typically wireframe or

above

Quick and cheap

1 2 3 4

Location Logistics Costs Fidelity to test Anywhere! Find a place Minimal to none Paper and pencil

Find some people

Ask some questions

Write down answers

Let’s talk about examples

OUTPUTS

What do testing outputs look like?

super quick

$

quick and cheap

$

middle of the road

$$

formal

$$$$

Engaging the whole team

I use not only all the brains I have, but “ all the ones I can borrow

- Woodrow Wilson ”

- Choosing Goals and AudienceEngage your - Reviewing Discussion Guide team at every - Observing and Note Taking step - Debriefing

- Probing During Analysis

- Prioritizing Results

- Determining Next Steps

What to do next?

What to do next?

- Build testing into your delivery process

- Set up consistent research cadence

- Keep a backlog of problems

- Join in the design and dev process

- Develop a pool of users

- Alternate hi-effort and low-effort activities

- Validate between qualitative and

quantitative

- Triangulate your data

Questions? אלותש

Vragen Preguntas

問題 Des questions

вопросов

คถาม frågor

Mentioned References

GOOGLE DOC WITH ALL THE MENTIONED BOOKS, ARTICLES AND VIDEO

https://bit.ly/2kOEpyS