57
Qual vs Quant Using Better Data to Build Better Products Laura Klein [email protected] @lauraklein

Qual vs. Quant: Using data to Build Better Products

  • Upload
    uxpin

  • View
    474

  • Download
    2

Embed Size (px)

Citation preview

Page 1: Qual vs. Quant: Using data to Build Better Products

Qual vs QuantUsing Better Data to Build Better Products

Laura [email protected]

@lauraklein

Page 2: Qual vs. Quant: Using data to Build Better Products

Who’s Tired Of Hearing About Data?*

*be honest

Page 3: Qual vs. Quant: Using data to Build Better Products

A/B Testing

Page 4: Qual vs. Quant: Using data to Build Better Products

A/B Testing

Funnel Analysis

Page 5: Qual vs. Quant: Using data to Build Better Products

A/B Testing

Cohorts

Funnel Analysis

Page 6: Qual vs. Quant: Using data to Build Better Products

A/B Testing

Cohorts

Funnel Analysis

Blah

Page 7: Qual vs. Quant: Using data to Build Better Products

A/B Testing

Cohorts

Funnel Analysis

Blah

Blah

Page 8: Qual vs. Quant: Using data to Build Better Products

A/B Testing

Cohorts

Funnel Analysis

Blah

Blah

Blah

Page 9: Qual vs. Quant: Using data to Build Better Products

Sorry. I’m Going To Talk About Data Some More.

Page 10: Qual vs. Quant: Using data to Build Better Products

Better Data==

Better Products

Page 11: Qual vs. Quant: Using data to Build Better Products

Well…Sometimes

Page 12: Qual vs. Quant: Using data to Build Better Products

What Data Can’t DoReplace Design

Replace Listening to Users

Tell Us What We Should Be Building

Page 13: Qual vs. Quant: Using data to Build Better Products

What Data Can DoInform Design

Inform Research

Tell Us What We Should Be Investigating

Page 14: Qual vs. Quant: Using data to Build Better Products

How About An Example?

Page 15: Qual vs. Quant: Using data to Build Better Products

A Funnel

Page 16: Qual vs. Quant: Using data to Build Better Products

A Checkout Funnel

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

Page 17: Qual vs. Quant: Using data to Build Better Products

A Leaky Checkout Funnel

100%

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

-20%

-10%

-50%

-33%

-25%== ???

Page 18: Qual vs. Quant: Using data to Build Better Products

What That Means to You

100 Buyers

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

-20 Buyers

-8 Buyers

-36 Buyers

-12 Buyers

-6 Buyers== 18 Buyers

Page 19: Qual vs. Quant: Using data to Build Better Products

100 To 18????How Do We Fix This?

Page 20: Qual vs. Quant: Using data to Build Better Products

Identifying a Friction Point

100 Buyers

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

-20 Buyers

-8 Buyers

-36 Buyers

-12 Buyers

-6 Buyers== 18 Buyers

Page 21: Qual vs. Quant: Using data to Build Better Products

Stop It

Page 22: Qual vs. Quant: Using data to Build Better Products

Quantitative Data Qualitative Data

Understanding the Friction Point

Page 23: Qual vs. Quant: Using data to Build Better Products

Quantitative Data

WHAT

Qualitative Data

WHY

Page 24: Qual vs. Quant: Using data to Build Better Products

How Can You Use Them Together?

Page 25: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Page 26: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Use Funnel Metrics

Page 27: Qual vs. Quant: Using data to Build Better Products

The Biggest Problem

100 Buyers

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

-20 Buyers

-8 Buyers

-36 Buyers

-12 Buyers

-6 Buyers== 18 Buyers

Page 28: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Use Funnel Metrics Quant

Page 29: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Use Funnel Metrics

Step 2: Understand Why

Page 30: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Use Funnel Metrics

Step 2: Understand Why

Observational Usability

Page 31: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Use Funnel Metrics

Step 2: Understand Why

Observational Usability Qual

Page 32: Qual vs. Quant: Using data to Build Better Products

Problem Possibilities

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

-36 People• Didn’t have payment info handy• Confused by specific question• Looked for promo code

Page 33: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Use Funnel Metrics

Step 2: Understand Why

Observational Usability

Step 3: Propose Solutions

Page 34: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Use Funnel Metrics

Step 2: Understand Why

Observational Usability

Step 3: Propose Solutions

Create Solution Hypotheses

Page 35: Qual vs. Quant: Using data to Build Better Products

Solution Hypothesis

Problem

People looked for a promo code.

Page 36: Qual vs. Quant: Using data to Build Better Products

Solution Hypothesis

Possible Solution

Remove promo code entirely.

Page 37: Qual vs. Quant: Using data to Build Better Products

Solution Hypothesis

What Do We Think Would Happen?

An increase in conversion.

Page 38: Qual vs. Quant: Using data to Build Better Products

Solution Hypothesis

What Might Go Wrong?

A decrease in acquisition or revenue.

Page 39: Qual vs. Quant: Using data to Build Better Products

Step 4: Learn & Iterate*

*The last step is always learn & iterate!

Quant & Qual

Page 40: Qual vs. Quant: Using data to Build Better Products

Learn…

How Would We Know We Were Right?

A/B test of the checkout flow with and without the promo code.

Page 41: Qual vs. Quant: Using data to Build Better Products

What Really Happened?

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

Page 42: Qual vs. Quant: Using data to Build Better Products

Why Did It Happen?

Qual!

Page 43: Qual vs. Quant: Using data to Build Better Products

…And Iterate

Another Possible Solution

Create custom landing pages rather than promo codes.

Page 44: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Use Funnel Metrics

Step 2: Understand Why

Observational Usability

Step 3: Propose Solutions

Create Solution Hypotheses

Step 4: Learn & Iterate!

Page 45: Qual vs. Quant: Using data to Build Better Products

Or…

Page 46: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Page 47: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Interview Current UsersQual

Page 48: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Interview Current Users

Step 2: Understand What

Page 49: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Interview Current Users

Step 2: Understand What

Study Relevant MetricsQuant

Page 50: Qual vs. Quant: Using data to Build Better Products

But Why?

Page 51: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Interview Current Users

Step 2: Understand What

Study Relevant Metrics

Step 3: Propose Solutions

Create Solution Hypotheses

Page 52: Qual vs. Quant: Using data to Build Better Products

Step 1: Identify the Biggest Problem

Look at Your Sales Pipeline

Step 2: Understand Why

Interview People Who Said No

Step 3: Propose Solutions

Create Solution Hypotheses

Page 53: Qual vs. Quant: Using data to Build Better Products

Step 4: Learn & Iterate*

*The last step is always learn & iterate!

Page 54: Qual vs. Quant: Using data to Build Better Products

Once Again…

Quantitative Data

WHAT

Qualitative Data

WHY

Page 55: Qual vs. Quant: Using data to Build Better Products

Better Data ==

Better Products

Page 57: Qual vs. Quant: Using data to Build Better Products