Chuck Liu

How to Do Be!er Market Research January 22nd, 2015

Chuck Liu Design Research Lead

KISSmetrics

@chuckjliu cliu@kissmetrics.com

Thue L Madsen Marketing Operations Specialist

KISSmetrics

@thuelmadsen tmadsen@kissmetrics.com

@thuelmadsen #KISSwebinar

Join the conversation on Twi!er

Market research helps you make decisions.

1 3 Essential Mantras of Market Research

Email

In-App/Site Opt-In

In-App/Site Pop-Up

2 3 Ways for Early Stage Businesses to Do Lean Market Research

3 3 Strategies for Existing Businesses/Enterprises to Ge!ing Faster Research Done

Table of Contents

4 Bonus: The KISSmetrics Market Research Flow

WATCH WEBINAR RECORDING NOW

@chuckjliu #KISSwebinar

Join the conversation on Twi!er

3 Essential Mantras of Market Research

1

Goal:

Make be!er decisions, faster.

Get things done in days, not

weeks or months.

Market research priorities are

different depending on what stage your

business is at

There are many FREE resources out

there.

Google Trends: Measure market potential and interest

AVINASH KAUSHIK HTTP://KISS.LY/LEANAC

Twi!er, LinkedIn, Quora: Get the lay of the land on problems

Talking to experts: Get the detailed scoop of workflows and processes

LOG IN WITH GOOGLE

Start Your Free KISSmetrics Trial

1. Hypothesis-driven

Have an idea to prove or disprove

2. Short and targeted

5 days, 2 weeks max

3. Focus on outcomes

Know what you want to know

“A#er this research is completed, we will be able to

decide on…”

https://www.flickr.com/photos/mikebehnken/

“A#er this research is completed, we will be able to

move forward with…”

https://www.flickr.com/photos/angeloangelo

“A#er this research is completed, we know whether

we need to do …. or do ….”

https://www.flickr.com/photos/mikebehnken/

3 Ways for Early Stage Businesses to

do Lean Market Research

2

What problems, needs, and motivations do people have?

Do people understand your product’s value proposition?

Which messages are most effective at explaining your product?

Can people figure out how to use your product?

Why do people stop using your product?

What do people need?

Measuring how people behave

What do people want?

Measuring what they say

Can people use it?

Measuring tasks

1. Survey + Social Distribution

Cheap (or free), but requires more work on your part

The super lazy version: ask #lazyweb on Twi!er

1. Make a screener or survey

2. Tweet/share it out 3. Analyze

Tweet a screener or survey out

Tweet a screener or survey out

Try the good ole’ “asking for a friend”

(except it’s you really asking)

Whether you’re actually asking for a friend or not, this actually works be!er, especially if you tag a potential competitor

1. Ask a question on Quora

2. Revitalize an old relevant thread with a new comment

3. Ask people to answer an existing question

Pros

• Li!le setup other than survey itself

• Piggyback off existing product’s customers and users

• Network with strangers you would have not recruited otherwise

• Advanced targeting to help you limit your audience

Cons

• Targeting limited to own follower count or others’ followers

• Meaningful conversations need to be taken offline

Social Distribution Pros and Cons

2. AdWords Easy setup, variable expenses

You Pay for Clicks, Which Is Pre!y Realistic

AdWords Keyword Planner does the work for you in volume and interest

1. Practice your pitch 2. Limited character

count = concise messaging

3. Bad ideas = no problem

Additional AdWords Gotchas

• Your keywords determine who sees what (mind your demographics)

• Use targeting area to limit your ads be!er to industries you’re focused on

• Potentially slow depending on your keyword/bidding level

• If your keyword is near $1, DON’T DO IT! Sweet spot: $0.25-0.75

• Set a daily maximum spend so your costs don’t go out of control

Pros

• Pay for clicks

• Keyword tool does market sizing and competition

• Advanced targeting for segmenting demographics

Cons

• Keywords that you want could be expensive

• Keywords you think you want are non-existent and no one searches for them

• Can take long if your ads themselves are poorly wri!en

AdWords Pros and Cons

3. Amazon Mechanical Turk Disclaimer: I haven’t tried yet,

but I want to

Mechanical Turk Plan: Simple

• Design a test

• Distribute a test

• Analyze the data

Mechanical Turk Plan: Expanded

• Design a test

• Two designs

• Two messaging concepts

• Two branding concepts

• Distribute a test

• Write simple instructions: “Imagine you are…”

• Build a survey (within MTurk itself or TypeForm/Google Forms)

• Analyze the data

Mechanical Turk Survey Link Example

Marketing Skills Time! Write a great title and description

Set costs, number of participants, time to complete, how long you want the task to be posted for

Mechanical Turk Caveats

• Professional survey takers

• You can limit your demographic somewhat, but some people live off Mechanical Turk as income ($0.05 per 30-second HIT is a $6.00 hourly wage)

• Spam

• Surveys should have codes at the end to verify that they were completed by an actual person, not a robot

• Surveys should take 2-4 minutes for tasks that pay less than $0.25. Anything longer will get skipped over and your responses will come in slower.

• Interested in a longer read? See Rapid User Testing with Mechanical Turk

Mechanical Turk Pros and Cons

Pros

• Pay for only good answers

• Flexible price points based on your budget

• Potentially cheaper than paying survey tools for their “audience”

Cons

• Longer setup

• Potential spam / low quality answers to filter through

3 Strategies for Existing

Businesses/Enterprises to Ge!ing Faster Research Done

3

Can new customers understand and figure out how to use the product?

What are customers’ existing workflow/process and pain points?

What are pros/cons of competitive products?

How satisfied are existing customers with the product?

How does customers’ usage change over time?

1. In-App Surveys Contextual, relevant, and

dismissible

Existing workflow and pain points

• Nudge your customers with in-app surveys

• Open-ended “What best describes why you logged in to use this app/service today?”

• Open-ended “What frustrated you recently about this app/service?”

• Target customers at least a#er 60 seconds have elapsed, otherwise it’s annoying

• Leave feedback forms accessible from any part of the app

Example question and choices for finding value and use cases

Targeting - don’t make people annoyed at you

In-App Survey Pros and Cons

Pros

• Low cost

• Low effort

• Can be turned on/off as you please to measure activities over time

• Quick responses based on targeting technique

Cons

• Limits demographic to your existing users

• Can potentially annoy your users

2. Experience Sampling

Uncover user needs and behaviors

A psychology technique, now also used for market research

• Measure ongoing experiences and events by asking people the same question(s) one or more times per day

• On demand via notification (in-app, or email)

• When certain conditions are met (used a feature at least 5 times in one day)

• At specific periods of time

• Recruitment requires incentive because of the commitment

• I like to do “a chance to win” incentives

• Make it large, so the chance is worth it

Jawbone Experience Sampling Recruitment Email

Jawbone Studied My Sleep Pa!erns

Questions To Ask For Your Own Experience Sampling

• Open-ended: What was the reason you recently did X?

• Likert-scale: How did you feel when you used this app today?

• Open-ended: What is the reason you are using this app today?

• Likert-scale: How much do you agree or disagree with the following statement?

On Likert Scales (pronounced like “lick”)

• 5 -point scale with symmetrical answers

• Not at all interested, not that interested, neutral, somewhat interested, very interested

• Not at all, not really, undecided, somewhat, very much

• Never, rarely, once in a while, sometimes, always

• Strongly disagree, disagree, neutral, agree, strongly agree

On Likert Scales (pronounced like “lick”)

Statements you could use with Likert scales as answers

• YOUR APP is easy to use.

• YOUR APP is easy to setup.

• YOUR APP has a pleasing color scheme.

• YOUR APP has useful FEATURE.

• YOUR BUSINESS looks trustworthy.

• YOUR BUSINESS looks professional.

• YOUR BUSINESS has a clear message on what it does.

• Does this design look too bright or too plain?

• Does this page look too busy or too minimal?

• Does this screen look too gimmicky or practical?

Pros

• Highlights behaviors, moods, stress levels

• Gives context to these behaviors depending on how it was administered (same time every day, multiple times a day, etc.)

• Measures differences over time

Cons

• Risk of participants dropping off or stopping participation

• Incentive needed to lure in

• Not good for checking if someone is doing a task repetitively

Experience Sampling Pros and Cons

3. Persona Advisory Board

Quick Review: Personas (thanks to Buffer for these images!

Quick Review: Personas (thanks to Buffer for these images!

Recruiting Script

I wanted to extend an invitation to you to be part of the YOUR_COMPANY Customer Advisory Board. I'd like to have your on the advisory board in order to shape and influence the types of tools we build in order to help you do your job better.

The engagement on your part is pretty low. We'd love to hang out with you initially on-site to get a sense of your workflow and process. After that, we ask for just 15-30 minutes of your time, twice a month.

We'll also hold optional get-togethers among advisory members to share strategies and best practices.

Let me know what you think. I think you'd be a great fit with your experience.

Cheers,YOUR NAME

Guaranteed market research sessions every 2 weeks

• Recruit at least 2 people from each persona you have developed to cover the spectrum of customers you have

• 15-30 min phone call, screenshare, or on-site visit every 2 weeks

• Ask them about the tools they use now or are considering using

• Are there any opportunities or threats?

• Ask them about any annoying but essential tasks they do right now

• Are there any opportunities for you to explore that problem?

Pros

• Highly contextual information about day in the life, workflow, and process

• Visibility into which tools are used for tasks

• Deeper relationship and trust built with customer

Cons

• High amount of effort on your part

• Recruiting can be hit or miss depending on your relationship with customers / their availability

Advisory Board Pros and Cons

The KISSmetrics Market Research

Process

4

Our Market Research Process This works for us, but may not work for you.

Hypothesis A guiding point for what your market research is about

Informs questions you ask in screeners, surveys, interviews, or onsite visits

Screener In-app pop up survey tool with targeting capabilities

Segmenting and targeting specific customer types.

Survey / 1:1 Remote Interview / Onsite visit

Methods for finding out broad / specific / contextual information

Manage opt-in lists and screeners for demographic data

Report or Write UpSummary of your findings, the methodology you used, demographic information, results of the research, and recommendations

Informing your team or management to make a decision

What It Is How It’s Used

P.S. Please don’t do focus groups. Ever. They are a waste of time.

Research Tools of the Trade Our toolkit for ge!ing survey’s done.

Typeform Beautiful survey forms that display well on web and mobile.

Surveys for market and user research

Qualaroo In-app pop up survey tool with targeting capabilities

Nudge or opt-in for a larger survey or getting permission to follow up for a another research study

Google Forms Simple screeners and opt-in forms for collecting people’s information.

Manage opt-in lists and screeners for demographic data

Salesforce Pardot / Gmail + Boomerang

Marketing automation and email marketing

Schedule and create emails for distributing survey links. Manage opt-in lists and research campaigns.

Calendly Calendar scheduling appPeople sign themselves up on an open slot that works for them. No email back and forths!

What It Is How It’s Used

Book Resource: Just Enough Research

Questions?

Chuck Liu Design Research Lead

KISSmetrics @chuckjliu

cliu@kissmetrics.com

Thue L Madsen Marketing Operations Specialist

KISSmetrics @thuelmadsen

tmadsen@kissmetrics.com

THANK YOU

Chuck Liu @chuckjliu