Upload
catherine-shyu
View
288
Download
0
Embed Size (px)
Citation preview
Experimenting Your
Way to MVP
Catherine ShyuPM @ FullContact
About
Product Manager at FullContact
Previously PM at SendGrid, Generalist at BandPage
Bay Area native
@cthrin
We all want to build this
Nobody wants to spend years building this
Our job is this
Address the right target market, with the right problem, with the right solution and marketing, at the right time… Easy right?
So how do we make sure we’re working on the right
things?
Shorten the Lean Startup cycle
Catch core problems before investing too much time/money
Validate your riskiest assumptions through experiments
Image from Google Ventures
What are experiments? We run experiments
to test hypotheses that...
1.Can be validated / invalidated
2.Address areas of risk in the product
No brainer, right? So why aren’t more companies doing it?
Top 5 Reasons for
not Experimenti
ng
1.I can’t justify taking time away from building the product.
2.I don’t have approval from management to do this.
3.We have a deeply complex product. There’s no way to simulate things on it.
4.Our product is too niche. I can’t find users to talk to.
5.I don’t want to damage our relationship with potential customers.
I can’t justify taking time away from building the product.
Excuse 1.
Time $ $$ $$$ $$$
$
Cost of Change Increases with Product Complexity
Example: Customer Development
Goal: Product/market fit
Assumption: [X] industry is in our target market
Methods: Customer development interviews with 5 people from each industry
Most powerful questions asked:
“What are the top three challenges you face in your industry?”
“How do you currently solve [x] problem?”
Results
● Findings helped me reorder our product roadmap● Saved us development time by cutting down v1
scope
Excuse 1.
I don’t have approval from management to do this.
Excuse 2.
Have a plan & explain it in cost/benefits
1.Make a list of big assumptions and costs associated with them if you get it wrong
a. Have a few experiments prepared to validate/invalidate
b. Lean towards experiments that are quick and easy to run but get you big insights
2.Are there easy tests you can run to mitigate big expensive risks?
3.Frame as “early risk mitigation”Results
● If you can validate/invalidate your most
important assumption, it will 1) inform your product direction and 2) pave the way for more experiments.
Excuse 2.
We have a deeply complex product. There’s no way to simulate things
on it.Niche industries, Hardware, AI, VR, etc.
Excuse 3.
Powerpoint is your friend
Take shortcuts however you canInvision: string together high-fidelity mockups to simulate interactivity
Powerpoint: when you need to mock something up very quickly
3D Printing: when you are testing the physical UX of a product
Results
● Ability to go through rounds of changes before engineering starts work
Excuse 3.
Our product is too niche. I can’t find users to talk to.
Excuse 4.
Use the power of LinkedIn stalking
LinkedIn has robust advanced search features (job title, company, industry)
Craigslist ads for local in-person interviews
Leverage existing customers
Results
● Once I got one person to talk to me, they would often introduce me to their peers in the industry. Always ask for intros to more people.
Excuse 4.
I don’t want to damage our relationship with potential
customers.
Excuse 5.
Frame the interviewsPick customers who will be sympathetic
Use framing techniques to prime the customer for giving feedback“I appreciate your time…”
“This is still early stages but we wanted to get your opinion at the start…”
“Since you are a long-time customer, I wanted to get your feedback on a change I’m considering…”
Make them feel specialResults
● Many of our private beta testers are still avid users of the app and respond with feedback when asked. They also have free access to the app.
Excuse 5.
Lean Experiments
: Basic Principles
Goal is to test your riskiest assumptions early in the game
Experiments run for 3-5 days
Multiple experiments can run at once
5 people = pattern
Don’t worry, it doesn’t have to be a huge ordeal
Team exercise: biggest risks in your productRisk Severity (1-5, 5
highest)Hypothesis 1 Experiment 1
People won’t like my Dilbert comics
5, obviously People hate the comics but are too nice to say so.
Distribute post-talk anonymous survey
What’d We Do at FullContact?Team of 5: two Sales, one PM, one PMM, one Customer Success, one
Engineer
Trello board for tracking experiments
Weekly check-ins to share learnings
Our prototype when we started.
Now: FullContact for Teams Beta - onwards to v1!
Results
De-scoped the feature set into two releases, by customer segment
Validated the need to invest in building a core set of features
Able to test out different pricing on potential customers
Discovered the problem we are solving is on a top 3 list of problems for multiple industries
Always Be Experimenting!
👋 Thanks for listening@cthrin