Upload
optimizely
View
69
Download
0
Embed Size (px)
Citation preview
Innovate Faster with the Modern Growth Stack
Brion Hickey
GrowthCisco
Principal Product ManagerAmplitude
Carter Foxgrover
Senior Product MarketerOptimizely
David Isquick
Brion Hickey
GrowthCisco
Principal Product ManagerAmplitude
Carter Foxgrover
Senior Product MarketerOptimizely
David Isquick
• We are recording today’s presentation• Slides will be available after the webinar is complete• Please submit questions via the text box
Housekeeping
Growth - Tech Stack
About Me
Over 120 companies worldwide Then Now
USA
USA EMEA
USA EMEAAPAC
Came home.Got a cat.
Named himMeatball.
Optimization > GrowthTest stuff > Knowing your end users
Transitional Period
Growth
Philosophy
Growth
Philosophy
Strategy
Growth
Philosophy
Strategy
Technologies
Growth
Philosophy
Strategy
Technologies
Tactics
Growth
Philosophy
Strategy
Technologies
Tactics
Growth Tools
Customer Journey
Marketing Analytics (auditing)
Customer Journey
Growth Tools
Product Analytics / BehaviorsMarketing Analytics (auditing)
Customer Journey
Growth Tools
Product Analytics / BehaviorsMarketing Analytics (auditing)
Experimentation framework
Customer Journey
Growth Tools
Product Analytics / BehaviorsMarketing Analytics (auditing)
Experimentation framework
User Testing
Customer Journey
Growth Tools
Marketing Analytics
Marketing Analytics
“It was an awful integration”
Marketing Analytics
“It was an awful integration”
“The reports can’t be right”
Marketing Analytics
“It was an awful integration”
“The reports can’t be right”
“The pixel isn’t firing”
Marketing Analytics
“It was an awful integration”
“The reports can’t be right”
“The pixel isn’t firing”
“I don’t trust (marketing analytics company here)”
Marketing Analytics
ObservePoint
Auditing Marketing Analytics
● “It was an awful integration”○ Specifically addressing the integrity of
the integration○ What tags are missing○ What pages need to be fixed
Auditing Marketing Analytics
● “The reports can’t be right”● “The Pixel isn’t firing”○ Complete inventory of tags firing
(working as expected) ○ List of requests sent for each page ○ Duplicate tags, requests
Auditing Marketing Analytics
● “I don’t trust (insert any marketing analyticscompany here)”○ Output of a initial score/ grade ○ Comprehensive list of elements to fix ○ Update score, gain trust
In-Product Behavior Analytics
Product Analytics / BehaviorsMarketing Analytics (auditing)
Customer Journey
Amplitude
Text HereAmplitude
Retention
NorthStar
Metrics
Text HereAmplitude
Retention
A ha Moment
NorthStar
Metrics
Text HereAmplitude
Retention
A ha Moment
Product Market
Fit
NorthStar
Metrics
Text HereAmplitude
Retention
A ha Moment
Product Market
Fit
NorthStar
Metrics
Text HereAmplitude
Retention
A ha Moment
Product Market
Fit
NorthStar
MetricsWhat toA/BTest
But wait...
Real End User Testing
Product Analytics / BehaviorsMarketing Analytics (auditing)
User Testing
Customer Journey
Usertest.io
User Testing
● It’s not expensive anymore / $15● Reporting drop off. Why?● Initiated user testing.● Our buttons didn’t work.○ Wait what? ○ Ad blockers
User Testing
● It’s not expensive anymore / $15● Reporting drop off. Why?● Initiated user testing.● Our buttons didn’t work.○ Wait what? ○ Ad blockers
Dude, you had 1 job :|
User Testing● Turn it into a working cadence ○ Weekly, monthly○ Stay consistent ○ Especially before and after redesigns/feature releases
● It take a lot of time, so what○ Create summary video ○ Invaluable feedback
Get Organized
● Start to document findings● Access to content isn’t hard● Easy for engineers to consume ○ Screenshots ○ Behaviors ○ Time stamp ○ How to reproduce○ Expected behaviors
Optimizely
Marketing
Customer Journey
Optimizely
Product Marketing
Customer Journey
Optimizely
Product Marketing
Experimentation framework
Customer Journey
Code Level (Full Stack)
Optimizely
Product Marketing
Experimentation framework
Customer Journey
Code Level (Full Stack)
Personalization Recommendations Mobile Native
Optimizely
I can change
I can change I’m listening
I did this
I did this
Product Analytics / BehaviorsMarketing Analytics (auditing)
Experimentation framework
User Testing
Customer Journey
Growth Tools
Stay In Touch Brion.tw [email protected]
Build Better Products
Account Owner | [email protected] | phone number
Product Analytics for The Modern Growth Stack
Corporate BackgroundAMPLITUDE
● Launched Jan 2014
● Backed by Benchmark, Battery & IVP
● Now power over 20k products for over 5k
companies across the globe
● Analyzed over 2.5T user actions
● #1 Retention Playbook in industry
● Leader in self-serve Product Analytics
The Fastest Moving Growth & Product Teams use AmplitudeOUR CUSTOMERS
Consumer Media B2B Ecommerce Finance Healthcare
Product Analyticsfor Growth
2
Science, not Art
Quarterly releases and NPS surveys
aren’t enough. Modern growth teams
run like optimization machines:
● Mine data for insights
● Rapidly test ideas for impact
● Improve experiences every day
ADAPT OR DIE
Growth Teams Need 3 Types of Product Insights to InnovateWHAT IS PRODUCT ANALYTICS FOR GROWTH?
How do CUSTOMERS
use our products?
Examples
Do some users binge watch shows? If so which ones? What shows are more likely to be binge watched?
Growth Teams Need 3 Types of Product Insights to InnovateWHAT IS PRODUCT ANALYTICS FOR GROWTH?
How do CUSTOMERS
use our products?
What FEATURES
do we prioritize right now?
Examples
Do some users binge watch shows? If so which ones? What shows are more likely to be binge watched?
Which features increased IBM Cloud Usage and revenue amongst our customers? How can we double down on them?
Growth Teams Need 3 Types of Product Insights to InnovateWHAT IS PRODUCT ANALYTICS FOR GROWTH?
How do CUSTOMERS
use our products?
What FEATURES
do we prioritize right now?
How are our METRICS
improving?
Examples
Do some users binge watch shows? If so which ones? What shows are more likely to be binge watched?
Did purchase frequency increase amongst shoppers who received coupon code notifications last week?
Which features increased IBM Cloud Usage and revenue amongst our customers? How can we double down on them?
Most Tools are not Designed to Correlate Product & ImpactWHAT IS PRODUCT ANALYTICS FOR GROWTH?
Product Engineering InfraMarketingBusiness Intelligence
Without Amplitude, growth teams have to rely solely on analysts/ data scientists to answer complex questions
Relying only on Analysts Creates Data BreadlinesWHAT IS PRODUCT ANALYTICS FOR GROWTH?
BI and Query Tools
Production Database
Growth
Marketing
Executives
Analysts
Sales
Finance
Without Amplitude, Growth Teams can’t explore user behavior on their own and spin cycles waiting for analytics facetime
4-8 weeks delay
Why Amplitude
3
Amplitude is your Growth Team’s Personal Data AssistantGROWTH/ PRODUCT DATA SCIENCE AS A SERVICE
Product Growth
Execs
Sales Mrkting
Analysts
Data Scientists
Engineers
Growth Teams can investigate and explore real time data for 90% of their needs while also sharing their insights with the entire organization
Same day answers
Amplitude has unrivalled Depth of Behavioral Analytics THREE LEVELS OF NEW AGE ANALYTICS
LEVEL 1: KPI Monitoring
How many people are using X feature?
LEVEL 2: Problem Diagnosis
What is the drop-off of our onboarding flow?
LEVEL 3: Behavioral Analysis
What user behaviors are causing feature adoption?
VP Growth @
Case Study: Intuit QuickBooks
4
Intuit QuickBooks has become their fastest growing unitMINI CASE STUDY: 3 BUSINESS UNITS AT INTUIT NOW USE AMPLITUDE
Intuit team initially had poor KPIs for their invoicing productMINI CASE STUDY: HOW QUICKBOOKS TEAM LAUNCHED “CONNECT”
How do CUSTOMERS
use our products?
Customers were creating invoices in Quickbooks but not emailing them. Fewer invoices were getting paid
Behavior Problem
Their product team quickly diagnosed the behavior issue MINI CASE STUDY: HOW QUICKBOOKS TEAM LAUNCHED “CONNECT”
How do CUSTOMERS
use our products?
What FEATURES
do we prioritize right now?
Amplitude helped them discover that this was because customers who preferred invoicing through Gmail were not able to get value from Quickbooks. Gmail has >1B DAUs so they prioritized a Quickbooks Gmail plugin
Customers were creating invoices in Quickbooks but not emailing them. Few invoices were getting paid on time.
FeatureInsight
Behavior Problem
They launched their new feature “Connect” w/ great impactMINI CASE STUDY: HOW QUICKBOOKS TEAM LAUNCHED “CONNECT”
How do CUSTOMERS
use our products?
What FEATURES
do we prioritize right now?
How are our METRICS
improving?
Quickbooks Gmail has dramatically increased # invoices paid on time and decreased # clicks needed to get paid opening up a new market opportunity for Intuit
Customers were creating invoices in Quickbooks but not emailing them. Fewer invoices were getting paid
FeatureInsight
Resolution and Impact
Behavior Problem
Amplitude helped them discover that this was because customers who preferred invoicing through Gmail were not able to get value from Quickbooks. Gmail has >1B DAUs so they prioritized a Quickbooks Gmail plugin
Intuit has built a data driven culture for GrowthMINI CASE STUDY: 3 BUSINESS UNITS AT INTUIT NOW USE AMPLITUDE
Connect: Intuit’s Fastest Growing Product
Optimizely
Virtuous CycleAnalytics + Experimentation
Senior Product MarketerOptimizely
David Isquick
Everyone plays a part in growth
Driving Awareness of Those Products
Building Great Products and Experiences
Understanding Product and Marketing Performance
Improving Product and Marketing Performance
Product-driven Marketing-Driven Analytics and Research Driven
Experimentation-Driven
Growth
Enable Rapid Experimentation by Everyone
Built for business teams
See results in real-time Act with confidenceand product
development teams
Always-onexperimentation
Analyze
HypothesizeExperiment
Virtuous CycleAnalytics meets Experimentation
Analyze
HypothesizeExperiment
Virtuous CycleAnalytics meets Experimentation
• Core Metrics• Funnels• Retention• LTV
Analyze
HypothesizeExperiment
Virtuous CycleAnalytics meets Experimentation
• Core Metrics• Funnels• Retention• LTV
• Product improvements
• Marketing improvements
Analyze
HypothesizeExperiment
Virtuous CycleAnalytics meets Experimentation • Website A/B tests
• Product experiments
• Core Metrics• Funnels• Retention• LTV
• Product improvements
• Marketing improvements
Analyze
HypothesizeExperiment
Virtuous CycleIn action
Analyze booking conversion funnel
Hypothesis: Adding More Context Will Help Drive More Bookings
Analyze
HypothesizeExperiment
Virtuous CycleIn action
Analyze booking conversion funnel
Adding a step to funnel to provide more context will increase conversion
Analyze
HypothesizeExperiment
Virtuous CycleIn action
Analyze booking conversion funnel
Split traffic and analyze initial results in terms of conversion
Adding a step to funnel to provide more context will increase conversion
Increase revenue with counterintuitive changes
Adding a step to the booking funnel produced a 5% increase in revenue
Questions?