Lean Analytics Cycle

  • Published on
    11-Aug-2014

  • View
    33.316

  • Download
    9

Embed Size (px)

DESCRIPTION

 

Transcript

<ul><li> Lean Analytics Lean Product &amp; Lean UX Silicon Valley - July 2014 Hiten Shah hnshah@gmail.com Download the slides: h p://kiss.ly/leanprod2014 </li> <li> Continuous Learning &amp; Improvement </li> <li> Dont know where to start? </li> <li> Wondering if youre working on the right things? </li> <li> Or, you just want to hear me babble! </li> <li> HTTP://KISS.LY/LEANLYTICSBOOK </li> <li> Lean Analytics helps you Optimize for Learning </li> <li> Lean Analytics Cycle AVINASH KAUSHIK HTTP://KISS.LY/LEANAC </li> <li> Metrics Hypothesis Experiment Act </li> <li> HTTP://HELLOBAR.COM </li> <li> What is Hello Bar? </li> <li> Metrics Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide whats next </li> <li> Our funnel showed a low installation rate Hello Bar Funnel Metrics Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Totals 15068 1868 716 688 398 % prev step 12.40% 38.33% 96.09% 57.85% % drop off -87.60% -61.67% -3.91% -42.15% % of visitors 100% 12.40% 4.75% 4.57% 2.64% </li> <li> We achieved a 40% increase for our installa2on rate with just this one rst experiment. The Lean Analytics Cycle At Work </li> <li> Original Hello Bar installation screen </li> <li> Metrics Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide whats next </li> <li> Find out why people dont install Hello Bar </li> <li> Didnt Meet My Expectations </li> <li> I Had Trouble Installing It </li> <li> I Dont Have A Website </li> <li> Not Ready To Install </li> <li> Other </li> <li> Hello Bar Experiment Hypothesis If more installa2on op2ons are given, then more people will add Hello Bar to their website due to the need for a WordPress plugin and ability to email the code to someone else. </li> <li> Metrics Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide whats next </li> <li> Created a new ow to test </li> <li> I can install code myself. </li> <li> 41.67% responded with WordPress Plugin </li> <li> 25% responded with email my developer / web designer </li> <li> I need help. </li> <li> Metrics Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide whats next </li> <li> Full funnel metrics for the experiment Variation Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Totals 7456 906 342 329 276 % prev step 12.15% 37.75% 96.20% 83.89% % drop off -87.85% -62.25% -3.80% -16.11% % of visitors 100% 12.15% 4.59% 4.41% 3.70% Control Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Totals 7469 925 353 341 197 % prev step 12.38% 38.16% 96.60% 57.77% % drop off -87.62% -61.84% -3.40% -42.23% % of visitors 100% 12.38% 4.73% 4.57% 2.64% </li> <li> Resulted in a 30% installation rate increase HTTP://ISVALID.ORG Experiment resulted in a 40% improvement </li> <li> ACer a dozen more experiments, our installa2on rate is now hovering between 5% and 7% Thats an addi;onal 89% increase!!! The Lean Analytics Cycle At Work </li> <li> Metrics Hypothesis Experiment Act How You Can Put It Into Practice </li> <li> Metrics Figure out what to improve </li> <li> What gets measured, gets managed. Peter Drucker </li> <li> Baselines DATA DNA EBOOK HTTP://KISS.LY/OPTDATADNA </li> <li> Funnels KISSMETRICS FUNNEL REPORT HTTP://KISS.LY/KMFUNNEL </li> <li> SaaS Funnel KISSMETRICS FUNNEL REPORT HTTP://KISS.LY/KMFUNNEL </li> <li> eCommerce Funnel KISSMETRICS FUNNEL REPORT HTTP://KISS.LY/KMFUNNEL </li> <li> Revenue BUFFER BAREMETRICS HTTP://KISS.LY/BUFFERBARE </li> <li> Spreadsheets and Calculations HELLOBAR HTTP://HELLOBAR.COM Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Created Second Bar Totals 15068 1868 716 688 398 193 % prev step 12.40% 38.33% 96.09% 57.85% 48.49% % drop off -87.60% -61.67% -3.91% -42.15% -51.51% % of visitors 100% 12.40% 4.75% 4.57% 2.64% 1.28% </li> <li> Hypothesis Make an educated guess </li> <li> A solid test hypothesis is an informed solution to a real problem not an arbitrary guess. The more research and data you have to base your hypothesis on, the be er it will be. Michael Aagaard, h p://kiss.ly/abthypo </li> <li> Research Tools &amp; Tac;cs </li> <li> Customer Development HTTP://WWW.CUSTDEV.COM </li> <li> Jobs To Be Done #JTBD HTTP://JOBSTOBEDONE.ORG </li> <li> Usability Testing HTTP://USERTESTING.COM </li> <li> Live Chat HTTP://OLARK.COM </li> <li> Mouse Tracking HTTP://CRAZYEGG.COM </li> <li> Long Form Surveying HTTP://SURVEYMONKEY.COM </li> <li> On-Page Surveying HTTP://QUALAROO.COM </li> <li> Personalized Messaging HTTP://INTERCOM.IO </li> <li> Email a survey to people HTTP://SURVEYMONKEY.COM </li> <li> Ask people why they canceled HTTP://SURVEYMONKEY.COM </li> <li> Get feedback right when they cancel </li> <li> Exit Surveys HTTP://QUALAROO.COM </li> <li> Discover whats missing HTTP://QUALAROO.COM </li> <li> Find out what words people use to describe your product HTTP://QUALAROO.COM </li> <li> Learn about peoples expectations HTTP://INTERCOM.IO </li> <li> Learn what frustrates people HTTP://INTERCOM.IO </li> <li> More actionable reading QUALITATIVE DATA HTTP://KISS.LY/QUALABT </li> <li> Experiment Hypothesis Formula DATA DNA EBOOK HTTP://KISS.LY/OPTDATADNA </li> <li> Experiment Test your guess </li> <li> Only 1 out of 5 tests win. 4 out of 5 tests cost you money. $ </li> <li> 1) Which top metric are we trying to improve? 2) What can you anchor your metric against? 3) What is your hypothesis about why this metric is at x%? 4) What do we expect from this experiment? 5) What was the result and what did we learn from it? HTTP://KISS.LY/6QSAB Answer key questions for each experiment </li> <li> Prioritize your experiments HTTP://KISS.LY/ABPIE </li> <li> Dont copy best practices blindly </li> <li> Learn what works best for you </li> <li> Act Decide whats next </li> <li> Explore the results and make data informed decisions. THE ANALYTICS THAT MATTER TO FACEBOOK HTTP://KISS.LY/FBINFORM </li> <li> KISSmetrics A/B Testing Report KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST </li> <li> We know that people love Google Analytics </li> <li> KISSmetrics Experiment Hypothesis If people authen2cate with Google, then more people will sign up due to their exis2ng familiarity with Google Analy2cs and alignment with our winning headline. </li> <li> KISSmetrics Homepage Test Control </li> <li> KISSmetrics Homepage Test Variation </li> <li> KISSmetrics Homepage Test Signup Conversion Rate KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST </li> <li> KISSmetrics Homepage Test Received Data KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST </li> <li> KISSmetrics Homepage New Control </li> <li> Inspiration </li> <li> KISSmetrics Homepage New Test Variation </li> <li> KISSmetrics Homepage Test Signup Conversion Rate KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST </li> <li> KISSmetrics Homepage Test Received Data KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST </li> <li> Growth! HTTP://KISSMETRICS.COM </li> <li> Metrics Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide whats next </li> <li> Hiten Shah hnshah@gmail.com @hnshah You will get all you want in life if you help enough other people get what they want. Zig Ziglar Download the slides: h p://kiss.ly/leanprod2014 </li> </ul>