Lean Analytics for Intrapreneurs by Allistair Croll

  • Published on

  • View

  • Download




<p>Lean Analyticsfor IntrapreneursLean Startup Conference 2013</p> <p>@acroll</p> <p>When youre a startupyour goal is to find a sustainable, </p> <p>repeatable business model.</p> <p>When youre a big company your goal is to perpetuate one.</p> <p>Lean Analytics:Use data to build a better business faster.</p> <p>Intrapreneur:Someone working to produce </p> <p>disruptive change in an organization that has already found a sustainable, </p> <p>repeatable business model.</p> <p>(Before we get into Lean Analytics, 2 key lessons.)</p> <p>Lesson one:Companies die because they fail to </p> <p>move to new business models.</p> <p>Clay Christensen, The Innovators Dilemma</p> <p>Cost</p> <p> per</p> <p> MB</p> <p>$1000</p> <p>$100</p> <p>$10</p> <p>$1</p> <p>Time</p> <p>14</p> <p>Mainfr</p> <p>ame</p> <p>8</p> <p>Minico</p> <p>mpute</p> <p>r</p> <p>5.25</p> <p>Deskt</p> <p>op</p> <p>3.5</p> <p>Noteb</p> <p>ook</p> <p>Technologies outstrip what the market needs, driven by feedback from the best current customer.</p> <p>Clay Christensen, The Innovators Dilemma</p> <p>$1000</p> <p>$100</p> <p>$10</p> <p>$1</p> <p>Time</p> <p>8 5.25</p> <p>High end </p> <p>customer Low end</p> <p>customer</p> <p>The new market has different criteria for success, which are uninteresting to incumbents.</p> <p>Clay Christensen, The Innovators Dilemma</p> <p>$1000</p> <p>$100</p> <p>$10</p> <p>$1</p> <p>Time</p> <p>Storagecapacity</p> <p>Portability</p> <p>Sometimes this has unintended consequences</p> <p>Clay Christensen, The Innovators Dilemma</p> <p>$1000</p> <p>$100</p> <p>$10</p> <p>$1</p> <p>Time</p> <p>Smaller disc size means less vibration impact, leading to greater density, increasing storage capacity</p> <p>Three kinds of innovation</p> <p>Sustain/core(optimizing for more of the same)</p> <p>Innovate/adjacent(introduce nearby product,</p> <p>market, or method)</p> <p>Disrupt/transformative(Fundamentally changing</p> <p>the business model)</p> <p>Improve alongcurrent metrics...</p> <p>...or alterthe rate of improvement</p> <p>Switch to a newvalue model</p> <p>Change the businessmodel entirely</p> <p>Lesson two:The difference between a rogue agent and a special operative is permission.</p> <p>Its not your job to prove youre clever.Its your job to change outcomes.</p> <p>A quick introductionto Lean Analytics</p> <p>Dont sell what you can make. Make what you can sell.</p> <p>Kevin Costner is a lousy entrepreneur.</p> <p>The core of Lean is iteration.</p> <p>Unfortunately,were all liars.</p> <p>Everyones idea is the best right?</p> <p>People love this part!</p> <p>(but thats not always a good thing)</p> <p>This is where things fall apart.</p> <p>No data, no learning.</p> <p>Analytics can help.</p> <p>Analytics is the measurement of movement towards your business </p> <p>goals.</p> <p>In a startup, the purpose of analytics is to iterate to product/market fit </p> <p>before the money runs out.</p> <p>In a big company,analytics replaces opinion with fact.</p> <p>It helps you innovate on product, market, and method.</p> <p>Some fundamentals.</p> <p>A good metric is:</p> <p>Understandable</p> <p>If youre busy explaining the data, you wont be busy acting on it.</p> <p>Comparative</p> <p>Comparison is context.</p> <p>A ratio or rate</p> <p>The only way to measure change and roll up the tension between two metrics (MPH)</p> <p>Behaviorchanging</p> <p>If youre busy explaining the data, you wont be busy acting on it.</p> <p>Thesimplestrule</p> <p>badmetric.</p> <p>If a metric wont change how you behave, its a</p> <p>h"p://www.ickr.com/photos/circasassy/7858155676/</p> <p>Metrics help you know yourself.</p> <p>Acquisition</p> <p>Hybrid</p> <p>Loyalty</p> <p>70%of retailers</p> <p>20%of retailers</p> <p>10%of retailers</p> <p>You are just like</p> <p>Customers that buy &gt;1x in 90d</p> <p>Once</p> <p>2-2.5per year</p> <p>&gt;2.5per year</p> <p>Your customers will buy from you</p> <p>Then you are in this mode</p> <p>1-15%</p> <p>15-30%</p> <p>&gt;30%</p> <p>Low acquisition cost, high checkout</p> <p>Increasing return rates, market share</p> <p>Loyalty, selection, inventory size</p> <p>Focus on</p> <p>(Thanks to Kevin Hillstrom for this.)</p> <p>Qualitative</p> <p>Unstructured, anecdotal, revealing, hard to aggregate, often too positive &amp; reassuring.</p> <p>Warm and fuzzy.</p> <p>Quantitative</p> <p>Numbers and stats. Hard facts, less insight, easier to analyze; often sour and disappointing.</p> <p>Cold and hard.</p> <p>Exploratory</p> <p>Speculative. Tries to find unexpected or interesting insights. Source of unfair advantages.</p> <p>Cool.</p> <p>Reporting</p> <p>Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception.</p> <p>Necessary.</p> <p>Rumsfeld on Analytics</p> <p>(Or rather, Avinash Kaushik channeling Rumsfeld)</p> <p>Things we</p> <p>know</p> <p>dontknow</p> <p>we know Are facts which may be wrong and should be checked against data.</p> <p>we dontknow</p> <p>Are questions we can answer by reporting, which we should baseline &amp; automate.</p> <p>we knowAre intuition which we should quantify and teach to improve effectiveness, efficiency.</p> <p>we dontknow</p> <p>Are exploration which is where unfair advantage and interesting epiphanies live.</p> <p>MayAprMarFeb</p> <p>Slicing and dicing data</p> <p>Jan0</p> <p>5,000</p> <p>Activ</p> <p>e use</p> <p>rs</p> <p>Cohort:Comparison of similar groups along a timeline.(this is the April cohort)</p> <p>A/B test:Changing one thing (i.e. color) and measuring the result (i.e. revenue.)</p> <p>MultivariateanalysisChanging several things at once to see which correlates with a result.</p> <p>Segment:Cross-sectional </p> <p>comparison of all people divided by </p> <p>some attribute (age, gender, etc.)</p> <p>Which of these two companiesis doing better?</p> <p> January February March April May</p> <p>Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50Is this company growing or stagnating?</p> <p>Cohort 1 2 3 4 5</p> <p>January</p> <p>February</p> <p>March</p> <p>April</p> <p>May</p> <p>$5 $3 $2 $1 $0.5</p> <p>$6 $4 $2 $1</p> <p>$7 $6 $5</p> <p> $8 $7</p> <p> $9</p> <p>How about this one?</p> <p>Cohort 1 2 3 4 5</p> <p>January</p> <p>February</p> <p>March</p> <p>April</p> <p>May</p> <p>Averages</p> <p>$5 $3 $2 $1 $0.5</p> <p>$6 $4 $2 $1 </p> <p>$7 $6 $5 </p> <p>$8 $7 </p> <p>$9 </p> <p>$7 $5 $3 $1 $0.5</p> <p>Look at the same data in cohorts</p> <p>The ability to do things in small batches is behind much of the lean movement.</p> <p>Short cycle time triggers the immune system of big company budgeting.</p> <p>(this is called foreshadowing.)</p> <p>Lagging</p> <p>Historical. Shows you how youre doing; reports the news. Example: sales.</p> <p>Explaining the past.</p> <p>Leading</p> <p>Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline.</p> <p>Predicting the future.</p> <p>A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya)</p> <p>If someone comes back to Zynga a day after signing up for a game, theyll probably become an engaged, paying user (Nabeel Hyatt)</p> <p>A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang)</p> <p>Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman)A LinkedIn user getting to X connections in Y days (Elliot Schmukler)</p> <p>Some examples</p> <p>(From the 2012 Growth Hacking conference. http://growthhackersconference.com/)</p> <p>Which means its time to talk about correlation.</p> <p>1</p> <p>10</p> <p>100</p> <p>1000</p> <p>10000</p> <p>Ice cream consumption DrowningsJan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec</p> <p>Correlated</p> <p>Two variables that are related (but may be dependent on something else.)</p> <p>Ice cream &amp; drowning.</p> <p>Causal</p> <p>An independent variable that directly impacts a dependent one.</p> <p>Summertime &amp; drowning.</p> <p>A leading, causal metricis a superpower.</p> <p>h"p://www.ickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/</p> <p>Growth hacking, demystified.</p> <p>Find correlation</p> <p>Test causality</p> <p>Optimize the causal factor</p> <p>Pick a metric to change</p> <p>Is social action a leading indicator of donation?</p> <p>http</p> <p>://bl</p> <p>og.ju</p> <p>stgiv</p> <p>ing.c</p> <p>om/n</p> <p>ine-re</p> <p>ason</p> <p>s-wh</p> <p>y-so</p> <p>cial-a</p> <p>nd-m</p> <p>obile</p> <p>-are</p> <p>-the-</p> <p>futu</p> <p>re-o</p> <p>f-fun</p> <p>drais</p> <p>ing/</p> <p>Is mobile use?ht</p> <p>tp://</p> <p>blog</p> <p>.just</p> <p>giving</p> <p>.com</p> <p>/nine</p> <p>-reas</p> <p>ons-</p> <p>why-</p> <p>socia</p> <p>l-and</p> <p>-mob</p> <p>ile-a</p> <p>re-th</p> <p>e-fu</p> <p>ture</p> <p>-of-f</p> <p>undr</p> <p>aising</p> <p>/</p> <p>The Lean Analytics framework.</p> <p>Erics three engines of growth</p> <p>Virality</p> <p>Make people invite friends.</p> <p>How many they tell, how fast they </p> <p>tell them.</p> <p>Price</p> <p>Spend money to get customers.</p> <p>Customers are worth more than </p> <p>they cost.</p> <p>Stickiness</p> <p>Keep people coming back.</p> <p>Approach</p> <p>Get customers faster than you </p> <p>lose them.</p> <p>Math that matters</p> <p>Daves Pirate MetricsAARRR</p> <p>AcquisitionHow do your users become aware of you?</p> <p>SEO, SEM, widgets, email, PR, campaigns, blogs ...</p> <p>ActivationDo drive-by visitors subscribe, use, etc?</p> <p>Features, design, tone, compensation, affirmation ...</p> <p>RetentionDoes a one-time user become engaged?</p> <p>Notifications, alerts, reminders, emails, updates...</p> <p>RevenueDo you make money from user activity?</p> <p>Transactions, clicks, subscriptions, DLC, analytics...</p> <p>ReferralDo users promote your product?</p> <p>Email, widgets, campaigns, likes, RTs, affiliates...</p> <p>Stage</p> <p>EMPATHY Ive found a real, poorly-met need that a reachable market faces.</p> <p>STICKINESS Ive figured out how to solve the problem in a way they will keep using and pay for.</p> <p>VIRALITY Ive found ways to get them to tell their friends, either intrinsically or through incentives.</p> <p>REVENUE The users and features fuel growth organically and artificially.</p> <p>SCALE Ive found a sustainable, scalable business with the right margins in a healthy ecosystem.</p> <p>GateTh</p> <p>e fiv</p> <p>e st</p> <p>ages</p> <p>Empathy stage:Localmind hacks Twitter</p> <p>Needed to find out if a core assumptionstrangers answering questionswas valid.Ran Twitter experiment instead of writing codeAsked senders of geolocated Tweets from Times Square random questions; counted response rateConclusion: high enough to proceed</p> <p>Stickiness stage:qidiq streamlines invites</p> <p>Survey owner adds recipient to groupSurvey owner asks question</p> <p>Recipient reads survey questionRecipient responds to questionRecipient sees survey results</p> <p>(Later, if needed)Recipient visits site; no password!Recipient does password recovery</p> <p>One-time link sent to emailRecipient creates password</p> <p>Recipient can edit profile, etc.</p> <p>Survey owner adds recipient to group</p> <p>Survey owner asks question</p> <p>Recipient gets invite</p> <p>Recipient reads survey question</p> <p>Recipient responds to question</p> <p>Recipient installs mobile app</p> <p>Recipient creates account, profile</p> <p>Recipient sees survey results</p> <p>Recipient can edit profile, etc.</p> <p>10-2</p> <p>5% R</p> <p>ESPO</p> <p>NSE R</p> <p>ATE</p> <p>70-9</p> <p>0% R</p> <p>ESPO</p> <p>NSE R</p> <p>ATE</p> <p>Six business model archetypes(Yours is probably a blend of these.)</p> <p> E-commerce SaaS (freemium?) Mobile app (gaming) Two sided marketplace Media User generated content</p> <p>(Which means eye charts like these.)</p> <p>Customer Acquisition Cost</p> <p>paid direct search wom inherent virality</p> <p>VISITOR</p> <p>Freemium/trial oer</p> <p>Enrollment</p> <p>User</p> <p>Disengaged User</p> <p>Cancel</p> <p>Freemium churn</p> <p>Engaged UserFree user </p> <p>disengagement</p> <p>Reactivate</p> <p>Cancel</p> <p>Trial abandonment rate</p> <p>Invite Others</p> <p>Paying Customer</p> <p>Reactivationrate</p> <p>Paid conversion</p> <p>FORMER USERS</p> <p>User Lifetime Value</p> <p>Reactivate</p> <p>FORMER CUSTOMERS</p> <p>Customer Lifetime Value</p> <p>Viral coecientViral rate</p> <p>Resolution</p> <p>Support data</p> <p>Account Cancelled Billing Info Exp.Paid Churn Rate</p> <p>Tiering</p> <p>Capacity Limit</p> <p>Upselling rate Upselling</p> <p>Disengaged DissatisfiedTrial Over</p> <p>Model + Stage = One Metric That Matters.</p> <p>One MetricThat Matters.</p> <p>The business youre in</p> <p>E-Com SaaS Mobile 2-Sided Media UCGEmpathy</p> <p>StickinessVirality</p> <p>RevenueScaleTh</p> <p>e st</p> <p>age </p> <p>you</p> <p>re a</p> <p>t</p> <p>Really? Just one?</p> <p>Yes, one.</p> <p>In a startup, focus is hard to achieve.</p> <p>Having only one metricaddresses this problem.</p> <p>www.theeastsiderla.com</p> <p>Moz cuts down on metricsSaaS-based SEO toolkit in the scale stage. Focused on net adds.</p> <p> Was a marketing campaign successful? Were customer complaints lowered? Was a product upgrade valuable?</p> <p>Net adds up:</p> <p> Can we acquire more valuable customers? What product features can increase engagement? Can we improve customer support?</p> <p>Net adds flat:</p> <p> Are the new customers not the right segment? Did a marketing campaign fail? Did a product upgrade fail somehow? Is customer support falling apart?</p> <p>Net adds down:</p> <p>Metrics are like squeeze toys.</p> <p>http://www.flickr.com/photos/connortarter/4791605202/</p> <p>Empathy</p> <p>Stickiness</p> <p>Virality</p> <p>Revenue</p> <p>Scale</p> <p>E-commerce SaaS Media</p> <p>Mobileapp</p> <p>User-gencontent</p> <p>2-sidedmarket</p> <p>Interviews; qualitative results; quantitative scoring; surveys</p> <p>Loyalty, conversion</p> <p>CAC, shares, reactivation</p> <p>Transaction, CLV</p> <p>Affiliates, white-label</p> <p>Engagement, churn</p> <p>Inherent virality, CAC</p> <p>Upselling, CAC, CLV</p> <p>API, magic #, mktplace</p> <p>Content, spam</p> <p>Invites, sharing</p> <p>Ads, donations</p> <p>Analytics, user data</p> <p>Inventory, listings</p> <p>SEM, sharing</p> <p>Transactions, commission</p> <p>Other verticals</p> <p>(Money from transactions)</p> <p>Downloads, churn, virality</p> <p>WoM, app ratings, CAC</p> <p>CLV, ARPDAU</p> <p>Spinoffs, publishers</p> <p>(Money from active users)</p> <p>Traffic, visits, returns</p> <p>Content virality, SEM</p> <p>CPE, affiliate %, eyeballs</p> <p>Syndication, licenses</p> <p>(Money from ad clicks)</p> <p>Better: bit.ly/BigLeanTable</p> <p>A company loses a quarter of its customers every year.</p> <p>Is this good or bad?</p> <p>Not knowing what normal ismakes you do stupid things.</p> <p>Listening to what normal ismakes you ignore disruptive things.</p> <p>(more foreshadowing.)</p> <p>Baseline:5-7% growth a week</p> <p>A good growth rate during YC is 5-7% a week, he says. If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing. At revenue stage, measure growth in revenue. Before that, measure growth in active users.</p> <p>Paul Graham, Y Combinator</p> <p> Are there enough people who really care enough to sustain a 5% growth rate? Dont strive for a 5% growth at the expense </p> <p>of really understanding your customers and building a meaningful solution Once youre a pre-revenue startup at or </p> <p>near product/market fit, you should have 5% growth of active users each week Once youre generating revenues, they </p> <p>should grow at 5% a week</p> <p>Baseline:10% visitor engagement/day</p> <p>Fred Wilsons social ratios</p> <p>30% of users/month use web or mobile app10% of users/day use web or mobile app1% of users/day use it concurrently</p> <p>Baseline:2-5% monthly churn The best SaaS get 1.5% - 3% a month. They have multiple Ph.Ds </p> <p>on the job. Get below a 5% monthly churn rate before you know youve got a </p> <p>business thats ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok)</p> <p> Last-ditch appeals and reactivation can have a big impact. Facebooks dont leave reduces attrition by 7%.</p> <p>Baseline:CLV calculation25%</p> <p>monthly churn</p> <p>100/25=4The average </p> <p>customer lasts 4 months</p> <p>5%monthly churn</p> <p>100/5=20The average </p> <p>customer lasts 20 months</p> <p>2%monthly churn</p> <p>100/2=50The average </p> <p>customer lasts 50 months</p> <p>Baseline:CAC under 1/3 of CLV CLV is wrong. CAC Is probably wrong, too. Time kills all plans: Itll take a long time to find </p> <p>out whether your churn and revenue projections are right</p> <p> Cashflow: Youre basically loaning the custome...</p>


View more >