Lean analytics for startups - Leweb2010

  • Published on
    18-Sep-2014

  • View
    8

  • Download
    2

Embed Size (px)

DESCRIPTION

20-minute speed-run presentation on what metrics and web analytics information startups need to collect. Focuses on companies with a lean methodology, and the kinds of data that will actually help them achieve product/market fit before the money runs out.

Transcript

  • Lean analytics for startups

    Wednesday, December 8, 2010

  • @acroll(if you have questions)

    Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Lean Not lean

    Wednesday, December 8, 2010

  • Ries, Mclure, and Blankare often misquoted.

    Wednesday, December 8, 2010

  • They never said fail faster.

    Wednesday, December 8, 2010

  • Instead:Learn and adapt.

    Wednesday, December 8, 2010

  • Waterfall, agile, and leanThree approaches for three situations.

    Wednesday, December 8, 2010

  • Waterfall approachYou know the problem and the solution.

    Wednesday, December 8, 2010

  • Known set of requirements

    Known ways to satisfy them

    Spec Build Test Launch

    Wednesday, December 8, 2010

  • Agile methodologiesKnow the problem, find the solution

    Wednesday, December 8, 2010

  • Known set of requirements

    Unclear how to satisfy them

    Build Test LaunchViable?Problem

    statement

    Adjust

    Sprints

    Unknown set of requirements

    Wednesday, December 8, 2010

  • Most startupsDont even know what problem they solve.

    Wednesday, December 8, 2010

  • Possible problem space

    Possible viable

    offeringTrial startup

    Possible viable

    offeringTrial startup

    Possible viable offering

    Trial startup

    Possible viable offering

    Trial startup

    You are here

    PIVOT

    Wednesday, December 8, 2010

  • All startups are lean at first.Which has some interesting consequences.

    Wednesday, December 8, 2010

  • Getting features right matters less.

    Knowing whats not working, quickly, matters more.

    Wednesday, December 8, 2010

  • We should matters less.

    What if matters more.

    Wednesday, December 8, 2010

  • Your roadmap matters less.

    Parallel experiments matter more.

    Wednesday, December 8, 2010

  • User groups and surveys matter less.

    Real-world adoption & usage analysis matters more.

    Wednesday, December 8, 2010

  • Business plans matter less.

    Business models matter more.

    Wednesday, December 8, 2010

  • http://www.flickr.com/photos/tinfoilraccoon/197640807/Wednesday, December 8, 2010

  • Cost

    Revenue Lemonade( ) x Units sold + tips

    Lemon + cups + sugar( ) x Units madeMarkers + cardboard( ) x Signs madeStaffing( ) x Hours worked

    Profit!Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • http://www.flickr.com/photos/ethanhein/3207831470/#/

    (maybe not that fun a game)

    Wednesday, December 8, 2010

  • Clouds and analyticsIts never been a better time to start a company.

    Wednesday, December 8, 2010

  • Clouds make experimenting easy, avoid upfront dilution.

    Analytics make experimenting smart & fast.

    Wednesday, December 8, 2010

  • http://www.flickr.com/photos/ecastro/3053916892/

    I love to talk about clouds(but not today)

    Wednesday, December 8, 2010

  • Im going to focus on the other partWhat to watch on your website, if youre a startup.

    Wednesday, December 8, 2010

  • So what could we watch?

    Wednesday, December 8, 2010

  • Analytics(what did they

    do on the site?)

    Competition(what are they

    up to?)

    Performability(could they do

    what they wanted to?)

    VoC(what were

    their motivations?)

    Usability(how did they interact with

    it?)

    Community(what were

    they saying?)

    Hard data

    Soft data

    Complete Web Monitoring

    Wednesday, December 8, 2010

  • http://www.flickr.com/photos/itsgreg/446061432/

    Analytics is the measurement of movement towards your business goals.

    Wednesday, December 8, 2010

  • NEWVISITORSGROWTH

    BOUNCERATE

    LOSS

    CONVERSIONRATE

    GOALVALUE

    xTIME

    ONSITE

    PAGESPER

    VISIT

    NUMBEROF VISITS

    SEARCHES

    TWEETS

    MENTIONS

    ADS SEEN

    ATTENTION ENGAGEMENT CONVERSION

    Wednesday, December 8, 2010

  • Conversions

    Visits

    Shopping cart

    Payment options

    Wednesday, December 8, 2010

  • Conversions

    Visits

    Shopping cart

    Payment options

    KPIsC

    onversionrateBounce

    rate

    Wednesday, December 8, 2010

  • http

    ://w

    ww

    .flic

    kr.c

    om/p

    hoto

    s/m

    rmoo

    rey/

    1606

    5423

    6

    Wednesday, December 8, 2010

  • http://www.flickr.com/photos/duncan/1252272164/Wednesday, December 8, 2010

  • http://www.flickr.com/photos/intherough/3573333256/Wednesday, December 8, 2010

  • http://www.flickr.com/photos/jetheriot/648950773/Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Impact of page load time on average daily searches per user

    -0.60%

    -0.45%

    -0.30%

    -0.15%

    0%

    50ms pre-header 100ms pre-header 200ms post-header 200ms post-ad 400ms post-header

    Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Impact of additional delay on business metrics

    -5.00%

    -3.75%

    -2.50%

    -1.25%

    0%

    50 200 500 1000 2000Queries/visitor Query refinement Revenue/visitorAny clicks Satisfaction

    Wednesday, December 8, 2010

  • Shopzillas makeover

    Wednesday, December 8, 2010

  • 5-12%Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Traffic levels

    0

    2,250

    4,500

    6,750

    9,000

    Optimized Unoptimized

    4,740

    8,505

    Tota

    l num

    ber o

    f visi

    ts

    Visitor experience

    Wednesday, December 8, 2010

  • Bounce rate

    0

    5

    10

    15

    20

    Optimized Unoptimized

    14.35%13.38%

    Visit

    s tha

    t bou

    nced

    Visitor experience

    Wednesday, December 8, 2010

  • Average time on site

    0

    8

    16

    23

    31

    Optimized Unoptimized

    23.8330.17

    Time o

    n site

    (min

    utes

    )

    Visitor experience

    Wednesday, December 8, 2010

  • Pages per visit

    0

    4

    8

    12

    16

    Optimized Unoptimized

    11.04

    15.64

    Aver

    age p

    ages

    seen

    Visitor experience

    Wednesday, December 8, 2010

  • Conversion rateand order value

    0

    5

    10

    15

    20

    Conversion rate Order value

    5.51

    16.07

    Die

    renc

    e due

    to op

    timiza

    tion

    Wednesday, December 8, 2010

  • Community management

    Hugging

    Bunnies

    Retweets

    Friends!

    Wednesday, December 8, 2010

  • Conversions

    Visits

    Shopping cart

    Payment options

    Unpaid search CommunitymentionsEmail

    campaign Banner ad

    Google PageRank Sessions-to-clicks

    ratio

    Cost of ads(CPM)

    Clickthroughrate

    Open rate Opt-out rate?

    Wednesday, December 8, 2010

  • But theres a danger:vanity metrics.

    Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • So what should you watch?The lean analytics shortlist.

    Wednesday, December 8, 2010

  • In a startup, the purpose of analytics is to iterate to a product/market fit before the money runs out.

    If you remember only one thing...

    Wednesday, December 8, 2010

  • Message reach

    Infrastructure health

    Market sentiment

    User engagement

    Business model

    Sustainability

    Viral coefficient

    Amplification

    Performability

    Customer research

    Community sentiment

    Time since last visit

    Core goals

    Extended funnel abandonment

    Movement towards goals

    Cost per engaged visitor

    Peak to average ratio

    Minimum sustainable burn

    Wednesday, December 8, 2010

  • Viral coefficientOn average, for each message they hear, how many people does someone tell?

    1Wednesday, December 8, 2010

  • ------------------------------------------------------Get your free private email at http://www.hotmail.com------------------------------------------------------

    Wednesday, December 8, 2010

  • !Wednesday, December 8, 2010

  • v 1, pt =p0 (1 vt+1) / (1 v) + p0

    http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/Wednesday, December 8, 2010

  • How well are messages being amplified?

    2Wednesday, December 8, 2010

  • 50% 75% 100% 50% = 4

    2 3 1 = 6

    14.5

    Downstream reach (aggregate followers)

    Amplification chance(how relevant the content

    is + what the recipient thinks of the sender)

    25%

    .5

    50%

    2

    50%

    1 = 3.5

    Wednesday, December 8, 2010

  • Repurposing(spread to othercommunities)

    Amplification(virality andmessage spread)

    Conversions

    Visits

    Shopping cart

    Payment options

    Seed (startingcommunity)

    Reach (impressions)

    Wednesday, December 8, 2010

  • Viral message spread

    Conversions

    Visits

    Shopping cart

    Payment options

    Reach (impressions) Emphasis on

    getting viral ratio above 1(Retweeting, Fan, Email

    forward, Reddit upvote, other

    loops)

    Wednesday, December 8, 2010

  • Megablogger proponents

    Conversions

    Visits

    Shopping cart

    Payment options

    Seed (startingcommunity)

    Reach (impressions)

    Emphasis on convincing highly-followed, highly

    acted-upon seed group to spread

    the word.

    Wednesday, December 8, 2010

  • A call to action

    Conversions

    Visits

    Shopping cart

    Payment options

    Reach (impressions)

    Emphasis on maximizing

    impression-to-click ratio within the community

    Wednesday, December 8, 2010

  • Goal attainment: funnel

    700,000s

    1,642

    150RT

    327

    1015

    x $100x $20

    x $7

    20Seed ratio:

    35,000:1

    Followers

    Visitors:0.23%

    Conversions:1.95%

    Revenues:Average: $39.54Median: $20

    Total: $1,005

    Amplification: 2.9%

    2 Repurposing

    2,000

    Wednesday, December 8, 2010

  • How performable are we?Health of the infrastructure (uptime, latency)

    3Wednesday, December 8, 2010

  • 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

    0

    5,000

    Uniq

    ue p

    age

    view

    s

    >4s or error

    2-4s

  • What do visitors think of us?Both on the site (VoC) and in the world (community monitoring)

    4&5Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • How engaged are users?Time since last visit, as a histogram, not an average, compared to baselines.

    6Wednesday, December 8, 2010

  • Days since last visit

    1200

    1000

    800

    600

    400

    200

    01 2 3 4 5 6 7 8 9

    Days since last engagement

    January February

    Disengaged(>10 days)

    25000

    20000

    15000

    10000

    5000

    0

    Num

    ber o

    f use

    rs

    Wednesday, December 8, 2010

  • Core goals driving the businessPick the 3-4 tasks youve set for users. Always know what these are and why they correlate to business growth.

    7Wednesday, December 8, 2010

  • Cost

    Revenue Lemonade( ) x Units sold + tips

    Lemon + cups + sugar( ) x Units madeMarkers + cardboard( ) x Signs madeStaffing( ) x Hours worked

    Profit!

    (Remember the model?)

    Wednesday, December 8, 2010

  • Extended funnel abandonmentNot just on the site: Email open rate

    8Wednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • In the last build, what moved us towards & away from goals?Requires knowing the goals and the key features added

    9Wednesday, December 8, 2010

  • Cost per visitorOperations costs per site visitor

    10Wednesday, December 8, 2010

  • (this had better be greater than the revenues and conversion rate.)

    Wednesday, December 8, 2010

  • Peak-to-average ratioHow much busier are you at busy times?

    What premium do you pay for that surplus?

    Can you decommission well?

    11Wednesday, December 8, 2010

  • http://community.microsoftadvertising.comWednesday, December 8, 2010

  • Wednesday, December 8, 2010

  • Minimum sustainable burn

    12Wednesday, December 8, 2010

  • http://www.flickr.com/photos/dougww/833127501/Wednesday, December 8, 2010

  • Message reach

    Infrastructure health

    Market sentiment

    User engagement

    Business model

    Sustainability

    Viral coefficient

    Amplification

    Performability

    Customer research

    Community sentiment

    Time since last visit

    Core goals

    Extended funnel abandonment

    Movement towards goals

    Cost per engaged visitor

    Peak to average ratio

    Minimum sustainable burn

    12thingsto

    watch

    Wednesday, December 8, 2010

  • Thanks!alistair@bitcurrent.com

    @acrollwww.yearonelabs.com

    www.rpmmtl.com

    www.watchingwebsites.com

    www.bitcurrent.com

    www.human20.com

    Wednesday, December 8, 2010