Lean Analytics and Local Government - Alistair Croll - Code for America

  • Published on
    12-Sep-2014

  • View
    1.464

  • Download
    1

Embed Size (px)

DESCRIPTION

Slides from a webinar on applying Lean Startup and Analytics to local government initiatives held April 22, 2013 by Code For America.

Transcript

  • LEAN ANALYTICS & LOCAL GOVALISTAIR CROLL

    April 22, 2013

    Tuesday, 23 April, 13

  • Alistair CrollCo-Author, Lean Analytics

    Tuesday, 23 April, 13

  • www.leananalyticsbook.com@leananalytics@byosko | @acroll

    Lean AnalyticsUse data to build a

    better business faster.

    Tuesday, 23 April, 13

  • http://www.flickr.com/photos/itsgreg/446061432/

    Analytics is the measurement of movement towards your business goals.

    Tuesday, 23 April, 13

  • Most startups dont know what theyll be when they grow up.

    Tuesday, 23 April, 13

  • Paypalfirst built for Palmpilots

    Most startups dont know what theyll be when they grow up.

    Tuesday, 23 April, 13

  • Paypalfirst built for Palmpilots

    Freshbookswas invoicing

    for a web design firm

    Most startups dont know what theyll be when they grow up.

    Tuesday, 23 April, 13

  • Paypalfirst built for Palmpilots

    Freshbookswas invoicing

    for a web design firm

    Wikipediawas to be written by

    experts only

    Most startups dont know what theyll be when they grow up.

    Tuesday, 23 April, 13

  • Paypalfirst built for Palmpilots

    Freshbookswas invoicing

    for a web design firm

    Wikipediawas to be written by

    experts only

    Mitelwas a

    lawnmower company

    Most startups dont know what theyll be when they grow up.

    Tuesday, 23 April, 13

  • Hotmailwas a

    database company

    Paypalfirst built for Palmpilots

    Freshbookswas invoicing

    for a web design firm

    Wikipediawas to be written by

    experts only

    Mitelwas a

    lawnmower company

    Most startups dont know what theyll be when they grow up.

    Tuesday, 23 April, 13

  • Hotmailwas a

    database company

    Flickrwas going to be an MMO

    Paypalfirst built for Palmpilots

    Freshbookswas invoicing

    for a web design firm

    Wikipediawas to be written by

    experts only

    Mitelwas a

    lawnmower company

    Most startups dont know what theyll be when they grow up.

    Tuesday, 23 April, 13

  • Hotmailwas a

    database company

    Flickrwas going to be an MMO

    Twitterwas a

    podcasting company

    Paypalfirst built for Palmpilots

    Freshbookswas invoicing

    for a web design firm

    Wikipediawas to be written by

    experts only

    Mitelwas a

    lawnmower company

    Most startups dont know what theyll be when they grow up.

    Tuesday, 23 April, 13

  • Hotmailwas a

    database company

    Flickrwas going to be an MMO

    Twitterwas a

    podcasting company

    Autodeskmade desktop

    automation

    Paypalfirst built for Palmpilots

    Freshbookswas invoicing

    for a web design firm

    Wikipediawas to be written by

    experts only

    Mitelwas a

    lawnmower company

    Most startups dont know what theyll be when they grow up.

    Tuesday, 23 April, 13

  • Kevin Costner is a lousy entrepreneur.

    Dont sell what you can make.Make what you can sell.

    Tuesday, 23 April, 13

  • Tuesday, 23 April, 13

  • The basic Lean message islearn and adapt, fast.

    Tuesday, 23 April, 13

  • What information consumes is rather obvious: it consumes the attention of its recipients.Hence a wealth of information creates a poverty of attention, and a need to allocate that attention eciently among the overabundance of information sources that might consume it.

    (Computers, Communications and the Public Interest, pages 40-41, Martin Greenberger, ed., The Johns Hopkins Press, 1971.)

    Tuesday, 23 April, 13

  • http://www.flickr.com/photos/art_es_anna/288880795/Tuesday, 23 April, 13

  • Tuesday, 23 April, 13

  • Tuesday, 23 April, 13

  • Tuesday, 23 April, 13

  • Lean Analytics lesson 1:Most government projects have an attention or a connectivity problem. This is where you will spend most of your time innovating.

    Tuesday, 23 April, 13

  • Empathy stage:Localmind hacks Twitter

    Stage: Empathy Model: UGC/mobile

    Real-time question and answer platform tied to locations. Needed to find out if a core behavioranswering questions about a place

    happened enough to make the business real

    Tuesday, 23 April, 13

  • Localmind hacks Twitter

    Before writing a line of code, Localmind was concerned that people would never answer questions. This was their biggest risk: if questions went unanswered users would have a

    terrible experience and stop using Localmind. Ran an experiment on Twitter

    Tracked geolocated tweets in Times Square Sent @ messages to people who had just tweeted, asking questions about

    the area: how busy is it; is the subway running on time; is something open; etc.

    The response rate to their tweeted questions was very high. Good enough proxy to de-risk the solution, and convince the team and

    investors that it was worth building Localmind.

    Tuesday, 23 April, 13

  • Tuesday, 23 April, 13

  • Hits A metric from the early, foolish days of the Web. Count people instead.

    Tuesday, 23 April, 13

  • Hits A metric from the early, foolish days of the Web. Count people instead.

    Page views Marginally better than hits. Unless youre displaying ad inventory, count people.

    Tuesday, 23 April, 13

  • Hits A metric from the early, foolish days of the Web. Count people instead.

    Page views Marginally better than hits. Unless youre displaying ad inventory, count people.

    Visits Is this one person visiting a hundred times, or are a hundred people visiting once? Fail.

    Tuesday, 23 April, 13

  • Hits A metric from the early, foolish days of the Web. Count people instead.

    Page views Marginally better than hits. Unless youre displaying ad inventory, count people.

    Visits Is this one person visiting a hundred times, or are a hundred people visiting once? Fail.

    Unique visitors This tells you nothing about what they did, why they stuck around, or if they left.

    Tuesday, 23 April, 13

  • Hits A metric from the early, foolish days of the Web. Count people instead.

    Page views Marginally better than hits. Unless youre displaying ad inventory, count people.

    Visits Is this one person visiting a hundred times, or are a hundred people visiting once? Fail.

    Unique visitors This tells you nothing about what they did, why they stuck around, or if they left.Followers/

    friends/likesCount actions instead. Find out how many followers will do your bidding.

    Tuesday, 23 April, 13

  • Hits A metric from the early, foolish days of the Web. Count people instead.

    Page views Marginally better than hits. Unless youre displaying ad inventory, count people.

    Visits Is this one person visiting a hundred times, or are a hundred people visiting once? Fail.

    Unique visitors This tells you nothing about what they did, why they stuck around, or if they left.Followers/

    friends/likesCount actions instead. Find out how many followers will do your bidding.

    Time on site, or pages/visit

    Poor version of engagement. Lots of time spent on support pages is actually a bad sign.

    Tuesday, 23 April, 13

  • Hits A metric from the early, foolish days of the Web. Count people instead.

    Page views Marginally better than hits. Unless youre displaying ad inventory, count people.

    Visits Is this one person visiting a hundred times, or are a hundred people visiting once? Fail.

    Unique visitors This tells you nothing about what they did, why they stuck around, or if they left.Followers/

    friends/likesCount actions instead. Find out how many followers will do your bidding.

    Time on site, or pages/visit

    Poor version of engagement. Lots of time spent on support pages is actually a bad sign.

    Emails collected How many recipients will act on whats in them?

    Tuesday, 23 April, 13

  • Hits A metric from the early, foolish days of the Web. Count people instead.

    Page views Marginally better than hits. Unless youre displaying ad inventory, count people.

    Visits Is this one person visiting a hundred times, or are a hundred people visiting once? Fail.

    Unique visitors This tells you nothing about what they did, why they stuck around, or if they left.Followers/

    friends/likesCount actions instead. Find out how many followers will do your bidding.

    Time on site, or pages/visit

    Poor version of engagement. Lots of time spent on support pages is actually a bad sign.

    Emails collected How many recipients will act on whats in them?

    Number of downloads

    Outside app stores, downloads alone dont lead to lifetime value. Measure activations/active accounts.

    Tuesday, 23 April, 13

  • 2-sided market model:AirBnB and photography

    Stage: Revenue Model: 2-sided marketplace

    Rental-by-owner marketplace that allows property owners to list and market their houses. Offers a variety of related services as well.

    Tuesday, 23 April, 13

  • AirBnB tests a hypothesis

    The hypothesis: Hosts with professional photography will get more business. And hosts will sign up for professional photography as a service.

    Built a concierge MVP

    Found that professionally photographed listings got 2-3x more bookings than the market average.

    In mid-to-late 2011, AirBnB had 20 photographers in the field taking pictures for hosts.

    Tuesday, 23 April, 13

  • 2008 2009 2010 2011 2012

    2 million

    4 million

    6 million

    8 million

    10 million

    NIGHTS BOOKED

    20 photographers

    Friday, November 9, 12Tuesday, 23 April, 13

  • http://www.flickr.com/photos/bootbearwdc/1243690099/

    Pick the right experiments

    Tuesday, 23 April, 13

  • If it wont changehow you behave,its a

    badmetric.http:/

    /www

    .flick

    r.com

    /pho

    tos/

    circa

    sass

    y/78

    5815

    5676

    /

    Tuesday, 23 April, 13

  • The five Stages of Lean Analytics

    Tuesday, 23 April, 13

  • The five Stages of Lean AnalyticsTh

    e st

    age

    you

    re a

    t

    Tuesday, 23 April, 13

  • The five Stages of Lean Analytics

    Empathy

    The

    stag

    e yo

    ure

    at

    Tuesday, 23 April, 13

  • The five Stages of Lean Analytics

    Empathy

    Stickiness

    The

    stag

    e yo

    ure

    at

    Tuesday, 23 April, 13

  • The five Stages of Lean Analytics

    Empathy

    Stickiness

    Virality

    The

    stag

    e yo

    ure

    at

    Tuesday, 23 April, 13

  • The five Stages of Lean Analytics

    Empathy

    Stickiness

    Virality

    Revenue

    The

    stag

    e yo

    ure

    at

    Tuesday, 23 April, 13

  • The five Stages of Lean Analytics

    Empathy

    Stickiness

    Virality

    Revenue

    Scale

    The

    stag

    e yo

    ure

    at

    Tuesday, 23 April, 13

  • The five Stages of Lean Analytics

    Empathy

    Stickiness

    Virality

    Revenue

    Scale

    The

    stag

    e yo

    ure

    at

    E-commerce SaaS Media

    Mobileapp

    User-gencontent

    2-sidedmarket

    The business youre in

    Tuesday, 23 April, 13

  • The five Stages of Lean Analytics

    Empathy

    Stickiness

    Virality

    Revenue

    Scale

    The

    stag

    e yo

    ure

    at

    E-commerce SaaS Media

    Mobileapp

    User-gencontent

    2-sidedmarket

    The business youre in

    One MetricThat Matters.

    Tuesday, 23 April, 13

  • Lean Analytics lesson 2:Choose one metric around which to rally support, and reject vanity metrics ruthlessly.

    Tuesday, 23 April, 13

  • Choose only one metric.

    Tuesday, 23 April, 13

  • http://www.flickr.com/photos/connortarter/4791605202/

    Metrics are likesqueeze toys.

    Tuesday, 23 April, 13

  • Metrics in practice:The Lean Analytics Cycle

    Tuesday, 23 April, 13

  • Metrics in practice:The Lean Analytics Cycle

    Pick OMTM

    Tuesday, 23 April, 13

  • Metrics in practice:The Lean Analytics Cycle

    Draw a linein the sand

    Pick OMTM

    Tuesday, 23 April, 13

  • Metrics in practice:The Lean Analytics Cycle

    Find a potential

    improvement

    Draw a linein the sand

    Pick OMTM

    Tuesday, 23 April, 13

  • Metrics in practice:The Lean Analytics Cycle

    Without data: make a good guess

    Find a potential

    improvement

    Draw a linein the sand

    Pick OMTM

    Tuesday, 23 April, 13

  • Metrics in practice:The Lean Analytics Cycle

    With data:find a

    commonality

    Without data: make a good guess

    Find a potential

    improvement

    Draw a linein the sand

    Pick OMTM

    Tuesday, 23 April, 13

  • Hypothesis

    Metrics in practice:The Lean Analytics Cycle

    With data:find a

    commonality

    Without data: make a good guess

    Find a potential

    improvement

    Draw a linein the sand

    Pick OMTM

    Tuesday, 23 April, 13

  • Make changes in production

    Hypothesis

    Metrics in practice:The Lean Analytics Cycle

    With data:find a

    commonality

    Without data: make a good guess

    Find a potential

    improvement

    Draw a linein the sand

    Pick OMTM

    Tuesday, 23 April, 13

  • Make changes in production

    Design a test

    Hypothesis

    Metrics in practice:The Lean Analytics Cycle

    With data:find a

    commonality

    Without data: make a good guess

    Find a potential

    improvement

    Draw a linein the sand

    Pick OMTM

    Tuesday, 23 April, 13

  • Measure the results

    Make changes in production

    Design a test

    Hypothesis

    Metrics in practice:The Lean Analytics Cycle

    With data:find a

    commonality

    Without data: make a good guess

    Find a potential

    improvement

    Draw a linein the sand

    Pick OMTM

    Tuesday, 23 April, 13

  • Did we move the needle?

    Measure the results

    Make changes in production

    Design a test

    Hypothesis

    Metrics in practice:The Lean Analytics Cycle

    With data:find a

    commonality

    Without data: make a good guess

    Find a potential

    improvement

    Draw a linein the sand

    Pick OMTM

    Tuesday, 23 April, 13

  • Success!

    Did we move the needle?

    Measure the results

    Make changes in production

    Design a test

    Hypothesis

    Metrics in practice:The Lean Analytics Cycle

    With data:find a

    commonality

    Without data: make a good guess

    Find a potential

    improvement

    Draw a linein the sand

    Pick OMTM

    Tuesday, 23 April, 13

  • Pivot orgive up

    Success!

    Did we move the needle?

    Measure the results

    Make changes in production

    Design a test

    Hypothesis

    Metrics in practice:The Lean Analytics Cycle

    With data:find a

    commonality

    Without data: make a good guess

    Find a potential

    improvement

    Draw a linein the sand

    Pick OMTM

    Tuesday, 23 April, 13

  • Draw a new linePivot orgive up

    Success!

    Did we move the needle?

    Measure the results

    Make changes in production

    Design a test

    Hypothesis

    Metrics in practice:The Lean Analytics Cycle

    With data:find a

    commonality

    Without data: make a good guess

    Find a potential

    improvement

    Draw a linein the sand

    Pick OMTM

    Tuesday, 23 April, 13

  • Draw a new linePivot orgive up

    Try again

    Success!

    Did we move the needle?

    Measure the results

    Make changes in production

    Design a test

    Hypothesis

    Metrics in practice:The Lean Analytics Cycle

    With data:find a

    commonality

    Without data: make a good guess

    Find a potential

    improvement

    Draw a linein the sand

    Pick OMTM

    Tuesday, 23 April, 13

  • Lean Analytics lesson 3:Theres no finished. Just more iterations.

    Tuesday, 23 April, 13

  • The B2B stereotype

    http

    ://ww

    w.te

    chdi

    gest

    .tv/2

    007/

    02/im

    _a_p

    c_im

    _a_m

    a.ht

    ml

    Domainexpert

    Disruptionexpert Operations

    Domain expert knows industry and the problem domain. Has a Rolodex; proxy for customers.

    Disruption expert knows tech that will produce a change Sees beyond the current model.

    Tuesday, 23 April, 13

  • The B2B stereotype

    http

    ://ww

    w.te

    chdi

    gest

    .tv/2

    007/

    02/im

    _a_p

    c_im

    _a_m

    a.ht

    ml

    Domainexpert

    Disruptionexpert Operations

    Domain expert knows industry and the problem domain. Has a Rolodex; proxy for customers.

    Disruption expert knows tech that will produce a change Sees beyond the current model.

    Tuesday, 23 April, 13

  • Three typical approaches

    Enterprise pivot

    Copy and rebuild

    Disrupt a problem

    Create a popular consumer product then pivot to tackle the enterprise

    Dropbox

    Take an existing consumer or open source idea and make it enterprise-ready

    Yammer, MapR

    Convince the enterprise to discard the old way because of overwhelming advantages.

    Taleo, Google Apps

    Tuesday, 23 April, 13

  • Lean Analytics lifecyclefor an enterprise-focused startup

    Empathy Consulting to test ideas and bootstrap the businessLock-in, IP control, overfitting

    Stage Do this Fear this

    Tuesday, 23 April, 13

  • Lean Analytics lifecyclefor an enterprise-focused startup

    Empathy Consulting to test ideas and bootstrap the businessLock-in, IP control, overfitting

    Stickiness Standardization and integration; shift f...