Download pdf - Lean Analytics and Local Government - Alistair Croll - Code for America

LEAN ANALYTICS & LOCAL GOVALISTAIR CROLL

April 22, 2013

Tuesday, 23 April, 13

Alistair CrollCo-Author, Lean Analytics


www.leananalyticsbook.com@leananalytics@byosko | @acroll

Lean AnalyticsUse data to build a

better business faster.


http://www.leananalyticsbook.com

http://www.leananalyticsbook.com

http://www.flickr.com/photos/itsgreg/446061432/

Analytics is the measurement of movement towards your business goals.




Most startups don’t know what they’ll be when they grow up.


Paypalfirst built for Palmpilots




Freshbookswas invoicing

for a web design firm






Wikipediawas to be written by

experts only







experts only

Mitelwas a

lawnmower company



Hotmailwas a

database company





experts only

Mitelwas a

lawnmower company



Hotmailwas a

database company

Flickrwas going to be an MMO





experts only

Mitelwas a

lawnmower company



Hotmailwas a

database company


Twitterwas a

podcasting company





experts only

Mitelwas a

lawnmower company



Hotmailwas a

database company


Twitterwas a

podcasting company

Autodeskmade desktop

automation





experts only

Mitelwas a

lawnmower company



Kevin Costner is a lousy entrepreneur.

Don’t sell what you can make.Make what you can sell.



The basic Lean message islearn and adapt, fast.


“What information consumes is rather obvious: it consumes the attention of its recipients.Hence a wealth of information creates a poverty of attention, and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.”

(Computers, Communications and the Public Interest, pages 40-41, Martin Greenberger, ed., The Johns Hopkins Press, 1971.)


http://www.flickr.com/photos/art_es_anna/288880795/Tuesday, 23 April, 13

http://www.flickr.com/photos/art_es_anna/288880795/

http://www.flickr.com/photos/art_es_anna/288880795/




Lean Analytics lesson 1:Most government projects have an attention or a connectivity problem. This is where you will spend most of your time innovating.


Empathy stage:Localmind hacks Twitter

• Stage: Empathy• Model: UGC/mobile

• Real-time question and answer platform tied to locations.• Needed to find out if a core behavior—answering questions about a place—

happened enough to make the business real


Localmind hacks Twitter

• Before writing a line of code, Localmind was concerned that people would never answer questions.• This was their biggest risk: if questions went unanswered users would have a

terrible experience and stop using Localmind.• Ran an experiment on Twitter

• Tracked geolocated tweets in Times Square• Sent @ messages to people who had just tweeted, asking questions about

the area: how busy is it; is the subway running on time; is something open; etc.

• The response rate to their tweeted questions was very high.• Good enough proxy to de-risk the solution, and convince the team and

investors that it was worth building Localmind.



Hits A metric from the early, foolish days of the Web. Count people instead.



Page views Marginally better than hits. Unless you’re displaying ad inventory, count people.




Visits Is this one person visiting a hundred times, or are a hundred people visiting once? Fail.





Unique visitors This tells you nothing about what they did, why they stuck around, or if they left.






Followers/friends/likes

Count actions instead. Find out how many followers will do your bidding.








Time on site, or pages/visit

Poor version of engagement. Lots of time spent on support pages is actually a bad sign.










Emails collected How many recipients will act on what’s in them?










Emails collected How many recipients will act on what’s in them?

Number of downloads

Outside app stores, downloads alone don’t lead to lifetime value. Measure activations/active accounts.


2-sided market model:AirBnB and photography

• Stage: Revenue• Model: 2-sided marketplace

• Rental-by-owner marketplace that allows property owners to list and market their houses. Offers a variety of related services as well.


AirBnB tests a hypothesis

• The hypothesis: “Hosts with professional photography will get more business. And hosts will sign up for professional photography as a service.”

• Built a concierge MVP

• Found that professionally photographed listings got 2-3x more bookings than the market average.

• In mid-to-late 2011, AirBnB had 20 photographers in the field taking pictures for hosts.


2008 2009 2010 2011 2012

2 million

4 million

6 million

8 million

10 million

NIGHTS BOOKED

20 photographers

Friday, November 9, 12Tuesday, 23 April, 13

http://www.flickr.com/photos/bootbearwdc/1243690099/

Pick the right experiments




If it won’t changehow you behave,it’s a

badmetric.http

://w

ww.

flickr.

com

/pho

tos/

circa

sass

y/78

5815

5676

/


http://www.flickr.com/photos/circasassy/7858155676/

http://www.flickr.com/photos/circasassy/7858155676/

The five Stages of Lean Analytics


The five Stages of Lean AnalyticsTh

e st

age

you’

re a

t



Empathy

The

stag

e yo

u’re

at



Empathy

Stickiness

The

stag

e yo

u’re

at



Empathy

Stickiness

Virality

The

stag

e yo

u’re

at



Empathy

Stickiness

Virality

Revenue

The

stag

e yo

u’re

at



Empathy

Stickiness

Virality

Revenue

Scale

The

stag

e yo

u’re

at



Empathy

Stickiness

Virality

Revenue

Scale

The

stag

e yo

u’re

at

E-commerce SaaS MediaMobile

appUser-gencontent

2-sidedmarket

The business you’re in



Empathy

Stickiness

Virality

Revenue

Scale

The

stag

e yo

u’re

at

E-commerce SaaS MediaMobile

appUser-gencontent

2-sidedmarket

The business you’re in

One MetricThat Matters.


Lean Analytics lesson 2:Choose one metric around which to rally support, and reject vanity metrics ruthlessly.


Choose only one metric.


http://www.flickr.com/photos/connortarter/4791605202/

Metrics are likesqueeze toys.




Metrics in practice:The Lean Analytics Cycle



Pick OMTM



Draw a linein the sand

Pick OMTM



Find a potential

improvement


Pick OMTM



Without data: make a good guess

Find a potential

improvement


Pick OMTM



With data:find a

commonality


Find a potential

improvement


Pick OMTM


Hypothesis


With data:find a

commonality


Find a potential

improvement


Pick OMTM


Make changes in production

Hypothesis


With data:find a

commonality


Find a potential

improvement


Pick OMTM



Design a test

Hypothesis


With data:find a

commonality


Find a potential

improvement


Pick OMTM


Measure the results


Design a test

Hypothesis


With data:find a

commonality


Find a potential

improvement


Pick OMTM


Did we move the needle?

Measure the results


Design a test

Hypothesis


With data:find a

commonality


Find a potential

improvement


Pick OMTM


Success!


Measure the results


Design a test

Hypothesis


With data:find a

commonality


Find a potential

improvement


Pick OMTM


Pivot orgive up

Success!


Measure the results


Design a test

Hypothesis


With data:find a

commonality


Find a potential

improvement


Pick OMTM


Draw a new linePivot orgive up

Success!


Measure the results


Design a test

Hypothesis


With data:find a

commonality


Find a potential

improvement


Pick OMTM


Draw a new linePivot orgive up

Try again

Success!


Measure the results


Design a test

Hypothesis


With data:find a

commonality


Find a potential

improvement


Pick OMTM


Lean Analytics lesson 3:There’s no “finished.” Just more iterations.


The B2B stereotype

http

://w

ww.

tech

dige

st.tv

/200

7/02

/im_a

_pc_

im_a

_ma.

htm

l

Domainexpert

Disruptionexpert Operations

• Domain expert knows industry and the problem domain. Has a Rolodex; proxy for customers.

• Disruption expert knows tech that will produce a change Sees beyond the current model.


http://www.techdigest.tv/2007/02/im_a_pc_im_a_ma.html


The B2B stereotype

http

://w

ww.

tech

dige

st.tv

/200

7/02

/im_a

_pc_

im_a

_ma.

htm

l

Domainexpert

Disruptionexpert Operations

• Domain expert knows industry and the problem domain. Has a Rolodex; proxy for customers.

• Disruption expert knows tech that will produce a change Sees beyond the current model.




Three typical approaches

Enterprise pivot

Copy and rebuild

Disrupt a problem

Create a popular consumer product then pivot to tackle the enterprise

Dropbox

Take an existing consumer or open source idea and make it enterprise-ready

Yammer, MapR

Convince the enterprise to discard the old way because of overwhelming advantages.

Taleo, Google Apps


Lean Analytics lifecyclefor an enterprise-focused startup

Empathy Consulting to test ideas and bootstrap the business

Lock-in, IP control, overfitting

Stage Do this Fear this





Stickiness Standardization and integration; shift from custom to generic

Ability to integrate; support








Virality Word of mouth, references, case studies

Bad vibes; exclusivity










Revenue Growing direct sales, professional services, support

Pipeline, revenue recognition, comp










Revenue Growing direct sales, professional services, support

Pipeline, revenue recognition, comp

Scale Channels, analysts, ecosystems, APIs, vertically targeted products

Crossing the chasm; Gorillas



The Zero Overhead principle

A central theme to this new wave of innovation is the application of core product tenets from the consumer space to the enterprise.In particular, a universal lesson that I keep sharing with all entrepreneurs building for the enterprise is the Zero Overhead Principle: no feature may add training costs to the user. DJ Patil


Lean Analytics lesson 4:Government can learn from enterprise-focused startups: Disrupt a known problem with new technology.


Skunk Works for intrapreneurs

• The Lockheed Martin Skunk Works


Span of control and the railroads

• Daniel C. McCallum


The BCG matrix

Question marks!(low market share, high growth rate)

May be the next big thing. Consumes investment, but

will require money to increase market share.

Stars!(high growth rate, high market share)

What everyone wants. As market invariably stops

growing, should become cash cows.

Dogs!(low market share, low growth rate)

Barely breaks even, may be a distraction from better

opportunities. Sell off or shut down.

Cash cows!(high market share,

low growth rate) Boring sources of cash, to be milked but not worth additional investment.

Gro

wth

rate

Market share

Pivot to increasemarketshare

throughvirality,

attention

Pivot toincrease growth

rate throughdisruption

Pivot toredefine problem/solution through

empathy

Milk withrevenue

optimization asgrowth slows

• How businesses think about products or companies

• Lean is about moving up and to the right


BCG and policy

Publ

ic s

uppo

rt

Impact on society

Widelypopular

Widelyridiculed

Tiny Huge


BCG and policy

Pork

Publ

ic s

uppo

rt

Impact on society

Widelypopular

Widelyridiculed

Tiny Huge


BCG and policy

Banningleaded gasoline

Pork

Publ

ic s

uppo

rt

Impact on society

Widelypopular

Widelyridiculed

Tiny Huge


BCG and policy

“I Declare today Jebbediah Springfield day”


Pork

Publ

ic s

uppo

rt

Impact on society

Widelypopular

Widelyridiculed

Tiny Huge


BCG and policy

“I Declare today Jebbediah Springfield day”


No more big soda

Pork

Publ

ic s

uppo

rt

Impact on society

Widelypopular

Widelyridiculed

Tiny Huge


The Lean Analytics lifecyclefor an Intrapreneur

Empathy Find problems; don’t test demand. Skip the business case, do analytics

Entitled, aggrieved customers


Beforehand Get buy-in Political fallout





Stickiness Know your real minimum based on expectations, regulations

Hidden “must haves”, feature creep









Virality Build inherent virality in from the start; attention is the new currency

Luddites who don’t understand sharing











Revenue Consider the ecosystem, channels, and established agreements

Channel conflict, resistance, contracts











Revenue Consider the ecosystem, channels, and established agreements

Channel conflict, resistance, contracts

Scale Hand the baton to others gracefully Hating what happens to your baby




Lean Analytics lesson 5:When working from within, the difference between a special operative and a rogue agent is a mandate.


E-commerce enterprise

• Stage: Scale• Model: e-commerce

• EMI was a big music company trying to understand how its customers bought content


David Boyle tackles a small problem

• David Boyle, SVP Insight, EMI Music, ran the insight group

• Had billions of rows of data, but nobody wanted to analyze it

• Instead started a survey project, got a million responses, used this data to sell the idea of “data-driven” business

• Then got support for the broader data initiative.


David Boyle tackles a small problem

• Talked to 1M people in 3y

• At any point, surveying 12 people in the world

• Boiled this down to a few fundamental profiles


Lean Analytics lesson 6:Start with a less important project (preferably one that involves intelligence gathering.) Then use that success for bigger undertakings.


Where your lean lives:It’s learning: Places with lots of dataIt’s rapidly iterated: Apps or softwareIt’s popularizing: Moving up a boxIt’s impact-increasing: Moving right a boxIt has high uncertainty: De-risk with an MVPIt’s boundable: Lean hates molasses


Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844Tuesday, 23 April, 13

http://www.flickr.com/photos/twodolla/3168857844

http://www.flickr.com/photos/twodolla/3168857844

ARCHIMEDES HAD TAKEN

BATHS BEFORE.


Once, a leader convinced others in the absence of data.


Now, a leader knows what questions to ask.


Alistair [email protected]@acroll

Ben [email protected]@byosko


mailto:[email protected]




Thank you!

Further discussion:[email protected]

Contact staff: [email protected]






