Lean Analytics and Local Government - Alistair Croll - Code for America

LEAN ANALYTICS & LOCAL GOVALISTAIR CROLL

April 22, 2013

Tuesday, 23 April, 13

Alistair CrollCo-Author, Lean Analytics

www.leananalyticsbook.com@leananalytics@byosko | @acroll

Lean AnalyticsUse data to build a

better business faster.

http://www.flickr.com/photos/itsgreg/446061432/

Analytics is the measurement of movement towards your business goals.

Most startups don’t know what they’ll be when they grow up.

Paypalfirst built for Palmpilots

Freshbookswas invoicing

for a web design firm

Wikipediawas to be written by

experts only

Mitelwas a

lawnmower company

Hotmailwas a

database company

experts only

Mitelwas a

lawnmower company

Hotmailwas a

database company

Flickrwas going to be an MMO

experts only

Mitelwas a

lawnmower company

Hotmailwas a

database company

Twitterwas a

podcasting company

experts only

Mitelwas a

lawnmower company

Hotmailwas a

database company

Twitterwas a

podcasting company

Autodeskmade desktop

automation

experts only

Mitelwas a

lawnmower company

Kevin Costner is a lousy entrepreneur.

Don’t sell what you can make.Make what you can sell.

The basic Lean message islearn and adapt, fast.

“What information consumes is rather obvious: it consumes the attention of its recipients.Hence a wealth of information creates a poverty of attention, and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.”

(Computers, Communications and the Public Interest, pages 40-41, Martin Greenberger, ed., The Johns Hopkins Press, 1971.)

http://www.flickr.com/photos/art_es_anna/288880795/Tuesday, 23 April, 13

Lean Analytics lesson 1:Most government projects have an attention or a connectivity problem. This is where you will spend most of your time innovating.

Empathy stage:Localmind hacks Twitter

• Stage: Empathy• Model: UGC/mobile

• Real-time question and answer platform tied to locations.• Needed to find out if a core behavior—answering questions about a place—

happened enough to make the business real

Localmind hacks Twitter

• Before writing a line of code, Localmind was concerned that people would never answer questions.• This was their biggest risk: if questions went unanswered users would have a

terrible experience and stop using Localmind.• Ran an experiment on Twitter

• Tracked geolocated tweets in Times Square• Sent @ messages to people who had just tweeted, asking questions about

the area: how busy is it; is the subway running on time; is something open; etc.

• The response rate to their tweeted questions was very high.• Good enough proxy to de-risk the solution, and convince the team and

investors that it was worth building Localmind.

Hits A metric from the early, foolish days of the Web. Count people instead.

Page views Marginally better than hits. Unless you’re displaying ad inventory, count people.

Visits Is this one person visiting a hundred times, or are a hundred people visiting once? Fail.

Unique visitors This tells you nothing about what they did, why they stuck around, or if they left.

Followers/friends/likes

Count actions instead. Find out how many followers will do your bidding.

Time on site, or pages/visit

Poor version of engagement. Lots of time spent on support pages is actually a bad sign.

Emails collected How many recipients will act on what’s in them?

Number of downloads

Outside app stores, downloads alone don’t lead to lifetime value. Measure activations/active accounts.

2-sided market model:AirBnB and photography

• Stage: Revenue• Model: 2-sided marketplace

• Rental-by-owner marketplace that allows property owners to list and market their houses. Offers a variety of related services as well.

AirBnB tests a hypothesis

• The hypothesis: “Hosts with professional photography will get more business. And hosts will sign up for professional photography as a service.”

• Built a concierge MVP

• Found that professionally photographed listings got 2-3x more bookings than the market average.

• In mid-to-late 2011, AirBnB had 20 photographers in the field taking pictures for hosts.

2008 2009 2010 2011 2012

2 million

4 million

6 million

8 million

10 million

NIGHTS BOOKED

20 photographers

Friday, November 9, 12Tuesday, 23 April, 13

http://www.flickr.com/photos/bootbearwdc/1243690099/

Pick the right experiments

If it won’t changehow you behave,it’s a

badmetric.http

flickr.

The five Stages of Lean Analytics

The five Stages of Lean AnalyticsTh

you’

Empathy

u’re

Empathy

Stickiness

u’re

Empathy

Stickiness

Virality

u’re

Empathy

Stickiness

Virality

Revenue

u’re

Empathy

Stickiness

Virality

Revenue

u’re

Empathy

Stickiness

Virality

Revenue

u’re

E-commerce SaaS MediaMobile

appUser-gencontent

2-sidedmarket

The business you’re in

Empathy

Stickiness

Virality

Revenue

u’re

E-commerce SaaS MediaMobile

appUser-gencontent

2-sidedmarket

The business you’re in

One MetricThat Matters.

Lean Analytics lesson 2:Choose one metric around which to rally support, and reject vanity metrics ruthlessly.

Choose only one metric.

http://www.flickr.com/photos/connortarter/4791605202/

Metrics are likesqueeze toys.

Metrics in practice:The Lean Analytics Cycle

Pick OMTM

Draw a linein the sand

Pick OMTM

Find a potential

improvement

Pick OMTM

Without data: make a good guess

Find a potential

improvement

Pick OMTM

With data:find a

commonality

Find a potential

improvement

Pick OMTM

Hypothesis

With data:find a

commonality

Find a potential

improvement

Pick OMTM

Make changes in production

Hypothesis

With data:find a

commonality

Find a potential

improvement

Pick OMTM

Design a test

Hypothesis

With data:find a

commonality

Find a potential

improvement

Pick OMTM

Measure the results

Design a test

Hypothesis

With data:find a

commonality

Find a potential

improvement

Pick OMTM

Did we move the needle?

Measure the results

Design a test

Hypothesis

With data:find a

commonality

Find a potential

improvement

Pick OMTM

Success!

Measure the results

Design a test

Hypothesis

With data:find a

commonality

Find a potential

improvement

Pick OMTM

Pivot orgive up

Success!

Measure the results

Design a test

Hypothesis

With data:find a

commonality

Find a potential

improvement

Pick OMTM

Draw a new linePivot orgive up

Success!

Measure the results

Design a test

Hypothesis

With data:find a

commonality

Find a potential

improvement

Pick OMTM

Draw a new linePivot orgive up

Try again

Success!

Measure the results

Design a test

Hypothesis

With data:find a

commonality

Find a potential

improvement

Pick OMTM

Lean Analytics lesson 3:There’s no “finished.” Just more iterations.

The B2B stereotype

Domainexpert

Disruptionexpert Operations

• Domain expert knows industry and the problem domain. Has a Rolodex; proxy for customers.

• Disruption expert knows tech that will produce a change Sees beyond the current model.

The B2B stereotype

Domainexpert

Disruptionexpert Operations

• Domain expert knows industry and the problem domain. Has a Rolodex; proxy for customers.

• Disruption expert knows tech that will produce a change Sees beyond the current model.

Three typical approaches

Enterprise pivot

Copy and rebuild

Disrupt a problem

Create a popular consumer product then pivot to tackle the enterprise

Dropbox

Take an existing consumer or open source idea and make it enterprise-ready

Yammer, MapR

Convince the enterprise to discard the old way because of overwhelming advantages.

Taleo, Google Apps

Lean Analytics lifecyclefor an enterprise-focused startup

Empathy Consulting to test ideas and bootstrap the business

Lock-in, IP control, overfitting

Stage Do this Fear this

Stickiness Standardization and integration; shift from custom to generic

Ability to integrate; support

Virality Word of mouth, references, case studies

Bad vibes; exclusivity

Revenue Growing direct sales, professional services, support

Pipeline, revenue recognition, comp

Revenue Growing direct sales, professional services, support

Pipeline, revenue recognition, comp

Scale Channels, analysts, ecosystems, APIs, vertically targeted products

Crossing the chasm; Gorillas

The Zero Overhead principle

A central theme to this new wave of innovation is the application of core product tenets from the consumer space to the enterprise.In particular, a universal lesson that I keep sharing with all entrepreneurs building for the enterprise is the Zero Overhead Principle: no feature may add training costs to the user. DJ Patil

Lean Analytics lesson 4:Government can learn from enterprise-focused startups: Disrupt a known problem with new technology.

Skunk Works for intrapreneurs

• The Lockheed Martin Skunk Works

Span of control and the railroads

• Daniel C. McCallum

The BCG matrix

Question marks!(low market share, high growth rate)

May be the next big thing. Consumes investment, but

will require money to increase market share.

Stars!(high growth rate, high market share)

What everyone wants. As market invariably stops

growing, should become cash cows.

Dogs!(low market share, low growth rate)

Barely breaks even, may be a distraction from better

opportunities. Sell off or shut down.

Cash cows!(high market share,

low growth rate) Boring sources of cash, to be milked but not worth additional investment.

Market share

Pivot to increasemarketshare

throughvirality,

attention

Pivot toincrease growth

rate throughdisruption

Pivot toredefine problem/solution through

empathy

Milk withrevenue

optimization asgrowth slows

• How businesses think about products or companies

• Lean is about moving up and to the right

BCG and policy

Impact on society

Widelypopular

Widelyridiculed

Tiny Huge

BCG and policy

Impact on society

Widelypopular

Widelyridiculed

Tiny Huge

BCG and policy

Banningleaded gasoline

Impact on society

Widelypopular

Widelyridiculed

Tiny Huge

BCG and policy

“I Declare today Jebbediah Springfield day”

Impact on society

Widelypopular

Widelyridiculed

Tiny Huge

BCG and policy

“I Declare today Jebbediah Springfield day”

No more big soda

Impact on society

Widelypopular

Widelyridiculed

Tiny Huge

The Lean Analytics lifecyclefor an Intrapreneur

Empathy Find problems; don’t test demand. Skip the business case, do analytics

Entitled, aggrieved customers

Beforehand Get buy-in Political fallout

Stickiness Know your real minimum based on expectations, regulations

Hidden “must haves”, feature creep

Virality Build inherent virality in from the start; attention is the new currency

Luddites who don’t understand sharing

Revenue Consider the ecosystem, channels, and established agreements

Channel conflict, resistance, contracts

Revenue Consider the ecosystem, channels, and established agreements

Channel conflict, resistance, contracts

Scale Hand the baton to others gracefully Hating what happens to your baby

Lean Analytics lesson 5:When working from within, the difference between a special operative and a rogue agent is a mandate.

E-commerce enterprise

• Stage: Scale• Model: e-commerce

• EMI was a big music company trying to understand how its customers bought content

David Boyle tackles a small problem

• David Boyle, SVP Insight, EMI Music, ran the insight group

• Had billions of rows of data, but nobody wanted to analyze it

• Instead started a survey project, got a million responses, used this data to sell the idea of “data-driven” business

• Then got support for the broader data initiative.

David Boyle tackles a small problem

• Talked to 1M people in 3y

• At any point, surveying 12 people in the world

• Boiled this down to a few fundamental profiles

Lean Analytics lesson 6:Start with a less important project (preferably one that involves intelligence gathering.) Then use that success for bigger undertakings.

Where your lean lives:It’s learning: Places with lots of dataIt’s rapidly iterated: Apps or softwareIt’s popularizing: Moving up a boxIt’s impact-increasing: Moving right a boxIt has high uncertainty: De-risk with an MVPIt’s boundable: Lean hates molasses

Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844Tuesday, 23 April, 13

ARCHIMEDES HAD TAKEN

BATHS BEFORE.

Once, a leader convinced others in the absence of data.

Now, a leader knows what questions to ask.

Alistair Crollacroll@gmail.com@acroll

Ben Yoskovitzbyosko@gmail.com@byosko

Thank you!

Further discussion:muni-innovation@googlegroups.com

Contact staff: pn-staff@codeforamerica.org

Lean Analytics and Local Government - Alistair Croll - Code for America

Business

Alistair Maclean - Zsilip

Alistair Deck_2

Alistair Johnston – CJC London Campbell Johnston Clark Paul … · 2017. 9. 30. · Alistair Johnston . alistair@cjclaw.com Paul Martin paulm@cjclaw.com Campbell Johnston Clark

Dan Croll - Planning

Software Assurance for Executives Bibliography · [Ardis 1989] Ardis, Mark A. & Ford ... Building Security In Maturity Model. BSIMM4 Download. ... pdf. [Croll 2012b] Croll, Paul

My research and My favourite student project Alistair Edwards alistair

Chenée and Alistair

Croll DG 90 - Saint Andrews - Homeheritage.saintandrews.org.au/userfiles/files/Croll DG 90.pdf · Title: Croll DG 90 Author: Warrick Created Date: 3/19/2016 2:27:21 AM

#FIRMday Birmingham 7th May 2015 Kiera Croll, City & Guilds

Lean Analytics for Intrapreneurs by Allistair Croll

Croll lean analytics workshop (3h) - lean ux nyc april 2014

Dementia Alistair Burns

Alistair cockburn

Alistair Crowley

Alistair Cockburn SCRUM

WRITING - Alistair Cockburn

Jennifer Croll - Annick Press

Film research alistair

10 Insights on Lean Analytics — Alistair Croll

Pomona - Alistair McDowall