Lean startup metrics

4th Feb 2012

@STUECCLES

LEAN STARTUP METRICS

I NEVER FELT LIKE I WAS LEARNING

ANYTHING IMPORTANT

“An actionable metric is one that ties specific and

repeatable actions to observed results.”

ASH MAURYA http://www.ashmaurya.com/

EVENT BASED TRACKING

FUNNELS

SIMPLE SIGN-UP FUNNEL

EXTENDED SIGN-UP FUNNEL

A/BTESTING

+152%

A/B TESTING USER MAP ON HOMEPAGE

GOOGLE BING

“THAT BLUE WAS WORTH $80M AT LEAST”

PAUL RAY - PRODUCT MANAGER BING

LOCALMAXIMA

AN INTUITIVE LEAP

PROPS @BOKARDO FOR LAST FEW SLIDES

@37SIGNALS

WHAT TO A/B TESTShort Answer: Everything

Longer Answer: That you have enough traffic to see significant results that are not trival optimisations (HINT you are not Bing/Google)

WHAT TO A/B TESTEvery new feature should go into A/B testing.

If has negative or no real effect, pull it.

Optimise existing features/copy/design (80% existing 20% new)

FRAMEWORK

ACQUISITION

ACTIVATION

RETENTION

REFERRAL

REVENUE

Dave McClure http://500hats.typepad.com/

DON’T TRY TO MEASURE TOO MUCH

EXAMPLES

SAY HELLO TO OUR 3 FICTIONAL STARTUPS

Social Network for Dog Lovers.

Makes money through advertising and affiliate sales of lush dog collars.

DOGGIECOLR

Key business drivers are:

Advertising and affiliates need large traffic/page views so growth and retention are key

DOGGIECOLR

SIGN-UP

COMMENT

VISIT

INVITE FRIENDS

POST PICTURES

BUY COLLAR

CREATE PROFILE

SHARE PICTURES

ACQUISITION

ACQUISITION

ACTIVATION

ACTIVATION

REFERRAL

REVENUE

Project management tool for Agile startups.

Makes money through monthly subscriptions. Has a free trial period.

AGILESAMBA

Key business drivers are:

Conversion to paid usage and cancelation rates

AGILESAMBA

SIGN-UP

VISIT

CREATES PROJECT

UPGRADES PAID

ACQUISITION

ACQUISITION

ACTIVATION

REVENUE

CANCELS ACCOUNT

REVENUE

Marketplace to rent/buy drills from your neighbour (AirBnB for drills)

Takes a small percentage of the transaction between two parties

RENTMYDRILL

RENTMYDRILL

Key business drivers are:

Two sided business model means acquisition of both lenders and borrowers and transactions per user

LENDER SIGN-UP

VISIT

SEARCHES BORROWERS

BORROW DRILL

ACQUISITION

ACQUISITION

ACTIVATION

REVENUE

BORROWER SIGN-UP

SEARCHES LENDERS

ONE KEY METRICThere is generally only one key metric you need to concentrate on at a time. Such as referral or sign-up conversion or cancellation rate.

ACQUISITION

TRACKING MARKETING CHANNELSGoogle AdwordsBannersSocial MediaPartnershipsPRBloggingAnything that links to your site!

TRACKING MARKETING CHANNELSUse unique urls (tracking parameters) on every url you create/give-out/pay for

Use bit.ly

Use HTTP referrers to group PR

MEASURING COST OF VISITSCost per click (site visit)

Cost of campaign / traffic generated

(Social media) Cost of effort / traffic generated

MEASURING COST OF ACQUISITIONCost of one visit / conversion rate = Cost of an acquisition

Acquisition could be paid for acquisition or free sign-up

Unfortunately not that simple

DIFFERENT CHANNELS CAN HAVE DIFFERENT CONVERSION RATES

MARKETING CHANNELSPer channel track:

Scale CostConversion

Optimise and scale channels with best Cost of Acquisition (CoA) and potential to Scale

ACQUISITIONCost of Acquisition < Customer Lifetime Profit

Then scale acquisition at a similar or lower rate

A/B test acquisition channels to optimise

ACQUISITION INCLUDING REFERRALIf it costs £3 to acquire a customer through advertising

But that customer brought in 2 other customers through referral

The COA of that channel is £1

ACTIVATION

SURE THEY SIGNED-UP BUT DID THEY LIKE IT?

ACTIVATION AT FACEBOOK(i) which data points predict whether a user will stay?

(ii) if they stay, which data points predict how active they’ll be after three months?

ACTIVATION AT FACEBOOK(i) having more than one session as a new user, and entering basic profile information.

(ii) how often a user was reached out to by others, frequency of third party application use, and how forthcoming a user was on the site.

ACTIVATION METRICS OFTEN DRIVE OTHER METRICS IN REFERRAL,

RETENTION, REVENUE

RETENTION

RETURNING USERSTracking unregistered users that return often (useful for content sites with advertising)

Tracking registered users that return and take some sort of action.

COHORT METRICS - CREATED PROJECT

COHORT METRICS - CAME BACK

REFERRAL

REFFERAL MECHANISMSSocial Media Sharing

Invite a friend type mechanics

App Reviews

Word of mouth

VIRAL COEFFICIENTThe average number of customers each customer refers

VIRAL COEFFICIENT - EXAMPLEEach Doggiecollr customer invites 5 other customers by email

20 % click on that email link

50 % of those users convert into customers

VIRAL COEFFICIENT - EXAMPLEViral coefficient =

Refferals x click through x conversion

5 x 0.2 x 0.5 = 0.5

VIRAL COEFFICIENTA viral co-effiecient greater than 1 means that every customer gets more than one other customer on average

Therefore your product will grow virally

VIRAL COEFFICIENTMeasure automatically for all your in built referral mechanisms.

Give customers unique invite URLs and track conversions

VIRAL CYCLE TIMEThe average time taken for a referral to turn into a customer

With the viral coeffiecent and viral cycle time you can determine your viral growth

NET PROMOTER SCORE

Number of people who would recommend your product to a friend.

A single question survey

NET PROMOTER SCOREHow likely is it you would recommend AgileSamba to a friend or colleague?

1 2 3 4 5 6 7 8 9 100

NPS = % Promoters - % Detractors

NET PROMOTER SCORE

Survey a small percentage of your customers by email weekly

Record who you sent to so you don’t repeat the same people every week

NET PROMOTER SCORE

NPS is not a direct measurement that translates to key metrics

Can act as a leading indicator of other metrics

REVENUE

MAKING MONEY

Key for any business is to find out how much profit they make for every customer and scaling the number of customers

LIFETIME CUSTOMER VALUE

How much money you make for every customer you acquire

For some this is a single transaction

For subscription businesses it depends on retention

CANCELLATION RATE

Is the percentage number of customers who cancel in any given month compared to total (paying) customers

[Cancellation rate] = [product utility] + [service quality] + [acceptable price]

LIFETIME CUSTOMER VALUE (LTV)

LTV = monthly revenue x no months in lifetime

No months = 1 / cancellation rate

LTV = monthly revenue / cancellation rate

LIFETIME CUSTOMER VALUE (LTV)

For example:

If £20 a month and cancelation rate is 10% a month

LTV = £200

LIFETIME CUSTOMER VALUE (LTV)

Often not that simple. Short term cancellation rates are much higher than long term

Use Cohort Analysis to determine

LIFETIME CUSTOMER VALUE (LTV)

r = short-term cancellation rate (e.g. 0.15)p = long-term cancellation rate (e.g. 0.03)s = number of months in the “short-term” age group (e.g. 3)

(1-r)^s × (s + 1/p) = expected months

TIPS

TIPSConcentrate on macro metrics

Everything you do should be an attempt to change a metric

Measure if it did

Try to use as few tools as possible

TIPSAutomate it and put it in a dashboard

If it can’t be automated make a simple spreadsheet and an easy way to update it

TIPSDon’t forget to measure marketing

Make metrics a habit

THANKS