Embed Size (px)
Citation preview
Measure
ImproveRepeat
How to use data to continuously improve what you sell
Edward Upton, Founder of LittleData.co.uk@LittleDataUK
Why measure?
• Every business wants to improve their sales and profitability
• How do data-driven companies do this, and do it at pace?
• How can you use similar techniques in your company?
Douglas Bowman, ex-head of Visual Design, Google (2009)
“A team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see which one performs better... I can’t operate in an environment like that.”
Data makes big profits for Google
“We saw which shades of blue people liked the most, demonstrated by how much they clicked on them. Given
the scale of our business, was that we made an extra
$200m a year in ad revenue.”
Dan Cobley, Google UK (2014)
A/B testing
A/B testing
A B
Decide what to measure
Ben Yoskovitz, author of Lean Analytics
“The One Metric That Matters is about finding the right thing
to track at the right time, based on the type of business
you're in”
Which metric for you?
Allows everyone to focusSimplifies the tracking setup
Reduces the chance of silly calculation errorsPrevents arguments about whether an AB test really worked
Picking just ONE METRIC
Dave Brailsford, Head of British Cycling
“Excellenceis the aggregation of marginal gains”
Repeat, repeat
• Question: Would you prefer to increase your sales by 3% a week or triple (3x) them next year?
Hint: 52 weeks in a year x 3% = 156% increase = 2.5x
3% a week = 4.6x over the year
The miracle of compounding means tiny impacts made every week stack up to huge achievement
Quicker is better
• Looking for small gains? You’ll need large numbers to work with
• Before you start a test, plan how much data will be needed to prove the point
• E.g– Assume 2% of web visits into customer enquiries.– You want to obverse an absolute difference of +/- 0.2%– At 1000 visits that would mean a baseline of 20 enquiries
and 22 for the improved. That extra 2 conversions could just be fluke
– 5000 visits would be the minimum for a sample – 100 enquiries baseline and 110 for improved
Small samples = dodgy statistics
What to use to measure
How to run A/B tests
• So you’ve set up your Google Analytics, and are running continuous AB tests
• How do you track progress?
a) Take a glance at the chart when you have time
b) Copy metrics into a weekly spreadsheet for comparison
c) Set up an automated export into Google Docs
d) Use a clever algorithm to spot the trends
How do you keep track?
• Our algorithms look for changes so you don’t wade through hundreds of reports
• Soon to launch a tool to track AB tests more easily
• Need a Google Analytics account with more than 1,000 visits a month
LittleData saves you time
Try LittleData
http://littledata.uk
