12
@matt_lacey12 #SASCon EXPLORING THE FULL RANGE OF A/B TESTING OPPORTUNITY Matt Lacey Head of Optimisation

Exploring the full range of a:b testing opportunity - SAScon 2015

  • Upload
    prwd

  • View
    79

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Exploring the full range of a:b testing opportunity - SAScon 2015

@matt_lacey12 #SASCon

EXPLORING THE FULL RANGE OF A/B TESTING OPPORTUNITY

Matt LaceyHead of Optimisation

Page 2: Exploring the full range of a:b testing opportunity - SAScon 2015

@matt_lacey12 #SASCon

Team of leading conversion optimisation specialists

Head office in Manchester, with clients across UK and Europe

Work with brands to analyse, research, test and optimise site experiences

ABOUT PRWD

Page 3: Exploring the full range of a:b testing opportunity - SAScon 2015

@matt_lacey12 #SASCon

“We’ll make the big changes and you can just do the optimisation.”

“Conversion Optimisation & Continuous Improvement are

part of our DNA.”

Page 4: Exploring the full range of a:b testing opportunity - SAScon 2015

@matt_lacey12 #SASCon

CONVERSION OPTIMISATION IS NOT JUST TESTING . . .

. . . BUT IT IS AN ESSENTIAL ELEMENT.

Page 5: Exploring the full range of a:b testing opportunity - SAScon 2015

@matt_lacey12 #SASCon

Four Options:

1. Control – (4 months FREE)

2. Variation A – (SAVE £22)

3. Variation B – (SAVE 38%)

4. Variation C – remove ‘(cancel anytime)’ from monthly

WHICH TEST WON?

12.5% overall increase in

subscriptions for Variation A (SAVE £22)

Page 6: Exploring the full range of a:b testing opportunity - SAScon 2015

@matt_lacey12 #SASCon

1.  Control - Annual £39.99 (Save £44)

2. Variation – £3.33 per month (£39.99 annually)

WHICH TEST WON?

77% overall increase in annual

subscriptions for the Variation

Page 7: Exploring the full range of a:b testing opportunity - SAScon 2015

@matt_lacey12 #SASCon

“We had gotten really good at optimising, tuning and iterating, but in the process we had lost sight of the purpose of those activities: testing a clear business hypothesis in the service of the company’s vision.” Eric Ries, Author the Lean Startup

Page 8: Exploring the full range of a:b testing opportunity - SAScon 2015

@matt_lacey12 #SASCon

GET REVISING SIGN-UP TABLE

“ Price doesn’t put me off, although the basic package is more than enough for me.

I think the site has already got everything I need.

Moderated user research participant

Page 9: Exploring the full range of a:b testing opportunity - SAScon 2015

@matt_lacey12 #SASCon

Page 10: Exploring the full range of a:b testing opportunity - SAScon 2015

@matt_lacey12 #SASCon

GET REVISING SIGN-UP TABLE

185% increase in premium subscriptions

with a small drop in free subscriptions. This was seen to be a good trade-off.

Page 11: Exploring the full range of a:b testing opportunity - SAScon 2015

@matt_lacey12 #SASCon

UI TWEAKING

BUSINESS HYPOTHESIS

TESTING

OPPORTUNITY

Page 12: Exploring the full range of a:b testing opportunity - SAScon 2015

@matt_lacey12 #SASCon

THANKS FOR LISTENING

MATT LACEY Head of Optimisation

@matt_lacey12