Exploring the full range of a:b testing opportunity - SAScon 2015

@matt_lacey12 #SASCon

EXPLORING THE FULL RANGE OF A/B TESTING OPPORTUNITY

Matt LaceyHead of Optimisation

@matt_lacey12 #SASCon

Team of leading conversion optimisation specialists

Head office in Manchester, with clients across UK and Europe

Work with brands to analyse, research, test and optimise site experiences

ABOUT PRWD

@matt_lacey12 #SASCon

“We’ll make the big changes and you can just do the optimisation.”

“Conversion Optimisation & Continuous Improvement are

part of our DNA.”

@matt_lacey12 #SASCon

CONVERSION OPTIMISATION IS NOT JUST TESTING . . .

. . . BUT IT IS AN ESSENTIAL ELEMENT.

@matt_lacey12 #SASCon

Four Options:

1. Control – (4 months FREE)

2. Variation A – (SAVE £22)

3. Variation B – (SAVE 38%)

4. Variation C – remove ‘(cancel anytime)’ from monthly

WHICH TEST WON?

12.5% overall increase in

subscriptions for Variation A (SAVE £22)

@matt_lacey12 #SASCon

1.  Control - Annual £39.99 (Save £44)

2. Variation – £3.33 per month (£39.99 annually)

WHICH TEST WON?

77% overall increase in annual

subscriptions for the Variation

@matt_lacey12 #SASCon

“We had gotten really good at optimising, tuning and iterating, but in the process we had lost sight of the purpose of those activities: testing a clear business hypothesis in the service of the company’s vision.” Eric Ries, Author the Lean Startup

@matt_lacey12 #SASCon

GET REVISING SIGN-UP TABLE

“ Price doesn’t put me off, although the basic package is more than enough for me.

I think the site has already got everything I need.

Moderated user research participant

@matt_lacey12 #SASCon

@matt_lacey12 #SASCon

GET REVISING SIGN-UP TABLE

185% increase in premium subscriptions

with a small drop in free subscriptions. This was seen to be a good trade-off.

@matt_lacey12 #SASCon

UI TWEAKING

BUSINESS HYPOTHESIS

TESTING

OPPORTUNITY

@matt_lacey12 #SASCon

THANKS FOR LISTENING

MATT LACEY Head of Optimisation

@matt_lacey12