Upload
andre-morys
View
341
Download
1
Embed Size (px)
DESCRIPTION
Most organization that do a/b testing are just tweaking buttons or testing headlines. In this presentation you will find 5 principles of growth that can change your a/b-testing team into a real growth machine.
Citation preview
© Andre Morys, Web Arts AG www.web-arts.comFRANKFURT - HAMBURG - MÜNCHEN
André Morys
The Million DollarOptimization Strategy
Uh Oh.
© Andre Morys, Web Arts AG www.web-arts.com8
André Morys @morys
© Web Arts AG www.web-arts.com
5.8 bn ! Retail / Lead-Volume
9
0
1,5
3
4,5
6
2008 2010 2012
CRO Volumen (Mrd. !)
© Web Arts AG www.web-arts.com
115 Conversion Experts in 15 Offices worldwide
10
Web Arts is founder of theworldwide alliance of leading
conversion optimizationcompanies
© Andre Morys, Web Arts AG www.web-arts.com
Agenda today
11
1. Don‘t call it „Conversion Rate Optimization“
2. Optimization is not Testing.
3. Don‘t let the numbers lie to you. (Case Study)
© Andre Morys, Web Arts AG www.web-arts.com12
Part I: It‘s about Growth, stupid.
Who has a business plan for CRO?
© Andre Morys, Web Arts AG www.web-arts.com14
Do You Have a Business Plan for Your CRO Strategy?
© Andre Morys, Web Arts AG www.web-arts.com
Net Sales
15
© Andre Morys, Web Arts AG www.web-arts.com
Profit Margin
16
Opportunity Cost: ! 500‘ Profit
© Andre Morys, Web Arts AG www.web-arts.com
Challenge needed?
17
© Andre Morys, Web Arts AG www.web-arts.com
Compare the effect of two optimization strategies:
18
© Andre Morys, Web Arts AG www.web-arts.com
Net Sales
19
© Andre Morys, Web Arts AG www.web-arts.com
Profit Margin
20
Opportunity Cost:
! 2.2m Profit
© Web Arts AG www.web-arts.com
Now you can re-invest ! 2.2m in optimization...
21
© Andre Morys, Web Arts AG www.web-arts.com
More-Growth-Through-Testing-Rule #1
‣It‘s not about conversion rates. It‘s about GROWTH.
‣ Set your goals properly.
‣ Measure your optimization efficiency.
‣ Average Uplift per Test
‣ Amount of Tests per Year
‣ Cumulated Uplift per Year
‣ Test Success Rate
‣ ROI per Test (%)
22
© Andre Morys, Web Arts AG www.web-arts.com23
„If you
double the number
of experiments
you do per year you're going to
double your
inventiveness“
Jeff Bezos, CEO amazon, 2004
© Andre Morys, Web Arts AG www.web-arts.com
Growth of Top 10 e-Commerce Ventures in 5 Years (DE)
24
© Web Arts AG www.web-arts.com
0
300
600
900
1200
2008 2009 2010 2011 2012
The Zalando Effect?
25
© Web Arts AG www.web-arts.com
So, what‘s the problem?
26
© Andre Morys, Web Arts AG www.web-arts.com
For a 25% Compounded Global CR Increase, You Need...
27
2.5 2.5 2.5 2.5 2.5 2.5 2.5 2.5 2.5 2.5 10 Tests à 2.5%?
5 5 5 5 5 5 5 5 5 5 10 Tests à 5%?
5 5 5 5 5
20 Tests à 5%and 50%
success rate?5 5 5 5 5
Problem #1:Organizations are too lame to execute 20 tests per year.
Problem #2:This is really expensive and
complex. Organizations hate both.
Problem #3:Most CMOs still prefer to
buy 25% more traffic.
© Andre Morys, Web Arts AG www.web-arts.com
„We do 200 experiments per year. We test everything. We don‘t need help.“
„Whats your success rate?“
„10%“
Reality Check:
28
© Andre Morys, Web Arts AG www.web-arts.com
Why only 5% average uplift per test?
30
‣In the last case study, they talked about 45% Uplift!-> E-Commerce conversion rate is not email subscription or download CR.
‣My last test had 33% Uplift!-> Was it real? Don‘t confuse statistical significance with validity. Tests should run around 4 weeks for a sufficient sample.
© Andre Morys, Web Arts AG www.web-arts.com31
A/B Test Result
Reality
„I can‘t feel the uplift!“
© Andre Morys, Web Arts AG www.web-arts.com32
Testresults - Significance versus Validity
1 week500 vs. 550 conversions
+10% Upliftsignificant - wohoo!
2 weeks1000 vs. 1030 conversions
+3% Upliftnot significant - :-(
Run your experiments longer to get valid insights!
© Andre Morys, Web Arts AG www.web-arts.com
KPI: Average E-Commerce CR Uplift per Template Type
33
0 %
2,5 %
5 %
7,5 %
10 %
Homepage Category Product Page Cart Checkout Site Wide
9,81 %
7,3 %6,59 %
9,28 %
6,35 %
3,32 %
Average Uplift by Template Type (n= 134)
© Andre Morys, Web Arts AG www.web-arts.com
More-Growth-Through-Testing-Rule #2
‣It‘s all about effectiveness. ‣ To increase real CR, you need many really good test results.
‣ Average Uplift per Test influences your growth directly.
‣ So: Do the right things!
‣ Don‘t do it too fast.
‣ Don‘t test everything. It is expensive.
34
© Andre Morys, Web Arts AG www.web-arts.com35
Part II: Where do good optimization ideas
come from?
© Andre Morys, Web Arts AG www.web-arts.com37
© Andre Morys, Web Arts AG www.web-arts.com39
where would you buy?
why?
© Andre Morys, Web Arts AG www.web-arts.com40
© Andre Morys, Web Arts AG www.web-arts.com41
quantitative qualitative
How much? Why?
© Andre Morys, Web Arts AG www.web-arts.com42
Traffic
Website
Web Analytics
© Andre Morys, Web Arts AG www.web-arts.com43
Traffic
Website
Web Analytics
Click
Hand
Motor Control
prefrontal Cortex
Limbic System
Perception
© Andre Morys, Web Arts AG www.web-arts.com44
Traffic
Website
Web Analytics
Click
Hand
Motor Control
prefrontal Cortex
Limbic System
Perception
99%
1%
© Andre Morys, Web Arts AG www.web-arts.com
Test: facebook Acount Deaktivierung
46
fb cancellation seite beispiel
© Andre Morys, Web Arts AG www.web-arts.com
Test: facebook Acount Deaktivierung
47
ROI
„I have to buy it.“
Brand, Emotion, Value
„I want to buy it.“
Psychology
„I can buy it.“
Usability, Accessability, Visibility
Web Arts ROI Pyramid
© Web Arts AG www.web-arts.com49
Factors, that influence the chance of conversion, can be
identified with a conversion framework
7E Framework
© Andre Morys, Web Arts AG www.web-arts.comFRANKFURT - HAMBURG - MÜNCHEN
© Web Arts AG www.web-arts.com62
You are too dumb to buy here.
© Web Arts AG www.web-arts.com71
Conversion Optimization means:„Get a relevant unique value
proposition in the users mind.“good feeling
© Andre Morys, Web Arts AG www.web-arts.com
More-Growth-Through-Testing-Rule #3
72
‣It is not hard to implement a testing tool. It‘s hard to find the right thing to optimize.
‣ Find out, what motivates users.
‣ Analyse your website from different perspectives:
‣ Web Analytics: Where is the problem?
‣ User Testing: How do useres think about it?
‣ Persona: What do they not tell us?
‣ Expert Evaluation: Which psychological principles could we use?
© Andre Morys, Web Arts AG www.web-arts.com
Conversion DMAIC® Lean Innovation Process by Web Arts AG www.web-arts.com
73
quantitative
qualtitative
LEARN
BACKLOG
CONTROL
IMPROVE ANALYSE
DEFINE
MEASURE
© Andre Morys, Web Arts AG www.web-arts.com90
André Morys
CEO
Web Arts AG
Tel. +49.6172.68097-15
www.web-arts.com
Thank You!