FROM THE CLICK AND…
BEY ND
A/B Testing; You’re Most Likely Doing It WrongJess Armstrong & Tomasz Borys 06.11.2015
Copyright 2015 WordStream, Inc. & KISSmetrics All rights reserved.
• Customer Onboarding Specialist
• UMass Amherst- Economics
• Foodie- Traveler- Gymnast
@Jess_Armstrong_
About
Jess
What Makes a Worthwhile Test?
Creating a Fail-Safe A/B Test
3 A/B Test to Consider
A/B Testing in PPC
Why PPC?
• No Cost Set Up
• The Data You need is already there!
• High Commercial Intent
LIVE WEBINAR#BeyondTheClick
Ignore the Trivial Things!
Gains from small changes tend to flatten out over time.
LIVE WEBINAR#BeyondTheClick
Define Success
There are so many
metrics in AdWords, You
need to prioritize ahead
of time!
LIVE WEBINAR#BeyondTheClick
Which One Wins?
0
5
10
15
20
25
30
35
40
45
50
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6
Total conversions by Week
Control Test
0
10
20
30
40
50
60
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6
Average CPA by Week
Control Test
Conflicting Winners!?
LIVE WEBINAR#BeyondTheClick
Set Up a Safety Net
• Remarketing Campaigns
• Catch lost conversions missed
during your test
• Users are 2x more likely to
convert after seeing your ad 7+
times
1. Change Your Call to Action
• This is a Business Decision
• Ads and Landing Pages
• What makes them click and
convert?
• Collaborate with other teams
LIVE WEBINAR#BeyondTheClick
2. Test Mobile Strategies
Some Business see huge increases in
conversions on Mobile!
LIVE WEBINAR#BeyondTheClick
2. Test Mobile Strategies
Things to consider:
• Mobile Bid Modifier
• Mobile Text-Ads
• Call Extensions
• Mobile Friendly Landing Pages
• Set up an experiment in AdWords
• 50/50 impression share
LIVE WEBINAR#BeyondTheClick
3. Change Your Flow
• Think about where on the funnel people are
falling off and change it
LIVE WEBINAR#BeyondTheClick
Key Take-Aways
Think Big! Aim for 2-3x conversion rate
Set Clear Goals and Priorities in advance
Capture lost conversions with remarketing
Tests to try:
o Change Call-to-Action
o Test Mobile Strategies
o Change the Conversion Flow
LIVE WEBINAR#BeyondTheClick
How We A/B Test at KISSmetrics
• Set a Goal
• Baby Steps
• Aim for Statistical Significance
• Never Lose Sight
• Blind Eye for 1 Week
LIVE WEBINAR#BeyondTheClick
Baby Steps
People want to move the
needle and see results fast, by
implementing multiple changes
at once.
But how can you be sure what
element had an impact?
Aim for Statistical Significance SET A GOAL
Aimfor Statistical
Significance
getdatadriven.com
LIVE WEBINAR
Blind Eye
for 1 Week
Data can be very erratic the
first several days, so it’s
easy to hit the panic button
#BeyondTheClick
Never Lose
Sight
We can caught up on
what we think is best
to optimize
conversions or
traffic…but data
doesn’t lie
Ways of Improving Funnel
• Find the Bottleneck
• Learn why there is a Bottleneck
LIVE WEBINAR#BeyondTheClick
Learn Why There is a Bottleneck
• A/B Testing
• Surveys and Reach
• Usability Testing
LIVE WEBINAR#BeyondTheClick
Recommended