Embed Size (px)
Citation preview
Habits of advanced conversion optimisers
Stephen PavlovichCEO @ Conversion.com
“To test it out, we tied together this fake trailer where we combined 25 clips from all these movies that inspired us.”
Matt Duffer
How most people see testing
Testing gives us a framework to take risks and be more inventive.
@conversion_com
The changing nature of testing
“Almost any questions can be answered, cheaply, quickly and finally, by a test campaign.
And that's the way to answer them – not by arguments around a table.”
“Almost any questions can be answered, cheaply, quickly and finally, by a test campaign.
And that's the way to answer them – not by arguments around a table.”
Claude HopkinsScientific Advertising (1923)
Tests are important and help us understand our customers.
Good selling is based on good testing.
Claude HopkinsScientific Advertising (1923)
“
WHAT WOULD YOU DO IFLIFE HAD AN UNDO BUTTON?
Testing can be an undo button for our most important decisions.
@conversion_com
Habits of testing
The hypothesis
1. Find out why people aren’t converting
2. Fix it.
1. Find out why people aren’t converting
2. Fix it.
Use this formula to write your test hypotheses instead...
We know that quant and qual data.
We believe that lever by test concept on area for audience will result in goal.
We’ll know by observing KPI for duration.
@conversion_com
We know that quant and qual data.
We believe that lever by test concept on area for audience will result in goal.
We’ll know by observing KPI for duration.
@conversion_com
We know that quant and qual data.
We believe that lever by test concept on area for audience will result in goal.
We’ll know by observing KPI for duration.
@conversion_com
We know that quant and qual data.
asd
We know that quant and qual data.
asdSales data, web analytics, form
analysis, session replay, heatmapping, onsite surveys, offsite surveys, focus
groups, usability tests...
We know that quant and qual data.
We know that 98% of visitors to the credit card landing page don’t apply. In usability tests, all users struggled to locate the call-to-action.
asd
We believe that lever by test concept...
asd
We believe that lever by test concept...
asdCore “theme” of the test
We believe that lever by test concept...
asdCore “theme” of the test The implementation
of that lever
We believe that lever by test concept...
We believe that emphasising the call-to-action by adding a button to the main content panel...
asd
...on area for audience will result in goal.
...on area for audience will result in goal.
The high-opportunity page or element we’re testing
...on area for audience will result in goal.
The high-opportunity page or element we’re testing
The high-opportunity audience segment we’re targeting
...on area for audience will result in goal.
...on the credit card landing page for desktop visitors will result in an increase in credit card customers.
We’ll know by observing KPI for duration.
We’ll know by observing KPI for duration.
How we’ll measure the goal
We’ll know by observing KPI for duration.
How we’ll measure the goal The expected test duration
We’ll know by observing KPI for duration.
We’ll know by observing successful 1-2-3 credit card applications for 2 weeks.
Test hypothesis
We know that 98% of visitors to the credit card landing page don’t apply. In usability tests, all users struggled to locate the call-to-action.
We believe that emphasising the call-to-action by adding a button to the main content panel on the credit card landing page for desktop visitors will result in an increase in credit card customers.
We’ll know by observing successful 1-2-3 credit card applications for 2 weeks.
We know that quant and qual data.
We believe that lever by test concept on area for audience will result in goal.
We’ll know by observing KPI for duration.
@conversion_com
We know that quant and qual data.
We believe that lever by test concept on area for audience will result in goal.
We’ll know by observing KPI for duration.
@conversion_com
Apply to anything
Habits of testing
Evolutionary testing
Lever
Lever 1
Lever 3
Lever 2
Increase test complexity and apply to new areas
Iterate on minimum
viable test
Minimum viable
test
Lever
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
TestLever 1
Lever 3
Lever 2
Increase test complexity and apply to new areas
Iterate on minimum
viable test
Minimum viable
test
Lever
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
TestLever 1
Lever 3
Lever 2
Increase test complexity and apply to new areas
Iterate on minimum
viable test
Minimum viable
test
Lever
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
TestLever 1
Lever 3
Lever 2
Increase test complexity and apply to new areas
Iterate on minimum
viable test
Minimum viable
test
Lever
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
TestLever 1
Lever 3
Lever 2
Start with theminimum viable test
then scale in reach and impact. @conversion_com
Habits of testing
Every change is a test
1810–1984
1998–2016
1995–2016
“If you double the number of experiments you do per year
you’re going to double your inventiveness.”
Jeff Bezos
Customer journey
TV
SEO
Display advertising
Social media
Outdoor advertising
PPC
Customer journey
Website
App
In-store
TV
SEO
Call centreDisplay advertising
Social media
Outdoor advertising
PPC
Customer journey
Website
App
In-store
TV
SEO
Call centreDisplay advertising
Social media
Product
Pricing
Outdoor advertising
PPC
Customer journey
Website
App
In-store
TV
SEO
Call centreDisplay advertising
Social media
Product
Pricing
Lifetime value
Referrals
Outdoor advertising
PPC
Customer journey
Website
App
In-store
TV
SEO
Call centreDisplay advertising
Social media
Product
Pricing
Lifetime value
Referrals
Outdoor advertising
PPC
Why are we only testing here?
Testing allows us to optimise advertising, content, design, functionality, price, product...
at scale.@conversion_com
Before After
Sales
+100%
In a fast-changing world, testing is our unfair advantage.
@conversion_com
