Fail and Win: Why a Failed Test Isn’
Caleb Whitmore Founder & CEO Analy<cs Pros
@CalebWhitmore
Hi, I’m @CalebWhitmore
I climbed a mountain… once.
I work @Analy=csPros
And get to work with some awesome clients!
Why Test?
Why Test?
TESTING… • Should focus on improving an outcome • Is the shortest distance between you and improvements • Is done best with rapid itera<on and lots of data
Some things may seem obvious to test…
Bad Form
Good Form
But you should s=ll test them!
Tes=ng depends on data
Op=mizely Data
Google Analy=cs Data
Not all tests are created equally
Some tests will win
Some tests will fail
Is winning a success?
Is failing a loss?
Tes=ng is a Success When:
• A process is followed
• An outcome is measured
• A complete picture is seen
When Things Shake, What Moves?
When Things Shake, What Moves?
Crea=ng a 360 Degree View
• A test will impact more than what you intended
• You must measure the impact across your spectrum of user interac<on
• Measure your test with your exis<ng infrastructure
Crea=ng a 360 Degree View
What happens to… • The “key” conversion point • Engagement on the test page • Length of the en<re visit • Total pageviews/interac<ons in the visit • Secondary conversion goals – view product, view content, start checkout, download a file, etc…
Here’s an example
Example Experiment
ASSUMPTIONS • Auto-‐play repels visitors – Audio without warning
• Thus, bounce rates are high and more people leave • Turn off auto play & more people will stay and engage
Op=mizely Shows a Winner
Bounce Rate Improved
Purchase Rate Tanked!
DO THIS
Setup a 360 Degree View of Tests
• Tip: your Digital Analy<cs plaXorm has this view already!
Op=mizely Automated Integra=ons
Pipes data straight into GA
Is this a good outcome?
What about this?
See the full picture
Capture Experiment Data
• Tie experiment data into your Tag Management and Digital Data frameworks
Leverage Built-‐in Repor=ng
• Create lots of Goals in Op<mizely – pages, clicks, revenue, etc…
QUESTIONS?
@CalebWhitmore of @Analy<csPros
Fail and Win: Why a Failed Test Isn’t a Bad Thing
Caleb Whitmore Founder & CEO Analy<cs Pros
@CalebWhitmore