Maximizing Campaign ROICase Study: Doubling Toolbar Install Conversion Rate with Multivariate Optimization
Sam Ee, MIVARobert Bergquist, Widemile
April 25, 2008
Today’s FacilitatorsSam Ee, MIVA Direct‣ MIVA Direct delivers on it's vision of "Making the Internet
easy" by developing and distributing category-specific toolbars and a proprietary home page solution, powered by Google search.
‣ Two core business:
‣ Vertical affinity ALOT Toolbars and ALOT Home
‣ Screensavers.com for free screensavers and wallpapers
Robert Bergquist, CEO, Widemile‣ Leading provider of multivariate testing and optimization
technologies and services for online agencies and advertisers
‣ Landing pages, microsites, sales funnels, registration pages
‣ Interactive ad campaigns and customer segment testing
Two Questions:
‣ Ad campaign costs rising?
‣ Satisfied with performance?
Test for Better Returns on Ad Spend
3%
21%
20%
16%
14%
27%
26%
43%
43%
67%
6%
22%
28%
29%
33%
38%
51%
56%
61%
62%
0% 20% 40% 60% 80% 100%
Integrate bid management and CRM
Add new engines
Integrate bid management and analytics
Integrate online/off-line campaigns
Local search
Add keyword match types
Target
Test landing pages
Test creative
Add new keywords
AgenciesAdvertisers
Question: Which tactics have had the most positive impact on your search marketing campaigns? (Select all that apply.)
Are You Optimizing for Better ROAS?
‣ Little experience with content or creative testing
‣ Done some A/B split testing
‣ Done some multivariate testing
‣ Managing MVT projects now
You are not alone
‣ 87% not satisfied with their campaign conversion rates
‣ 60% doing no testing or optimization of any kind
‣ Majority of those testing use just A/B
n=540, 9/6/07, Widemile webinar with SMN and Jupiter Research
Key Steps to Optimized Campaigns
‣ Know your audience
‣ Embrace design best practices
‣ Test marketing assumptions
What is Multivariate Optimization?
‣ Test multiple element variations quickly
‣ Learn how each influences performance
‣ Determine the optimal page configuration
‣ Tailor pages for traffic source and other customer segments
Sample Landing Page
MIVA Case Study: Weather Toolbar‣ Downloadable browser add-on toolbar shows various links to online weather sources
‣ Paid search campaigns deliver traffic
‣ Business objective is higher download and install rate
Marketing Questions:‣ What is the best type of hero shot?
‣ What is the impact of the TRUSTe logo?
‣ Are people worried about spyware?
‣ Do people prefer specific toolbar features over lifestyle/benefits statements?
‣ Are people confused by the Google feature list?
‣ How much should we highlight the toolbar?
Optimization ProcessAnalysis‣ Understand audience and value
proposition
‣ Review existing conversion process
Action‣ Develop testing plan (new templates,
recipe matrix, and content variables)
‣ Execute tests
‣ Report and analyze results
‣ Deploy optimal pages
‣ Leverage lessons learned
Baseline Page
Assessment and Redesign - Wireframe
Google Product Information
Header
TRUSTeSpyware FREE
Hero Shot
Call To Action Button
Product Description
Headline
2
3
4
5
6
7
8
1
Defining Test Matrix
No. Factor (Variable)Levels
(versions)
1 Header 4
2 Hero shot 4
3 TRUSTe logo 4
4 Headline 2
5 Call-to-action 2
6 Product description 2
7 Spyware-free message 2
8 Google info 2
2048 potential combinations tested with 16 experiments
Example: Header - which one works best?
Level 2
Level 3
Level 4
Level 1
MVT Determines What Works Best
2
3
4
5
6
7
8
1
Which TRUSTe logo works best?
Level 2
Level 3
Level 4
Level 1
OFF
Test Complete in Two Weeks
Copyright 2007 – 2008 Widemile Inc. All Rights Reserved. Confidential Slide 17
Test Duration
Test Data Stabilization
Convers
ion R
ate
And the Winner is…
And the Winner is…Header
Hero Shot
Google Product Information
TRUSTe
Spyware FREE
Call To Action Button
Product Description
Headline
0 10 20 30 40 50
Factor 2
Factor 6
Factor 1
Factor 5
Factor 3
Factor 7
Factor 4
Factor 8
What’s Most Important?
Google Info
Headline
Spyware
TRUSTe
Call-to-action
Header
Product Description
Hero shot
HIG
HM
ED
IUM
LO
W
Influence Index
Analysis‣ Spyware, TRUSTe, Headline, and Google information
factors had the greatest influence on performance
‣ All references to the toolbar and specific features lost to lifestyle descriptions
‣ De-emphasizing the Google product features increased conversions
‣ People favored toolbar benefits over features
1
2
3
4OFF
Factor Analysis: TRUSTe logo
‣ Two new TRUSTe logos beat the old logo (and the logo turned off) – 8% lift delta
‣ Tested redesigned logos to reduce call-to-action confusion
‣ Recommend seals of approval with a small text link below to avoid competition with primary call-to-action objective
Rela
tive C
onvers
ion
Rate
Greater Campaign Efficiency
Twice the number of toolbar installs for the same ad spend
10
0%
21
5%
0
20
40
60
80
100
120
140
160
180
200
Baseline Optimized
100
200
Ind
ex:
Convers
ions
/ A
d S
pend
The Three Steps - Recap
‣ Know your audience‣ Leverage your SEM and analytics resources
‣ Refine your value proposition
‣ Embrace design best practices‣ Simplify the page and remove distractions
‣ Educate yourself
‣ Test your marketing assumptions‣ Value proposition and incentives
‣ Traffic characteristics
Helpful Resources
Conversational Marketingby Ian Lurie
Portent Interactive
Web Design for ROIby Lance Loveday & Sandra Niehaus
Closed Loop Marketing
Next Step
Test current optimal page
versus
New creative design using optimal content
Thank You
Sam Ee – [email protected] Robert Bergquist – [email protected]
Content Informationwww.widemile.com
www.miva.com