Google’s Testing Platform

Google Confidential and Proprietary 1

Tom Leung

Senior Business Product Manager

Google Website Optimizer

Mea Culpa

• I’m about to pitch you really hard

• But it’s free

• It’s also something we use internally at Google for our own PPC campaigns

2

What We’ll Cover

�What is GWO

�Case Studies

�Demo

�Real Time Testing Diagnoses

�Top 5 Things We Learned So Far

�Tips & Resources (most free)

�Q & A 3Quick Survey on What Audience wants

Where Website Optimizer Fits In

Optimize Optimize ContentContent

Website Optimizer

4

Drive the Right Traffic

Monitor Performance

Bad Landing Pages: Where Good Leads Go to Die

Landing

Closing Pages

Ad Creatives

Landing Pages

5

0-3 seconds; 3% of

pixels, no commitment

0-20 seconds, 100% of

pixels, demonstrated

commitment

2-3 minutes, 100% of

pixels, significant

commitment

Life Before Testing: The Dark Ages

• “Cool ideas” that you just implement

• HiPPO

• Before and After

6

How Content Experimentation Works

Customers

Visit your Website

1/3 visitors

1/3 visitors

14% Make Purchase

20% Make Purchase

Original

Combo 1

Winning Combo

Test variations of the same page

1/3 visitors

1/3 visitors 20% Make Purchase

2% Make Purchase Customers Convert

Combo 2

Web’s #1 Testing Platform

• Benefits

� Free

• Features

� A/B Split Testing� Free

� Easy to use

� Does not impact SEO

� Backed by Google

� Increases in conversions >25% not uncommon

� Discussion groups, tutorials, consultant network

� Multivariate Testing

� Follow Up Testing

� Works with ALL traffic

� Statistical Analysis

� Platform Independent

� Quarterly feature updates

8

A/B and Multivariate Made Easy

9

Tagging Assistance

10Color Coded Sample Snippets

Combination Report Shows the Customer Favorite

Combination 11 is

performing the best

Bars represent relative

improvement

Original is highlighted

here

What We’ll Cover

�What is GWO

�Case Studies

�Demo

�Real Time Testing Diagnoses

�Top 5 Things We Learned So Far

�Tips & Resources (most free)

�Q & A 12

A: Uses “Free”, Picture, Action Oriented

13

B: Cleaner, Button, Value Proposition

14

B Page Increased Downloads by 30%!

WiderFunnel Case Study: Old Landing Page

Page Variations

• Now they know which home page to use for their new site.

• And, they have a 125% increase in lead conversions!

Landing Page Experiment Results

Winning Page – 125% Conversion Lift!

Eval Download Page Control

WiderFunnel’s Eval Download Page Variations

Eval Download Page Experiment Results

• Ran a follow-up to:

• Verify the results

• Protect revenue by eliminating the poor performing variation from the experiment

• Majority of traffic sent to winning page

• More accurate results look even better – 20.2% lift!

Eval Download Page Follow-up Results

• More accurate results look even better – 20.2% lift!

Can You Guess the Winner?

Winning Page – 20.2% Conversion Lift!

Nice Clean Page

28

Personal Welcome with Link to Live Person

29

What We’ll Cover

�What is GWO

�Case Studies

�Demo

�Real Time Testing Diagnoses

�Top 5 Things We Learned So Far

�Tips & Resources (most free)

�Q & A 30

What We’ll Cover

�What is GWO

�Case Studies

�Demo

�Real Time Testing Diagnoses

�Top 5 Things We Learned So Far

�Resources (most free)

�Q & A 31

What We’ll Cover

�What is GWO

�Case Studies

�Demo

�Real Time Testing Diagnoses

�Top 5 Things We Learned So Far

�Resources (most free)

�Q & A 32

Top 5 Things We’ve Learned So Far

1. Content Experimentation is a technology-enabled

managed process

2. Testing takes great alternate content, proportionate

traffic, and discipline.

3. Many people mistakenly assume the more complexthe test, the better.

4. It is totally possible to double your sales with testing

5. There are lots of myths out there about GWO

33

What We’ll Cover

�What is GWO

�Case Studies

�Demo

�Real Time Testing Diagnoses

�Top 5 Things We Learned So Far

�Tips & Resources (most free)

�Q & A 34

Some Pages Better Than Others

Tra

ffic

High

Landing Pages

Home Pages

Step 1 of Funnels

Previously

Optimized Pages

Tra

ffic

Known Problems with ContentLow High

Final Steps of Funnels

Ancillary Detail Pages

Terms of Use Pages

Contact Us Pages

Every Page Has an Optimal Recipe

Info RichShort n Sweet

Left Layout

Right Layout

A

A C

B

36

Facts Aspirations

Leads Sales

A

A

D

E

More Experimentation Ideas

• Headline / Image /

Call to Action

• Layout

• Coupons

• RSS button clicks

• Colors, fonts, sizes

• Testimonials

• Copy

• Embedded widgets

• Sales incentives

• Newsletter subscriptions

• Form Fields

Best Practices in Marketing Experimentation

• Test a small number of variations

– Rule of thumb is ~100 conversions per combination

• Test bold changes

– If you can’t see difference between two combos in 8 seconds, visitors probably won’t either

• Consider early indicators if you don’t have enough conversions

– E.g. if you have modest conversion volume, optimize for leading indicators such as request info, view product details, remain on page features page >5 seconds.

• Don’t jump to conclusions

– Less than 2 weeks is no good, focus on absolute conversion difference, don’t get too excited by sliver of green or red

Testing Resource Center

39Google.com/WebsiteOptimizer

WebsiteOptimizer.Blogspot.Com

40Weekly Posts about Experimentation, Content Optimization, and GWO

For blog, case studies, videos, more advice, access to your free GWO account, partner

directory, user forum, free technical support, premium technical support…

Tips &

Google Confidential and Proprietary

directory, user forum, free technical support, premium technical support…

Google.com/WebsiteOptimizer

41

Tips &

What We’ll Cover

�What is GWO

�Case Studies

�Demo

�Real Time Testing Diagnoses

�Top 5 Things We Learned So Far

�Tips & Resources (most free)

�Q & A 42