SES San Jose Ad Testing Research & Findings

1

Ad Testing: Research & Findings

2

What are the Keys to Successful Testing?

• Design your test around a single question• Have a large differentiation b/w tested elements• Account for changes in temporal behavior• Collect enough data to reduce margin of error• Segment results to provide added insight• Act on the results!

3

Don’t Guess! - Ads for “loan”

loanGreat Rates. On Time ClosingNo lender fees. Bad Credit OKwww.loans.brand.com

brand loans apply nowLow Rates. Approval in MinutesPersonal Service. Apply Online Now!www.brand.com

A

B

C

low rate loansLow Rates. Approval in MinutesNo lender fees. Bad Credit OKwww.loans.brand.com

4

Don’t Guess! - loan adgroup

loanGreat Rates. On Time ClosingNo lender fees. Bad Credit OKwww.loans.brand.com

brand loans apply nowLow Rates. Approval in MinutesPersonal Service. Apply Online Now!www.brand.com

A

B

C

low rate loansLow Rates. Approval in MinutesNo lender fees. Bad Credit OKwww.loans.brand.com

3.79% Conversion

2.32% Conversion

0.95% Conversion

5

–Impression–Interest–Expectation–Persuasion

6

Ads Persuade!

7

Ads (the meat of the relevance sandwich)

8

Intention

Enticement

Goal Fulfillment

9

Test Ads by

Match Type!

10

Test 1: Is Running Ads on Brand Queries a Waste of Money?

Example SERP w/o Ad Example SERP w/ Ad

11

Test Design

• 1 week ads on 1 week off• No issues of seasonality or promotion• 30 Branded Keywords• 1 version of ad

11

12

Did the Ads Drive Traffic?

• Google -1.3%• Yahoo -8.7%• Live -7.3%

NO

13

Did the Ads Increase Conversion Rate?

• Google +17.9%• Yahoo +32.1%• Live +11.7%

YES

14

Did the Ads Increase RPV?

• Google +15.4%• Yahoo +47.0%• Live +27.5%

YES

15

New Questions

• Marketing to Intent?• Power of Persuasion?• Power of Brand in Search?

15

16

Test 2: What is the Best Ad to Increase Conversion?

Ad TestingMultivariate Testing WorksIncrease Conversion Ratesottodigital.com/ads

Element 1

Element 2

Element 3

Element 4

4x3 L9 Orthogonal Array

4 Elements 3 Variations

17

Test Design

• 9 Ads Created and Tagged w/ URL Parameter• Landing and Thank You Page Tagged• Ad Serving Optimization Turned OFF

17

18

What does an L9 Array Really Look Like?Recipe (9) Title (3) D1 (3) D2 (3) URL (3)

A - Control {keyword} Search {AdGroup} suppliers

Compare Products Today

www.brand.com

B {keyword} Find {keyword} suppliers Search Thousands of Products

www.brand.com/{keyword}

C {keyword} Find Parts fast at (brand) {keyword} catalog search

www.brand.com/catalogs

D {keyword} suppliers Search {AdGroup} suppliers

Compare Products Today

www.brand.com/catalogs

E {keyword} suppliers Find {keyword} suppliers Search Thousands of Products

www.brand.com

F {keyword} suppliers Find Parts fast at (brand) {keyword} catalog search

www.brand.com/{keyword}

G {keyword} catalog Search {AdGroup} suppliers

Compare Products Today

www.brand.com/{keyword}

H {keyword} catalog Find {keyword} suppliers Search Thousands of Products

www.brand.com/catalogs

I {keyword} catalog Find Parts fast at (brand) {keyword} catalog search

www.brand.com

19

What Were the Results?

Recipe Visitors Conversion Lift

A - Control 11.80% 5.88% ---

B 14.30% 4.29% -27.05%

C 14.04% 6.08% 3.42%

D 9.34% 2.57% -56.29%

E 13.00% 6.16% 4.72%

F 9.42% 5.38% -8.50%

G 10.06% 5.04% -14.32%

H 10.41% 5.13% -12.82%

I 7.63% 8.04% 36.71%

20

What Were the Results?

Recipe Visitors Conversion Lift

A - Control 11.80% 5.88% ---

B 14.30% 4.29% -27.05%

C 14.04% 6.08% 3.42%

D 9.34% 2.57% -56.29%

E 13.00% 6.16% 4.72%

F 9.42% 5.38% -8.50%

G 10.06% 5.04% -14.32%

H 10.41% 5.13% -12.82%

I 7.63% 8.04% 36.71%

21

What is the breakdown?

• Other Ads the Used Winning Title– -14%

– -12%

• Other Ads the Used Winning D1– +3.42%

– -8.5%

• Other Ads the Used Winning URL– Control

– +4.72%

www.brand.com

Find Parts fast at (brand)

{keyword} catalog

22

What Ad Elements Influenced Conversion?

Element Winning Alternative Influence on Conversion

Title {keyword} suppliers 13.54%

D1 Find {keyword} suppliers 53.53%

D2 Search Thousands of Products

23.56%

URL www.brand.com/{keyword} 9.37%

23

What’s Better?

{keyword} catalogFind parts fast at (brand){keyword} catalog searchwww.brand.com

{keyword} suppliersFind {keyword} suppliersSearch Thousands of Productswww.brand.com/{keyword}

Winning Recipe Best Predicted

Best Predicted +52% Lift in Conversion Rate

24

Jonathan Mendez, Chief Strategy Officer

My Blog:

optimizeandprophesize.com