The Science of Ad Testing By Brad Geddes

#SMX #11b @bgTheory

Brad Geddes, Co-Founder AdAlysis

The Science of Ad Testing

#SMX #11b @bgTheory

Ad Testing is about Improvement and/or insights

#SMX #11b @bgTheory

Single Ad Group Testing

#SMX #11b @bgTheory

Multi Ad Group Testing

Idea 1Idea 2Idea 3Idea 4Idea 5

#SMX #11b @bgTheory

List Your Themes Create a label per theme Examine how each theme performs Pick your winning theme and continue

Image Ad Testing Can be Single or Multi-Ad Group

#SMX #11b @bgTheory

Use Cases

Single Ad Group TestingInsights within an ad group- Best for high traffic ad groups- Best for Brand ad groups- Good for other ad groups after you’ve done some multi-ad group testing for global insights

Multi-Ad Group TestingInsights across a segment- Best for template created ads

- Best for small data accounts

- Best for market research

#SMX #11b @bgTheory

Some Quick Examples

#SMX #11b @bgTheory

Ask the Big Questions First• Are consumers price sensitive? (discounts, prices in ads)• Does Location matter? (use geographies in ads)• Does shipping matter? (Free or Rush shipping)• Is selection important? (large, exclusive, variety, etc)• How much does quality matter? (cheap, luxury)• What’s your best CTA? (Call, Shop, Find, Discover, etc)• Should your display URL be products, geo, CTAs, or something else?• Do we need to pre-qualify clicks? (b2b vs b2c)• etc…

#SMX #11b @bgTheory

Should You Use Customizers?

Testing Problem:• 804 ads in test 1• 75 ads in test 2Solution: Use same ad groups for your test

#SMX #11b @bgTheory

Static Ad is Higher Conversion Rate, Lower in CPACustomizer is Higher in CTR & CPIWhat metrics are important to you?

Exact Same Ad Groups in Test

#SMX #11b @bgTheory

Lead Gen Company 1: GEOs are Clear Winners

Next Step: Change a few campaigns to only target singe region with non-geo keywords and rerun test to see if there’s a difference. Then decide account organization.

Should You Localize Your Ads?

#SMX #11b @bgTheory

Ideas from presenters are Ideas to Test: Not absolute facts for you to implement!Desktop

Mobile

Company 2: Is the data consistent across devices?

#SMX #11b @bgTheory

Think Through Your ResultsIn this case, ads were localized – landing pages were not.GEO increased CTR, but nothing else. Is that the market or the website?Solution: Redo test with localized pages

#SMX #11b @bgTheory

First Test, it appears that the brand is so strong that you should use the brand name in non-branded ad groups.

It’s a good idea to test your results to ensure they are consistent.

Should You Use Your Brand in Non-Brand Ad Groups?

#SMX #11b @bgTheory

Results aren’t 100% in yet…However, non-brand ad 1 is an ‘incentive’ ad. • This is for a travel company. Problem with many bookings is people are afraid that

the price will drop and they will overpay.• Ad 1 talks to that issue with a coupon if the price drops

Don’t Just Do One TestSame test: different results

#SMX #11b @bgTheory

How to Start Testing

#SMX #11b @bgTheory

Determine Your Segments

Basic Segments: • Your brand• 3rd party brands • Products• Informational • Long tail• By buying cycle entrance• Etc…

Ad Type: • Text ads• DSAs• Image ads• Call only ads• HSAs• etc…

Devices:• Mobile• Desktops• All

#SMX #11b @bgTheory

Targeting Method is a Segment with Rich Media Ads

Basic Segments: • Your brand• 3rd party brands • Products• Informational • Long tail• By buying cycle entrance• Etc…

Targeting Type:• Remarketing• Topic• Placement• Interest• Affinity• etc

Devices:• Mobile• Desktops• All

#SMX #11b @bgTheory

Define Your Testing Metrics

Metric Best UsageCTR: Click Through Rate Max Traffic (note: CPI is usually still better)

Conversion rate Most conversions from landing pageCPI: Conversion per Impression Most conversions possible

CPA: Cost per Conversion Lowering cost per conversionROAS: Return on Ad Spend Hitting ROAS goalsRPI: Revenue per Impression Maximizing revenue

#SMX #11b @bgTheory

What Minimum Data To Define

Metric Impressions Clicks Conversions Timeframe

CTR Yes Optional YesCPA Yes YesConversion rate Optional Yes YesCPI Yes Optional YesROAS Yes YesRPI Yes Optional Yes

#SMX #11b @bgTheory

Minimum Data Guidelines

Impressions Clicks Conversions

Low Traffic 350 300 7Mid Traffic 750 500 13High Traffic 1000 1000 20Well known brand terms 100,000 10,000 100 - 1000

The MINIMUM Data – It’s OK to have HIGHER Minimums|

#SMX #11b @bgTheory

Define Minimum Confidence Levels

Term Type Minimum ConfidenceLong Tail Keywords 90%Mid data terms 90% – 95%3rd Party Brands you Sell 90% (small brands) to 95% (large brands)

Top Keywords (the ones you watch daily)

95% - 99%

Your Brand Terms 99%

#SMX #11b @bgTheory

Option 1: 3 months

Option 2: Define maximums for each metric Define timeframes

Define Maximum Data

#SMX #11b @bgTheory

Test by Device (if appropriate)

 Count of AdRow Labels Mobile All Effective Device

Ad Group 1 2 AllAd Group 2 NoneAd Group 3 1 2 Mobile/DesktopAd Group 4 3 1 Mobile/DesktopAd Group 5 10 AllAd Group 6 2 6 Mobile/DesktopAd Group 7 1 7 Mobile/DesktopAd Group 8 1 6 Mobile/DesktopAd Group 9 7 All

#SMX #11b @bgTheory

If below minimum data, wait. If above minimum, calculate confidence factors If below minimum confidence – wait If above minimum confidence – take action

If above maximum data and a test has no results, take action

Action = Pausing loser OR pausing loser & creating new challenger In the case of multi-ad group testing: Replace entire string with a

new pattern to test (optional): Use insight in other areas, such as email, landing

pages, etc

Actions

#SMX #11b @bgTheory

For Remarketing: Use at Least 3 Offersad clicks impressions CTR conversions conversion rate1 4075 613621 0.66% 132 3.24%2 6725 835389 0.81% 321 4.77%3 4399 565363 0.78% 197 4.48%4 1610 672062 0.24% 27 1.68%5 8547 939540 0.91% 41 0.48%

Total 25,356 3,625,975 0.70% 718 2.83%

ad clicks impressions CTR conversions conversion rate2 19,294 3,720,132 0.52% 443 2.30%

#SMX #11b @bgTheory

Determine what you want to test: Single ad group Multi-ad group

Define: Testing metrics Minimum data Maximum data Segments

Confidence levels per segment

Run Your test When you reach minimum data:

Run confidence calculations If winners & losers – take action If no winners – wait

When you reach max data Pause test and reset

When you have winners in Multi-Ad Group testing:

Apply that data to other channels

Recap

#SMX #11b @bgTheory

THANK YOU!

SEE YOU @SMX WEST

SAN JOSE, CAMARCH 1-3, 2016