Landing Page Optimization: 6 Years of Testing Distilled Into 5 Critical Lessons

Landing Page Optimization6 Years of Testing Distilled

Into 5 Critical Lessons

Taboola is the world's leading content discovery platform, serving 300B

recommendations to over 550M unique visitors each month on the web's most

innovative publisher sites, including NBC, USA Today, The Weather Channel, The

Atlantic and Fox Sports.

Landing Page Optimization6 Years of Testing Distilled

Into 5 Critical Lessons

Control Treament

Control Treament

Triple digit increase in CTR

Double digit increase in sign-ups

Lesson 1.

Build Landing Pages for Fast Thinkers

I’m a split test junkie!

@ContentVerve

How many animals of each kind did Moses bring on the ark?

2

0

I’m a split test junkie!

System 1 System 2

FastAutomaticEmotionalSubconscious

Slow EffortfulLogical Conscious

(Intuitive thinking) (Analytical thinking)

System 1 System 2

A machine for jumping to conclusions

The lazy controller

“In the economy of action, effort is a cost… Laziness is built deep into our nature.”

The Law of Least Effort

- Daniel Kahneman

Cognitive Ease

System 1 System 2“Easy” “Strained”

Cognitive Ease

System 1 System 2“Ahhhh” “ARGHH”

Cognitive Ease

ARGHH!!!!

5 Second Test

Great way to check what System 1 is up to…

What do you think this company sells?


A/B Testing


Building Landing Pages

http://bit.ly/1LMFFsx

Usability Hub:

Lesson 2.

Your Landing Page is a Piece of a Bigger Puzzle

Landing Page

Ad SERP

Banner Email

Landing Page Conf. PageInteraction

Step

PPC Organic Display Email Social

Ad SERP

Banner Email


Step


Ad SERP

Banner Email


Step


Ad SERP

Banner Email


Step


Ad SERP

Banner Email


Step


Ad SERP

Banner Email


Step


Ad SERP

Banner Email

Landing Page Conf. Page


Desktop - Mobile - Tablet

Interaction Step

Ad SERP

Banner Email




Interaction Step

Lesson 3.

Understand Your Conversion Problems Before You Try to Solve Them

Control: Treatment:

Type ZIP and city

A: Form with 9 fields

B: Form with 6 fields


B: Form with 6 fieldsType ZIP and city


B: Form with 6 fieldsType ZIP and city

✓ 4 full weeks of data✓ 512 conversions

✓ 95% confidence level



14.23% drop in leads

Type ZIP and city



Highest interaction

Lowest interaction

Type ZIP and city

Highest drop-off

Moved to top

Tweaked label copy

Tweaked label copy

Type ZIP and city

Control: Treatment B:

Type ZIP and cityType ZIP and city

A: Control (9 fields)

B: Variation (9 fields w/ tweaked labels)

Type ZIP and city



Type ZIP and city



✓ 4 full weeks of data✓ 671 conversions ✓ 96% confidence level

Type ZIP and city




19.21% increase in leads

Type ZIP and city

14.23% drop in leads 19.21% increase in leads

Treatment A Treatment B

Type ZIP and city


@ContentVerve

19.21% increase in leads

Treatment A Treatment B

IT IS DIFFICULT TO SOLVE A PROBLEM YOU DON’T UNDERSTAND

Is there anything holding you from doing conversion research?


1. Time

2. Client/Company Buy-In

3. Budget

4. Don’t know where to start

Expert Marketer’s

Magic Crystal Ball

SPLIT TESTING IS NOT AN EXCUSE TO SKIP YOUR HOMEWORK!!!

DON’T THINK OF SPLIT TESTING AS A RESEARCH TOOL

Ad SERP

Banner Email




Interaction Step

Aagaard’s Custom LPO Report

Goal 1 (Completions) Goal 1 (Conversion Rate)

http://bit.ly/1LtdEUY


/landing-page/1

/landing-page/2

/landing-page/3

/landing-page/4

/landing-page/5

- Look at Goal Completions - Be careful with low sample sizes (<100 = unreliable, > 250 = good, > 500 = better) - Bounce Rate is good for comparing pages & performance

- What is the device mix? - Compare Bounce Rate across devices - Compare goal completions across devices


- Get into the individual channels - Look at flow composition, where is everything coming from?- Compare Bounce Rate & Goal Completions across channels- Traffic quality (high traffic low quality vice versa)


- Look at proportion of New & Returning users

- Compare Goal Completions & Bounce Rate


Landing Page

Performance Across Devices Performance

Across Channels

Performance Based on User Type

Bounce Rates

Traffic Composition

Conversion Rates

Sample Sizes

Goal Completions

Ad SERP

Banner Email




Interaction Step

Type URL structure

Write down number of users

http://bit.ly/1KdiQdM

Basic Users / Page Report

1. Landing page (URL): 80,280 users

2. Form page (URL): 15,545 users

3. Success (URL): 4,599 users

0

22500

45000

67500

90000

Landing Page Sign-up Page Confirmation

Landing Page

Sign-up Page

Conf. Page

0

22500

45000

67500

90000


Landing Page

Sign-up Page

Conf. Page

-81%

-70%

@ContentVerve

0

22500

45000

67500

90000


Landing Page

Sign-up Page

Conf. Page

-24%

-92%

Second Page: Where do users go after they hit your landing page?

Ad SERP

Banner Email




Interaction Step

1. What are the top 3 questions from potential

2. What do you answer when you get

3. Are there any particular aspects of X that people don’t understand?

4. What aspects of X do people like the most/least?

5. Did I miss anything important? Got something to add?

Questions For Customer Success

0

20

40

60

80

No, here's why: Yes, I looked at: Not yet, but I'm going to look at:

Yes, I looked at:No, here’s why:Not yet, but I’m going to look at:

70%

17% 13%

-- -------

Other Research Tools / Methods

SEE THROUGH THE EYES OF YOUR USERS


1. Time

2. Client/Company Buy-In

3. Budget

4. Don’t know where to start

Lesson 4.

A/B Testing is Not a Magical Solution to Your Conversion Problems

@ContentVerve

An example from the real world

Control Treatment(progress bar) (no progress bar)

@ContentVerve

9 full weeks / 1506 conversions / inconclusive

-3.7% drop in leads

Average

Inconclusive

Sample: 1506 conversions

-3.7% drop in leads

Average

Inconclusive


Desktop

Sample: 1000+ conversions

-9,8% drop in leads Confidence 99%

-3.7% drop in leads

Average

Inconclusive


Desktop


-9,8% drop in leads Confidence 99%

10.3% increase in leadsConfidence 95%


Mobile

Control Treatment(progress bar) (no progress bar)

@ContentVerve

X Misinterpreting the data / wrong conclusionX Believing the average / not getting the whole storyX Illusion of being “scientific” X Not actually testing your hypothesisX Thinking your testing one thing - actually testing anotherX Cross-device mixup

How do I know when I can stop my A/B test?

95% confidence does NOT guarantee valid test results

After 1 day

+151.4% / confidence 97%

655 visitors / 18 conversions

After 2 days



After 1 week



After 2 weeks



After 4 weeks


Total sample: 9396 visitors / 122 conversions

Sample Size

What we have now: 9,396

What we need: 34,304 visitors in total

What we’re missing:24,908 visitors 73% of the required sample size…

http://www.evanmiller.org/ab-testing/sample-size.html

Confidence alone is no guarantee that you’ve collected a big enough sample of representative data.!

Confidence alone is no guarantee that you’ve collected a big enough sample of representative data.

Sample size and test duration are absolutely crucial in judging whether your test is cooked…

!!

Test duration: 4 weeks (2 or more business cycles)

Sample size: Pre-calculated sample size (visitors)

400+ conversions (goals)

Confidence: 95% or higher (no overlapping intervals)

What I look for:

Lesson 5.

Best Practice Isn’t Best




Type ZIP and city



Triple digit increase in CTR

Double digit increase in sign-ups

Ad Landing Page Conf. Page

PPC Brand

Campaign

Desktop

Sign-Up


PPC Brand

Campaign

Desktop

Sign-Up Process


PPC Brand

Campaign

Desktop

Sign-Up ProcessX X

JadeLou

@ContentVerve

A: Control

B: Variation


(Power level 100% / P-value 0%)

A: Control

B: Variation


157.67% increase in CTR


A: Control

B: Variation


157.67% increase in CTR

52.23% increase in sign-ups


Thanks for listening - YOU ROCK!

[email protected]

http://try.unbounce.com/taboola/

Take Unbounce for a spin & get 50% off for 3 months:

[email protected]

THANK YOU!

www.linkedin.com/in/inbaryagur

Marketing

Landing Page Optimization: 6 Years of Testing Distilled Into 5 Critical Lessons