Taboola is the world's leading content discovery platform, serving 300B
recommendations to over 550M unique visitors each month on the web's most
innovative publisher sites, including NBC, USA Today, The Weather Channel, The
Atlantic and Fox Sports.
System 1 System 2
FastAutomaticEmotionalSubconscious
Slow EffortfulLogical Conscious
(Intuitive thinking) (Analytical thinking)
“In the economy of action, effort is a cost… Laziness is built deep into our nature.”
The Law of Least Effort
- Daniel Kahneman
Ad SERP
Banner Email
Landing Page Conf. Page
PPC Organic Display Email Social
Desktop - Mobile - Tablet
Interaction Step
Ad SERP
Banner Email
Landing Page Conf. Page
PPC Organic Display Email Social
Desktop - Mobile - Tablet
Interaction Step
A: Form with 9 fields
B: Form with 6 fieldsType ZIP and city
✓ 4 full weeks of data✓ 512 conversions
✓ 95% confidence level
A: Form with 9 fields
B: Form with 6 fields
14.23% drop in leads
Type ZIP and city
✓ 4 full weeks of data✓ 512 conversions
✓ 95% confidence level
A: Control (9 fields)
B: Variation (9 fields w/ tweaked labels)
✓ 4 full weeks of data✓ 671 conversions ✓ 96% confidence level
Type ZIP and city
A: Control (9 fields)
B: Variation (9 fields w/ tweaked labels)
✓ 4 full weeks of data✓ 671 conversions ✓ 96% confidence level
19.21% increase in leads
Type ZIP and city
Is there anything holding you from doing conversion research?
1. Time
2. Client/Company Buy-In
3. Budget
4. Don’t know where to start
Ad SERP
Banner Email
Landing Page Conf. Page
PPC Organic Display Email Social
Desktop - Mobile - Tablet
Interaction Step
Goal 1 (Completions) Goal 1 (Conversion Rate)
/landing-page/1
/landing-page/2
/landing-page/3
/landing-page/4
/landing-page/5
- Look at Goal Completions - Be careful with low sample sizes (<100 = unreliable, > 250 = good, > 500 = better) - Bounce Rate is good for comparing pages & performance
- What is the device mix? - Compare Bounce Rate across devices - Compare goal completions across devices
Goal 1 (Completions) Goal 1 (Conversion Rate)
- Get into the individual channels - Look at flow composition, where is everything coming from?- Compare Bounce Rate & Goal Completions across channels- Traffic quality (high traffic low quality vice versa)
Goal 1 (Completions) Goal 1 (Conversion Rate)
- Look at proportion of New & Returning users
- Compare Goal Completions & Bounce Rate
Goal 1 (Completions) Goal 1 (Conversion Rate)
Landing Page
Performance Across Devices Performance
Across Channels
Performance Based on User Type
Bounce Rates
Traffic Composition
Conversion Rates
Sample Sizes
Goal Completions
Ad SERP
Banner Email
Landing Page Conf. Page
PPC Organic Display Email Social
Desktop - Mobile - Tablet
Interaction Step
0
22500
45000
67500
90000
Landing Page Sign-up Page Confirmation
Landing Page
Sign-up Page
Conf. Page
0
22500
45000
67500
90000
Landing Page Sign-up Page Confirmation
Landing Page
Sign-up Page
Conf. Page
-81%
-70%
@ContentVerve
0
22500
45000
67500
90000
Landing Page Sign-up Page Confirmation
Landing Page
Sign-up Page
Conf. Page
-24%
-92%
Ad SERP
Banner Email
Landing Page Conf. Page
PPC Organic Display Email Social
Desktop - Mobile - Tablet
Interaction Step
1. What are the top 3 questions from potential
2. What do you answer when you get
3. Are there any particular aspects of X that people don’t understand?
4. What aspects of X do people like the most/least?
5. Did I miss anything important? Got something to add?
Questions For Customer Success
0
20
40
60
80
No, here's why: Yes, I looked at: Not yet, but I'm going to look at:
Yes, I looked at:No, here’s why:Not yet, but I’m going to look at:
70%
17% 13%
Is there anything holding you from doing conversion research?
1. Time
2. Client/Company Buy-In
3. Budget
4. Don’t know where to start
-3.7% drop in leads
Average
Inconclusive
Sample: 1506 conversions
Desktop
Sample: 1000+ conversions
-9,8% drop in leads Confidence 99%
-3.7% drop in leads
Average
Inconclusive
Sample: 1506 conversions
Desktop
Sample: 1000+ conversions
-9,8% drop in leads Confidence 99%
10.3% increase in leadsConfidence 95%
Sample: 500+ conversions
Mobile
X Misinterpreting the data / wrong conclusionX Believing the average / not getting the whole storyX Illusion of being “scientific” X Not actually testing your hypothesisX Thinking your testing one thing - actually testing anotherX Cross-device mixup
Sample Size
What we have now: 9,396
What we need: 34,304 visitors in total
What we’re missing:24,908 visitors 73% of the required sample size…
http://www.evanmiller.org/ab-testing/sample-size.html
Confidence alone is no guarantee that you’ve collected a big enough sample of representative data.
Sample size and test duration are absolutely crucial in judging whether your test is cooked…
!!
Test duration: 4 weeks (2 or more business cycles)
Sample size: Pre-calculated sample size (visitors)
400+ conversions (goals)
Confidence: 95% or higher (no overlapping intervals)
What I look for:
A: Form with 9 fields
B: Form with 6 fields
14.23% drop in leads
Type ZIP and city
✓ 4 full weeks of data✓ 512 conversions
✓ 95% confidence level
A: Control
B: Variation
✓ 4 full weeks of data✓ 577 conversions ✓ 99% confidence level
(Power level 100% / P-value 0%)
A: Control
B: Variation
✓ 4 full weeks of data✓ 577 conversions ✓ 99% confidence level
157.67% increase in CTR
(Power level 100% / P-value 0%)
A: Control
B: Variation
✓ 4 full weeks of data✓ 577 conversions ✓ 99% confidence level
157.67% increase in CTR
52.23% increase in sign-ups
(Power level 100% / P-value 0%)
Thanks for listening - YOU ROCK!
http://try.unbounce.com/taboola/
Take Unbounce for a spin & get 50% off for 3 months: