View
989
Download
3
Category
Preview:
Citation preview
The Data Never LiesHow to Get the Truth from Marketing Data
If you torture the data long enough, it will confess.”
- Ronald Coase
“
You, the marketer
The poor,poor data
Client Case: Men’s Skincare Product
• Holiday Gift Promotion
• Bootstrapped (Small Budget)
• Sell Primarily Online
• Ran Facebook Ads, No Sales
What’s the problem?
• Strategy
• Tactics
• Creative
Types of Problems You Can Have
or
Marketing Problems Statistics Problems
• Not Enough Data
• Bad Experiment Design
• Correlation vs. Causation
These are “real” problems.Things that we should test.
We’re wrong.But, we don’t know we’re wrong.
Resources:
Today’s Full Slide Deck
• On SlideShare
Open Source Statistics Textbook
After Today’s Talk
Subscribe at nickrosener.co/breakfast
Extra Goodies for Subscribers
Download and Use With Clients:
Excel Files
• Simulations and Calculations
Marketing Experiment Report Template
“Real” Marketing Problems
Strategy
Tactics
Creative
Things Like:
• Segmentation
• Targeting
• Positioning
• Product
“Real” Marketing Problems
Strategy
Tactics
Creative
Things Like:
• Social Media
Platform Choice
• Ad Format
“Real” Marketing Problems
Strategy
Tactics
Creative
Things Like:
• Copy
• Images
• Video
The Stats ProblemsWhat we’re going to cover
Estimation(basically, measurement)
Experiments(how to test stuff)
and
Up next:
Model the Process
Impressions
Profit
Visits to Landing Page
Purchases
Impressions:
Clickthrough Rate:
Cost / Click: $1
Conversion Rate:
Retail Price: $100/item
Gross Profit Margin: 50%
Gross Profit: $50/item
Model the Process
Impressions
Profit
Visits to Landing Page
Purchases
Impressions: Not Important*
Clickthrough Rate: Not Important*
Cost / Click: $1
Conversion Rate:
Retail Price: $100/item
Gross Profit Margin: 50%
Gross Profit: $50/item
Model the Process
Impressions
Profit
Visits to Landing Page
Purchases
Impressions: Not Important*
Clickthrough Rate: Not Important*
Cost / Click: $1
Conversion Rate:
Retail Price: $100/item
Gross Profit Margin: 50%
Gross Profit: $50/item
At breakeven, total ad cost = total gross profit
Model the Process
Impressions
Profit
Visits to Landing Page
Purchases
Impressions: Not Important*
Clickthrough Rate: Not Important*
Cost / Click: $1
Conversion Rate: 2% @ breakeven
Retail Price: $100/item
Gross Profit Margin: 50%
Gross Profit: $50/item
At breakeven, total ad cost = total gross profit
Now, New Questions
How do we find out if the adhas a conversion rate that is above breakeven?
(vs. a conversion rate of zero)
and
How much money do we needto spend to find out?
Estimate “The Truth”
Estimating a Proportion
Lots of thingsin marketing
are proportions.
Conversion Rates
Engagement Rates
Clickthrough Rates
Open Rates
How it Works
Your Data (Sample)
Estimating a Proportion
To estimate a conversion rate:
1. Set up an ad
2. Let it run
3. Count the # of purchases attributed to the ad
4. Divide by total # of clicks
Estimating a Proportion
Smaller Sample=
Higher Probabilityof Being Wrong
(and not knowing it)
=
More Affordable
Larger Sample
=
More ConfidenceThat You’re Right
=
More Expensive
>
Spending $5 on an Ad
90.4% of the time, our estimate of conversion is 0%while the true conversion rate is 2%.
Estimated Conversion %
Probability
0% 20% 40% 60% 80%
90.4% 9.2% <1% 0% 0%
100%
0%
Watch Your Stopping Point
$100
Bigger Budget = More Reliable
$500
Nick’s Budget Rule of Thumb
In our example, that’s $250 - $500+ @ $1/click and a 2% conversion rate.
Spend enough where you would expect to getat least 5 – 10+ “events.”
The Stats ProblemsWhat we’re going to cover
Estimation(basically, measurement)
Experiments(how to test stuff)
and
Let’s switch gears:
Study Population
Why Do Experiments?
Using “When to Poston Social Media” Studies
Here’s why:
Your Community
= , IMHO.
≠
Correlation
Why Do Experiments?
Using “When to Poston Social Media” Studies
Here’s why:
Causation
= , IMHO.
≠
Retrospective
Why Do Experiments?
Using “When to Poston Social Media” Studies
Here’s why:
Experiment
= , IMHO.
≠
Why Experiments > Retrospective
Meet John,a Social Media Manager
When are my customersmost engaged?
I’ll analyze my posts,and see when I got
the highest engagement!
Why Experiments > Retrospective
Looks like my customers prefer
to engage at midday.
This may be true.But there could be
another explanation.
Why Experiments > Retrospective
Wait. I also eat baconand drink coffee at
10:00 every day.
Hungry andTired
Bacon-InspiredBrilliance
Post-CoffeeCrash Remember:
Correlation ≠ Causation
Typical Problems with Retrospectives
Hidden Third Factor
ObservableThing 1
ObservableThing 2
HiddenThing
Causes Also Causes
Correlated,but not causal
Stats Concept:Lurking Variable
Typical Problems with Retrospectives
“Fishing” for Insights
What
Stats Concept:Spurious Correlation
Disclaimer:This is also a problem for poorly-designed
experiments.
Post Afternoon
Simple Randomized Experiment Design
Post Early
Randomizeinto Groups
Write Social Media Posts
Post Midday
MeasureMeasure
Measure
Sample Size:At least 5 – 10 events
per group.
What We Went OverWe covered a ton.
1. “Real” Marketing Problems vs. Statistics Problems
2. Model the Process
3. Estimate a Proportion
4. Bigger Sample = More Confident
5. At Least 5 – 10+ “Events” Per Group
6. Experiments > Retrospective, When Possible
Resources:Slides, Excel Files, Report Templateshttp://nickrosener.co/breakfast
Contact:@nickrosenernick@nickrosener.co
?
Questions
Recommended