Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Bid Optimization and Inventory Scoring
Claudia Perlich, Brian Dalessandro, Rod Hook,
Ori Stitelman, Troy Raeder, Foster Provost
Media6Degrees
1
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Shopping at one of our campaign sites
cookies
100 Million
URL’s
100 Million
Browsers
0.0001% to 1%
baserate Billions of
Auctions
per day
conversion
Ad Exchange
Where should
we advertise and
at what price?
Does the ad
have an effect?
Can we group
content?
Data Quality
Management?
Attribution?
Who should
we target for
a product?
M6D Display Advertising in a Nutshell
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Life of a Browser
3
1. Initiate: create Cookie when the browsers first comes across one of our data partners
2. Monitor:
track browsing activity in the scope of our data
partners
track ‘brand actions’ for our advertisers
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
The Non-Branded Web
BrowserId: 1234
URLId:Type abkcc:SN
kkllo:blog
88iok:SN
7uiol:twitter
Cookie stats user agent
date of first seen
# of interactions
Purchases 3012L20 4199L30 … 3075L50
A consumer’s online activity
The Branded Web
gets recorded like this:
Monitor: This is what we see
4
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Life of a Browser
5
1. Initiate
2. Monitor
3. Score and Segment
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Media
Traditional Approach
Use web browsing behavior to model
consumers into buckets, then sell those
buckets to advertisers. This is a three-
model process that is largely a legacy
of offline marketing science.
Use web browsing behavior to model
direct associations between media and
brands. This is a one-model process that
is only possible because of innovations in
web-scale technology.
Advances in web-scale data and technology require a new approach
to audience selection.
Brand Media Brand
Computational Advertising
Descriptive Buckets Description Not Necessary
6
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Modeling Your ‘Brand Affinity’/ Prospect Rank
Using Bayesian Statistics and Stochastic Gradient Decent Logistic Regression, we
estimate statistical correlations between 10s of millions of web URLs and 1000s of
branded actions.
The output of this process is the basis of our ProspectRank scoring
Lik
elih
oo
d t
o C
on
ve
rt g
ive
n V
isit
Passion
Aversion
non-branded websites
7
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Brand Interest
Ad Ad Ad
Category Awareness
Dynamic Scoring and ‘Segmenting’
8
Ad
m6d ProspectRank
OBSERVATION
Achieves
ProspectRank
Visits Branded
Site
ProspectRank
Threshold
site visit with positive correlation
site visit with negative correlation
ENGAGEMENT
Some prospects fall
out of favor once their
in-market indicators
decline.
Sudden spikes in prospect rank
suggest in-market behavior
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Summary
9
• We have ‘segments’ of good
prospects (top 1 percentile) that
come out of a zoo of predictive
models in millions of dimensions
that we cannot really explain to
anybody
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Life of a Browser
10
1. Initiate: create Cookie
2. Monitor
3. Score and Segment
4. Sync with the exchange
5. Activate ‘Segment’
6. Receive Bid request
7. Bid
8. Show Impression
9. Track Conversion
10. Cycle ….
11. Cookie Deletion
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Real Time Bidding
11
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Modern Display Advertising:
Ad-Exchanges and Real Time Bidding
• Ad Exchange
Marketplace that brings together supplier of inventory - places to show ads and advertisers wanting to show ads
• Real Time Bidding
second price Vickrey auction
some strange rule that nobody knows
15 ms to submit a bid (not much room for deliberation …)
• Some M6D Numbers
# of bid request per day: ~3 Billion
# of bids: ~200 Million
# of ads: ~35 Million
# of exchanges: ~20
# of unique cookies we see per day: ~56 Million
12
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
What exactly is Inventory?
13
Where the ad will be shown: 7K unique inventories + default buckets
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Why should the inventory matter?
Causal Impact on
Conversion Propensity
• Perceptiveness
• Inventory Quality
14
Information about the Organic
Conversion Propensity
• Contextual Relevance
• Current Intention
• Cookie Life Expectation
The inventory tells us what the browser is doing RIGHT NOW
How do we measure this and how do we incorporate this into our targeting and bidding?
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
What does it mean anyway: Bid-Optimization?
Some musing …
Optimal?
o Vickrey suggest that you bid what it is worth …
o We do not really know what a conversion is worth …
Why separate the bid vs. no bid from the bid price?
o Inventory effect is probably fairly consistent across good prospects
o Targeting happens BEFORE we run media, inventory only later
o I do not have a lot of positives so I need to keep the dimensionality down
…
Why do we need a model at all: can’t we just measure it?
o Segments have different quality and trafficking decisions introduce
confounding
Other considerations: frequency, volume, attribution/timing
Interaction effects between campaigns …
15
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Inventory Model Estimation
Measure how much better a single piece of inventory i is than the
‘average’ inventory j for campaign a over all browsers u
Build separate models for each campaign a
Need to control for the trafficking decisions of the different segment s,
not really for the user u (speed up scoring time)
Instead of integrating the expectation separately, we just estimate it by
removing that features!
),|(),,|( iucpaiucp a
),|(),|( iscpiucp aa
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Campaign Properties in Empirical Comparison
DATA
• 100 active campaigns
• 3 weeks of impressions
• conversions within 7 day
ESTIMATION DETAILS
• L1 constrained logistic
• down-sample negatives
• 50K test set for performance
• same training set for both, just one less feature
• correct predictions for down-sampling
• feature selection: online include i with 5 positives in expectation
• keep all instances for calibration
17
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Out of Sample Model Performance
Does the inventory carry ANY information?
18
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Example of Model Scores for Hotel Campaign
• Scores are calculated on de-duplicated training
pairs (i,s)
• We even integrate out s
• Nicely centered around 1
19
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Bidding Strategies
Strategy 0:
• always bid base price for segment
• equivalent to constant score of 1 across all inventories
• consistent with an uninformative inventory model
Strategy 1:
• auction-theoretic view: bid what it is worth in relative terms
• So we multiply the base price with ratio
Strategy 2:
• optimal performance is not to bid what it is worth but to trade of value for quality and only bid on the best opportunities
• apply a step function to the model ratio to translate it into a factor applied to the price:
ratio below 0.8 yields a bid price of 0 (so not bidding),
ratios between 0.8 and 1.2 are set to 1 and ratios above
1.2 bid twice the base price
20
1
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Impact on Life Campaigns
We either want better conversion at constant margin or higher margin at constant performance (we are typically paid on CPM)
Conversion Rate (PVSVR) strategy 2 should do well Percentage of impressions leading to site visits within 7 days.
Higher conversion rates are better.
Cost per Acquisition (CPA) strategy 1 should do well This metric combines cost and conversion rate and looks at the total
cost of impressions for a given strategy relative to the total number of conversion.
Lower CPA is better.
Potential Unintended Effects:
cross impact on other campaigns
delivery
21
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Performance Results of Bidding
Experiments:
• 15 campaigns in the initial trail
• 2 weeks of data for evaluation
• A/B test where we run in parallel all 3 strategies on random
subsets of browsers
22
Comparison Type S1 vs. S0 S2 vs. S0
Conversion Prospects +6% + +21% +
Retargeting +3% +24% +
CPA Prospects +1% +18% +
Retargeting - 2% - 4% +
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Closing Thoughts/Future work
• Bid optimization: Include frequency and attribution (in production)
• Data incest: By bidding more we are winning better browsers and it becomes a self-fulfilling prophesy
• Integrate the continuous targeting score into the ‘value’ assessment of the impression in addition to the inventory
• Measure cross-campaign impacts: we recently switched over all campaigns
• Pacing: if you know there are many good opportunities available for one campaign and few for the other, the second should get precedence
23
© 2012 Media6Degrees. All Rights Reserved. Proprietary and Confidential
Acknowledgement
24
Data Science/Tech team
Foster Provost
Brian Dalessandro
Troy Raeder
Ori Stitelmann
Other Stuff we do in computational advertising
Paper on privacy preserving targeting at KDD 2009
Paper on attribution at KDD 2012
Paper on large scale machine learning in advertising at KDD 2012
Paper on observational methods to measure adfx at KDD 2011
Paper on clicks and alternative proxies submitted to JAR