The Roles of Uncertainty and Randomness in Online Advertising Ragavendran Gopalakrishnan Eric Bax Raga Gopalakrishnan 2 nd Year Graduate Student (Computer Science), Caltech Product Manager (Marketplace Design), Yahoo!

AD-SLOT Display Advertising

Simple Model for Display Advertising AD-SLOT AD SELECTION ALGORITHM ad calls ads w/ bids resultant matching (selected ad for each ad call) implement feedback webpage

Objective  Make Money! ? m ad calls ad slot ad 1ad 2ad n k1k1 k2k2 knkn b1b1 b2b2 bnbn s1s1 s2s2 snsn... Bid Value Response Rate May not be the right thing to do, for two reasons: – Reason 1: Not Incentive Compatible – Reason 2: Coming up…

The Caveat The response rate is not known, it has to be estimated. The actual revenue differs from the estimated expected revenue due to two factors: – Uncertainty (error in estimating response rates s i ) – Randomness (fluctuations around the response rate: )

billion ad calls per day AD 1AD 2 ad slot $1 per response w/ prob w/ prob ½ w/ prob ½ $1 million $1000 (0.1%)$0.3 million (30%) Bid Value Estimated Response Rate Estimated Expected Revenue Standard Deviation of Revenue How bad can Uncertainty be?

How can we combat it? How much time do we have? Long-TermShort-Term LEARNING RISK SPREADING MAIN FOCUS ? Future Work Again, these solutions are not automatically incentive compatible.

m ad calls ad slot ad 1ad 2ad n k1k1 k2k2 knkn b1b1 b2b2 bnbn S1S1... Bid Value Response Rate S2S2 SnSn RevenueX 1 (S 1 ) X 2 (S 2 )X n (S n )X i (S i ) X 1i (S i ) X 2i (S i )X mi (S i )... Model for Variance of Revenue

Model for Variance (contd.) The variance of the revenue can be derived as: Independent Returns Case: UNCERTAINTYRANDOMNESS

ad 1 S k ad calls Mean = pStd. Dev. = d*p X(S) is Bernoulli w/ parameter S Fraction of Variance Due to Uncertainty is Factors affecting Variance

Uncertainty or Randomness?

Bottom Line Uncertainty can be really bad Real World – Uncertainty dominates Long-TermShort-Term LEARNING RISK SPREADING SOLUTION

ad 1 v learning ad calls u responses p real : Real response rate (unknown) Estimate p real as p = u/v ad 1 k ‘real’ ad calls Fraction of Variance due to Uncertainty is Effect of Learning

Bottom Line Uncertainty can be really bad Real World – Uncertainty dominates Long-TermShort-Term LEARNING RISK SPREADING SOLUTION

AD 1AD 2 $1 per response w/ prob w/ prob ½ w/ prob ½ $1 million $1000 (0.1%)$0.3 million (30%) Bid Value Estimated Response Rate Estimated Expected Revenue Standard Deviation of Revenue billion ad calls per day Variance of Revenue New Strategy: Use each of a billion ads iid to AD 2 on each ad call Variance of revenue = Effect of Risk Sharing

Formalize Risk-Sharing The goal of sharing risk and bringing the variance down motivates the following optimization problem:

generate response rates Normal Distribution  = 0.001,  = “CPC” ADS generate response rates Normal Distribution  = ,  = “CPA” ADS Bid $1 Bid $10 Simulations Start with an assumed prior (uniform, approximate or exact) All 20 ads are given learning ad calls each, responses are counted, corresponding posteriors are obtained using Bayes’ Rule Method 1 (Portfolio): Compute the optimal portfolio and allocate ad calls accordingly Method 2 (Single Winner): Allocate all ad calls to the ad with the highest estimated expected revenue Compare Results

Estimated Expected Revenue

Uniform Prior – Actual Expected Revenue

Uniform Prior – Efficiency

Uniform Prior – Allocation by share of ad calls

Uniform Prior – Allocation by actual expected revenue

Exact Prior – Actual Estimated Revenue

Exact Prior – Allocation by share of ad calls

Exact Prior – Allocation by actual expected revenue

Approximate Prior – Actual Expected Revenue

Approximate Prior – Allocation by share of ad calls

Approximate Prior – Allocation by actual expected revenue

A Word of Caution – Covariance Randomness is usually uncorrelated over different ad calls. More often than not, uncertainty is correlated over multiple ads, as their response rates could be estimated through a common learning algorithm. Covariance can be estimated from empirical data, using models that are specific to the contributing factors (e.g., specific learning methods used).

Summary Actual Revenue differs from Estimated Expected Revenue for two reasons – uncertainty and randomness. Uncertainty can be very bad, and dominates randomness in most cases. Learning helps reduce uncertainty in the long run, but in the short run, portfolio optimization (risk distribution) is one way to combat uncertainty. Simulations show that actual revenue can improve as an important side effect of reducing uncertainty.

Further Directions… Can we tie up the long term and short term solutions? – Example: Consider the explore-exploit family of learning methods. – After every explore step, we have better estimates of response rates, but they may still be bad. So the exploit phase could be replaced with the portfolio optimization step! – Side Effect: Additional exploration in the “exploit” phase. – Is this an optimal way of mixing the two? Financial Markets – does it make sense for risk- neutral investors to employ portfolio optimization? Incentive Compatibility – can we deal with it?

Thank You Questions?