1. The Roles of Uncertainty and Randomness in Online Advertising Ragavendran Gopalakrishnan Eric Bax Raga Gopalakrishnan 2 nd Year Graduate Student (Computer Science), Caltech Product Manager (Marketplace Design), Yahoo!

2. AD-SLOT Display Advertising

3. Simple Model for Display Advertising AD-SLOT AD SELECTION ALGORITHM ad calls ads w/ bids resultant matching (selected ad for each ad call) implement feedback webpage

4. Objective  Make Money! ? m ad calls ad slot ad 1ad 2ad n k1k1 k2k2 knkn b1b1 b2b2 bnbn s1s1 s2s2 snsn... Bid Value Response Rate May not be the right thing to do, for two reasons: – Reason 1: Not Incentive Compatible – Reason 2: Coming up…

5. The Caveat The response rate is not known, it has to be estimated. The actual revenue differs from the estimated expected revenue due to two factors: – Uncertainty (error in estimating response rates s i ) – Randomness (fluctuations around the response rate: )

6. billion ad calls per day AD 1AD 2 ad slot $1 per response 0.001 w/ prob 1 0.0007 w/ prob ½ 0.0013 w/ prob ½ $1 million $1000 (0.1%)$0.3 million (30%) Bid Value Estimated Response Rate Estimated Expected Revenue Standard Deviation of Revenue How bad can Uncertainty be?

7. How can we combat it? How much time do we have? Long-TermShort-Term LEARNING RISK SPREADING MAIN FOCUS ? Future Work Again, these solutions are not automatically incentive compatible.

8. m ad calls ad slot ad 1ad 2ad n k1k1 k2k2 knkn b1b1 b2b2 bnbn S1S1... Bid Value Response Rate S2S2 SnSn RevenueX 1 (S 1 ) X 2 (S 2 )X n (S n )X i (S i ) X 1i (S i ) X 2i (S i )X mi (S i )... Model for Variance of Revenue

9. Model for Variance (contd.) The variance of the revenue can be derived as: Independent Returns Case: UNCERTAINTYRANDOMNESS

10. ad 1 S k ad calls Mean = pStd. Dev. = d*p X(S) is Bernoulli w/ parameter S Fraction of Variance Due to Uncertainty is Factors affecting Variance

11. Uncertainty or Randomness?

12. Bottom Line Uncertainty can be really bad Real World – Uncertainty dominates Long-TermShort-Term LEARNING RISK SPREADING SOLUTION

13. ad 1 v learning ad calls u responses p real : Real response rate (unknown) Estimate p real as p = u/v ad 1 k ‘real’ ad calls Fraction of Variance due to Uncertainty is Effect of Learning

14. Bottom Line Uncertainty can be really bad Real World – Uncertainty dominates Long-TermShort-Term LEARNING RISK SPREADING SOLUTION

15. AD 1AD 2 $1 per response 0.001 w/ prob 1 0.0007 w/ prob ½ 0.0013 w/ prob ½ $1 million $1000 (0.1%)$0.3 million (30%) Bid Value Estimated Response Rate Estimated Expected Revenue Standard Deviation of Revenue billion ad calls per day 0 + 100000090000000000 + 1000000 Variance of Revenue New Strategy: Use each of a billion ads iid to AD 2 on each ad call Variance of revenue = 90 + 1000000 Effect of Risk Sharing

16. Formalize Risk-Sharing The goal of sharing risk and bringing the variance down motivates the following optimization problem:

17. generate response rates Normal Distribution  = 0.001,  = 0.0001 10 “CPC” ADS generate response rates Normal Distribution  = 0.0001,  = 0.00001 10 “CPA” ADS Bid $1 Bid $10 Simulations Start with an assumed prior (uniform, approximate or exact) All 20 ads are given 100000 learning ad calls each, responses are counted, corresponding posteriors are obtained using Bayes’ Rule Method 1 (Portfolio): Compute the optimal portfolio and allocate ad calls accordingly Method 2 (Single Winner): Allocate all ad calls to the ad with the highest estimated expected revenue Compare Results

18. Estimated Expected Revenue

19. Uniform Prior – Actual Expected Revenue

20. Uniform Prior – Efficiency

21. Uniform Prior – Allocation by share of ad calls

22. Uniform Prior – Allocation by actual expected revenue

23. Exact Prior – Actual Estimated Revenue

24. Exact Prior – Allocation by share of ad calls

25. Exact Prior – Allocation by actual expected revenue

26. Approximate Prior – Actual Expected Revenue

27. Approximate Prior – Allocation by share of ad calls

28. Approximate Prior – Allocation by actual expected revenue

29. A Word of Caution – Covariance Randomness is usually uncorrelated over different ad calls. More often than not, uncertainty is correlated over multiple ads, as their response rates could be estimated through a common learning algorithm. Covariance can be estimated from empirical data, using models that are specific to the contributing factors (e.g., specific learning methods used).

30. Summary Actual Revenue differs from Estimated Expected Revenue for two reasons – uncertainty and randomness. Uncertainty can be very bad, and dominates randomness in most cases. Learning helps reduce uncertainty in the long run, but in the short run, portfolio optimization (risk distribution) is one way to combat uncertainty. Simulations show that actual revenue can improve as an important side effect of reducing uncertainty.

31. Further Directions… Can we tie up the long term and short term solutions? – Example: Consider the explore-exploit family of learning methods. – After every explore step, we have better estimates of response rates, but they may still be bad. So the exploit phase could be replaced with the portfolio optimization step! – Side Effect: Additional exploration in the “exploit” phase. – Is this an optimal way of mixing the two? Financial Markets – does it make sense for risk- neutral investors to employ portfolio optimization? Incentive Compatibility – can we deal with it?

32. Thank You Questions?