Hybrid Keyword Auctions Ashish Goel Stanford University Joint work with Kamesh Munagala, Duke University

1 Online Advertising Pricing Models CPM (Cost per thousand impressions) CPC (Cost per click) CPA (Cost per acquisition) Conversion rates: Click-through-rate (CTR), conversion from clicks to acquisitions, … Differences between these pricing models: Uncertainty in conversion rates: Sparse data, changing rates, … Stochastic fluctuations: Even if the conversion rates were known exactly, the number of clicks/conversions would still vary, especially for small samples

2 Cost-Per-Click Auction Advertiser Auctioneer (Search Engine) Bid = Cost per Click C

3 Cost-Per-Click Auction Advertiser Auctioneer (Search Engine) Bid = Cost per Click C CTR estimate Q

4 Cost-Per-Click Auction Advertiser Auctioneer (Search Engine) Bid = Cost per Click C CTR estimate Q Value/impression ordering: C 1 Q 1 > C 2 Q 2 > … Give impression to bidder 1 at CPC = C 2 Q 2 /Q 1

5 Cost-Per-Click Auction Advertiser Auctioneer (Search Engine) Bid = Cost per Click C CTR estimate Q VCG Mechanism: Truthful for a single slot, assuming static CTR estimates Can be made truthful for multiple slots [ Vickrey-Clark-Groves, Myerson81, AGM06 ] This talk will focus on single slot for proofs/examples Value/impression ordering: C 1 Q 1 > C 2 Q 2 > … Give impression to bidder 1 at CPC = C 2 Q 2 /Q 1

6 When Does this Work Well? High volume targets (keywords) Good estimates of CTR What fraction of searches are to high volume targets? Folklore: a small fraction Motivating problem: How to better monetize the low volume keywords?

7

8

9 Possible Solutions Coarse ad groups to predict CTR: Use performance of advertiser on possibly unrelated keywords Predictive models Regression analysis/feature extraction Taxonomies/clustering Collaborative filtering Learn the human brain! Our approach: Richer pricing models + Learning

10 Hybrid Scheme: 2-Dim Bid Advertiser Auctioneer (Search Engine) M = Cost per Impression C = Cost per Click

11 Hybrid Scheme Advertiser Auctioneer (Search Engine) M = Cost per Impression C = Cost per Click CTR estimate Q

12 Hybrid Scheme Advertiser Auctioneer (Search Engine) M = Cost per Impression C = Cost per Click CTR estimate Q Advertisers score R i = max { M i, C i Q i }

13 Hybrid Scheme Advertiser Auctioneer (Search Engine) M = Cost per Impression C = Cost per Click CTR estimate Q Advertisers score R i = max { M i, C i Q i } Order by score: R 1 > R 2 > …

14 Hybrid Scheme Advertiser Auctioneer (Search Engine) M = Cost per Impression C = Cost per Click CTR estimate Q Advertisers score R i = max { M i, C i Q i } Order by score: R 1 > R 2 > … Give impression to bidder 1: If M 1 > C 1 Q 1 then charge R 2 per impression If M 1 < C 1 Q 1 then charge R 2 / Q 1 per click

15 Example: CPC Auction Bidder 1Bidder 2 Per click cost C 510 Conversion rate estimate Q C * Q CPC auction allocates to bidder 2 at CPC = 0.5/0.08 = 6.25

16 Example: Hybrid Auction Bidder 1Bidder 2 Per click cost C 510 Conversion rate estimate Q C * Q Hybrid auction allocates to bidder 1 at CPI = 0.8 Per impression cost M Max {M, C * Q}

17 Why Such a Model? Per-impression bid: Advertisers estimate or belief of CTR May or may not be an accurate reflection of the truth Backward compatible with cost-per-click (CPC) bidding

18 Why Such a Model? Per-impression bid: Advertisers estimate or belief of CTR May or may not be an accurate reflection of the truth Backward compatible with cost-per-click (CPC) bidding Why would the advertiser know any better? Advertiser aggregates data from various publishers Has domain specific models not available to auctioneer Is willing to pay a premium for internal experiments

19 Provable Benefits 1. Search engine: Better monetization of low volume keywords n Typical case: Unbounded gain over CPC auction n Pathological worst case: Bounded loss over CPC auction 2. Advertiser: Opportunity to make the search engine converge to the correct CTR estimate without paying a premium 3. Technical: a) Truthful b) Accounts for risk characteristics of the advertiser c) Allows users to implement complex strategies

20 Key Point Implementing the properties need both per impression and per click bids

21 Multiple Slots Show the top K scoring advertisers Assume R 1 > R 2 > … > R K > R K+1 … Generalized Second Price (GSP) mechanism: For the i th advertiser, if: If M i > Q i C i then charge R i+1 per impression If M i < Q i C i then charge R i+1 / Q i per click

22 Multiple Slots Show the top K scoring advertisers Assume R 1 > R 2 > … > R K > R K+1 … Generalized Second Price (GSP) mechanism: For the i th advertiser, if: If M i > Q i C i then charge R i+1 per impression If M i < Q i C i then charge R i+1 / Q i per click Can also implement VCG [ Vickrey-Clark-Groves, Myerson81, AGM06 ] Need separable CTR assumption Details in the paper

23 Uncertainty Model for CTR For analyzing advantages of Hybrid, we need to model: Available information about CTR Asymmetry in information between advertiser and auctioneer Evolution of this information over time We will use Bayesian model of information Prior distributions Specifically, Beta priors (more later)

24 Bayesian Model for CTR Advertiser Auctioneer (Search Engine) True underlying CTR = p

25 Bayesian Model for CTR Advertiser Auctioneer (Search Engine) True underlying CTR = p Prior distribution P adv Prior distribution P auc (Private) (Public)

26 Bayesian Model for CTR Advertiser Auctioneer (Search Engine) CTR estimate Q True underlying CTR = p Prior distribution P adv Prior distribution P auc Per-impression bid M (Private) (Public)

27 Bayesian Model for CTR Advertiser Auctioneer (Search Engine) CTR estimate Q True underlying CTR = p Prior distribution P adv Prior distribution P auc Per-impression bid M Each agent optimizes based on its current belief or prior: Beliefs updated with every impression Over time, become sharply concentrated around true CTR (Private) (Public)

28 What is a Prior? Simply models asymmetric information Sharper prior More certain about true CTR p E[ Prior ] need not be equal to p Main advantage of per-impression bids is when: Advertisers prior is more resolved than auctioneers Limiting case: Advertiser certain about CTR p Priors are only for purpose of analysis Mechanism is well-defined regardless of modeling assumptions

29 Truthfulness Advertiser assumes CTR follows distribution P adv Wishes to maximize expected profit at current step E[ P adv ] = x = Expected belief about CTR Click utility = C Expected profit = C x - Expected price

30 Truthfulness Advertiser assumes CTR follows distribution P adv Wishes to maximize expected profit at current step E[ P adv ] = x = Expected belief about CTR Click utility = C Expected profit = C x - Expected price Bidding ( Cx, C ) is the dominant strategy Regardless of Q used by auctioneer and true CTR p Elicits advertisers expected belief about the CTR! Holds in many other settings (more later)

Specific Class of Priors

32 Conjugate Beta Priors Auctioneers P auc for advertiser i = Beta(, ), are positive integers Conjugate of Bernoulli distribution (CTR) Expected value = / ( + )

33 Conjugate Beta Priors Auctioneers P auc for advertiser i = Beta(, ), are positive integers Conjugate of Bernoulli distribution (CTR) Expected value = / ( + ) Bayesian update with each impression: Probability of click = / ( + ) If click, new P auc (posterior) = Beta(, ) If no click, new P auc (posterior) = Beta(, )

34 Evolution of Beta Priors 1,1 2,11,2 2,23,11,3 3,22,31,44,1 1/2 2/31/3 2/3 3/4 1/4 1/2 1/4 3/4 Denotes Beta(1,1) Uniform prior Uninformative E[P auc ] = 1/4 E[P auc ] = 2/5 Click No Click

35 Properties of Beta Priors Larger, Sharper concentration around p Uninformative prior: Beta(, ) = Uniform[0,1] Q = E[ P auc ] = / ( + ) Auctioneers expected belief about CTR Could be different from true CTR p

Benefits to Auctioneer and Advertisers

37 Advertiser Certain of CTR Advertiser Auctioneer (Search Engine) CTR estimate Q = / ( + ) True underlying CTR = p Knows p P auc = Beta(, ) Per-impression bid M = Cp (Private) (Public)

38 Properties of Auction Revenue properties for auctioneer: Typical case benefit: log n times better than CPC scheme Bounded pathological case loss: 36% of CPC scheme Unbounded gain versus bounded loss! Flexibility for advertiser: Can make P auc converge to p without paying premium But pays huge premium for achieving this in CPC auction

39 Better Monetization Adv. 1 Adv. 2 Adv. 3 Adv. n … Auctioneer p1p1 p2p2 p3p3 pnpn P auc = Beta (, log n) Q 1 / log n Low volume keyword: Auctioneers prior has high variance Some p i close to 1 with high probability

40 Better Monetization Hybrid auction: Per-impression bid elicits high p i CPC auction allocates slot to a random advertiser Theorem: Hybrid auction generates log n times more revenue for auctioneer than CPC auction

41 Flexibility for Advertisers Advertiser Auctioneer (Search Engine) Q = / ( + ) Knows p P auc = Beta(, ) Per-impression bid M = p Assume C = 1

42 Flexibility for Advertisers Advertiser Auctioneer (Search Engine) Q = / ( + ) Knows p P auc = Beta(, ) Per-impression bid M = p Wins on per impression bid Pays at most p per impression

43 Flexibility for Advertisers Advertiser Auctioneer (Search Engine) Q = / ( + ) Knows p P auc = Beta(, ) Per-impression bid M = p T impressions N clicks Makes Q converge to p Wins on per impression bid Pays at most p per impression

44 Flexibility for Advertisers Advertiser Auctioneer (Search Engine) Q = / ( + ) Knows p P auc = Beta(, ) Per-impression bid M = p T impressions N clicks Makes Q converge to p Now switch to CPC bidding

45 Flexibility for Advertisers If Q converges in T impressions resulting in N clicks: ( N T) p Since Q = /( + ) < p, this implies N T p Value gain = N ; Payment for T impressions at most T * p No loss in utility to advertiser! In the existing CPC auction: The advertiser would have to pay a huge premium for getting impressions and making the CTR converge

Dynamic Properties

47 Uncertain Advertisers Advertiser wishes CTR p to resolve to a high value In that case, she can gain utility in the long run … but CTR resolves only on obtaining impressions! Should pay premium now for possible future benefit What should her dynamic bidding strategy be?

48 Uncertain Advertisers Advertiser wishes CTR p to resolve to a high value In that case, she can gain utility in the long run … but CTR resolves only on obtaining impressions! Should pay premium now for possible future benefit What should her dynamic bidding strategy be? Key contribution: Defining a new Bayesian model for repeated auctions Dominant strategy exists!

49 The Issue with Dynamics Adv. 1 Adv. 2 Auctioneer P adv1 P adv2 P auc1 P auc2 Advertiser 1 underbids so that: Advertiser 2 can obtain impressions Advertiser 2 may resolve its CTR to a low value Advertiser 1 can then obtain impressions cheaply [Bapna & Weber 05, Athey & Segal 06]

50 Semi-Myopic Advertiser Maximizes utility in contiguous time when she wins the auction Priors of other advertisers stay the same during this time Once she stops getting impressions, cannot predict future … since future will depend on private information of other bidders!

51 Semi-Myopic Advertiser Maximizes utility in contiguous time when she wins the auction Priors of other advertisers stay the same during this time Once she stops getting impressions, cannot predict future … since future will depend on private information of other bidders! Advertiser always has a dominant hybrid strategy Bidding Index: Computation similar to the Gittins index Advertiser can optimize her utility by dynamic programming Socially optimal in many reasonable scenarios Implementation needs both per-impression and per-click bids

52 Summary Allow both per-impression and per-click bids Same ideas work for CPM/CPC + CPA Significantly higher revenue for auctioneer Easy to implement Hybrid advertisers can co-exist with pure per-click advertisers Easy path to deployment/testing Many variants possible with common structure: Optional hybrid bids Impression + Click [S. Goel, Lahaie, Vassilvitskii, 2010]

53 Conclulsion and Open Questions Learning is important in Online advertising Remember that there are multiple strategic participants Design richer pricing and communication signals Some issues that need to be addressed: Whitewashing: Re-entering when CTR is lower than the default Fake Clicks: Bid per impression initially and generate false clicks to drive up CTR estimate Q Switch to per click bidding when slot is locked in by the high Q

54 Open Questions Some issues that need to be addressed: Whitewashing: Re-entering when CTR is lower than the default Fake Clicks: Bid per impression initially and generate false clicks to drive up CTR estimate Q Switch to per click bidding when slot is locked in by the high Q Analysis of semi-myopic model Other applications of separate beliefs?

55 Open Questions Some issues that need to be addressed: Whitewashing: Re-entering when CTR is lower than the default Fake Clicks: Bid per impression initially and generate false clicks to drive up CTR estimate Q Switch to per click bidding when slot is locked in by the high Q Analysis of semi-myopic model Other applications of separate beliefs? Connections of Bayesian mechanisms to: Regret bounds and learning [ Nazerzadeh, Saberi, Vohra 08 ] Best-response dynamics [ Edelman, Ostrovsky, Schwarz 05 ]

56 Truthfulness Advertiser assumes CTR follows distribution P adv Wishes to maximize expected profit at current step E[ P adv ] = x = Expected belief about CTR Utility from click = C Expected profit = C x - Expected price Let Cy = Per impression bid R 2 = Highest other score If max ( Cy, C Q ) < R 2 then Price = 0 Else: If y < Q then: Price = x R 2 / Q If y > Q then: Price = R 2