Optimization Problems in Internet Advertising Cliff Stein Columbia University Google Research

Internet Advertising  Multi-billion dollar business  Google purchased DoubleClick for over 3 billion dollars, 2 years ago  A new paradigm for advertising  Many interesting mathematical, economic, and computational problems

Some Statistics VendorAd viewers Google1,118 DoubleClick1,079 Yahoo362 MSN309 AOL156 Adbrite73 Total3,087 (millions, one month in 2008) **From Wikipedia

Internet Advertising  2 main types Adwords (right column, auction) Adwords (right column, auction) Display ads (scattered, matching) Display ads (scattered, matching)

This talk  Overview of some issues in internet advertising  Talk about two particular algorithmic results, one on adwords optimization, one on display ads  No attempt to comprehensively survey the area

Adwords  Advertisers bid to have the right to appear when a user enters a search query  Advertisers are chosen according to some kind of auction mechanism  Ads are displayed for free. Advertiser pays when a user clicks.

New Kind of Auctions  Traditional auctions are for one item, or a small number of items, and are done on a human time scale  Many types of auctions (ascending, sealed-bid, second price …) are well understood mathematically  Adwords auctions are a new type of auction, where a bidder bids on multiple items for many items on a “computer” time scale  Mathematically, it is a whole new ballgame

Technical Challenges  Mechanism Design - designing rules of a game or system to achieve a specific outcome, even though each agent may be self-interested  Technology to implement the auctions  Optimization for Advertiser (maximize clicks, minimize money spent) Advertiser (maximize clicks, minimize money spent) Internet company (maximize profit, maximize user happiness, minimize bad publicity/lawsuits) Internet company (maximize profit, maximize user happiness, minimize bad publicity/lawsuits)

Basic Mechanism Design  Let’s set up an auction for  Suppose it is worth $20 to me, but $10 to Alex Rodriguez (and these values are private)  We each bid some amount: Who gets the hat? Who gets the hat? How much do they pay? How much do they pay?

Basic Mechanism Design Auction Mechanism Private value 2 Private value 1 Bidders Bid 2 Bid 1 Gives item to someone Collects payments

Some goals in an auction  Truth revealing (bidders want to bid their values)  Revenue maximizing (auctioneer maximizes profit)  Social welfare (good for bidders) These goals may be in conflict

First price auctions  Everyone bids, highest bidder gets item for the price he bid, others pay nothing and get nothing.  Bidders Utility = value of item received – payment.

First price auction  My utility = $20 - $20 = 0  Arod utility = $0 - $0 = 0. Arod $10 Me $20 First Price Auction Mechanism Cliff gets it. Cliff pays $20 Arod pays $0 Bid 20 Bid 10

Can I do better?

First price auction (2)  My utility = $20 - $15 = 5  Arod utility = $0 - $0 = 0. Arod $10 Me $20 First Price Auction Mechanism Cliff gets it. Cliff pays $15 Arod pays $0 Bid 15 Bid 10

First Price Auctions Suppose I bid $15, Arod bids $10.  I get item for $15, Arod gets nothing for nothing.  My utility = = 5  Arod utility = 0-0 = 0  I am happier  I am not telling the “truth” anymore  My bid needs to be a function of everyone else’s bid (in addition to my value)  Auctioneer makes less money

Second Price Auctions  Everyone bids. Item goes to the highest bidder, who pays the bid of the second- highest bidder.  Bidders Utility = value of item received – payment

Second price auction  My utility = $20 - $10 = 10  Arod utility = $0 - $0 = 0. Arod $10 Me $20 Second Price Auction Mechanism Cliff gets it. Cliff pays $10 Arod pays $0 Bid 20 Bid 10

Second Price Auctions Theorem: In a second-price auction, no bidder has any incentive to lie. Proof: I don’t lie.  If I lie any amount >= 10, nothing changes.  If I lie < 10, I lose the auction and my utility is 0.  Therefore I don’t want to lie.

Second Price Auctions Theorem: In a second-price auction, noone has any incentive to lie. Proof: Arod doesn’t lie.  If he lies any amount < 20, nothing changes for him.  If he lies at x > 20, he wins for $20, but his utility is 20 –x which is negative, so he is worse off.

Notes on Second Price Auctions  Everyone tells the truth  May yield less revenue than first-price auctions  Note: we assume that I get no benefit from seeing you pay more money.

Back to internet auctions  We are not auctioning off baseball hats, or paintings, or cars that were used in the commission of a crime….  We are auctioning off slots on a web page  There are: Multiple slots Multiple slots Advertisers bidding on many keywords Advertisers bidding on many keywords Many auctions going on during the course of a day Many auctions going on during the course of a day

Multiple Slots: Generalized Second Price Auctions  Users are bidding for slots, where slot i is more valuable than slot i+1.  Higher slots tend to get more clicks, and therefore cost more (cost per click)  Sort bids, i’th bid gets slot i and pays bid of slot i+1.

Example of GSP Auction  Bids: $1, $0.50, $2.25, $1.20, $0.10, $2.50, $1.25  5 slots

Example 2 of GSP Auction  Bids: $1, $0.50, $2.25, $1.20, $0.10, $2.50, $1.25  5 slots SlotBidPayment 1$2.50$2.25 2$2.25$1.25 3$1.25$1.20 4$1.20$1.00 5$1.00$0.50

Some Complications from GSP  We are bidding one amount, but don’t know which slot we will win.  Different advertisers have different preferences/beliefs about slots  You actually might have some incentives to lie. In “reality” (google), your bid is actually modified by the “quality” of your ad.

Advertisers bidding on many keywords  An advertiser sets up a campaign in which they advertise on many keywords  Each time a user issues a query that matches one of their keywords, the ad for that keyword enters the auction.  Depending on their bid (and other factors), they may get a slot in the auction  If someone clicks on their ad, they are charged

Question?  How does the advertiser control the total amount that they are charged? They specify a budget on how much they want to spend in a day

Budgets  Advertiser specifies a daily budget B.  They participate in an auction as long as the daily budget is not spent.

Example Advertiser Bid on “skiing” “hiking” Daily Budget Joe’s outdoor store $0.50$1.00$2.00 Campmor$1.00$0.75$3.00

Example  Auction 1: Hiking. Joe wins. When click happens, he pays $0.75 Advertiser Bid on “skiing” “hiking” Daily Budget Remaining Budget Joe’s outdoor store $0.50$1.00$2.00$2.00 Campmor$1.00$0.75$3.00$3.00

Example  Auction 1: Hiking. Joe wins. When click happens, he pays $0.75 Advertiser Bid on “skiing” “hiking” Daily Budget Remaining Budget Joe’s outdoor store $0.50$1.00$2.00$1.25 Campmor$1.00$0.75$3.00$3.00

Example  Auction 2: Skiing. Campmor. When click happens, he pays $0.50 Advertiser Bid on “skiing” “hiking” Daily Budget Remaining Budget Joe’s outdoor store $0.50$1.00$2.00$1.25 Campmor$1.00$0.75$3.00$3.00

Example  Auction 2: Skiing. Campmor. When click happens, he pays $0.50 Advertiser Bid on “skiing” “hiking” Daily Budget Remaining Budget Joe’s outdoor store $0.50$1.00$2.00$1.25 Campmor$1.00$0.75$3.00$2.50

Example  Auction 3: Hiking. Joe wins. When click happens, he pays $0.75 Advertiser Bid on “skiing” “hiking” Daily Budget Remaining Budget Joe’s outdoor store $0.50$1.00$2.00$1.25 Campmor$1.00$0.75$3.00$2.50

Example  Auction 3: Hiking. Joe wins. When click happens, he pays $0.75 Advertiser Bid on “skiing” “hiking” Daily Budget Remaining Budget Joe’s outdoor store $0.50$1.00$2.00$0.50 Campmor$1.00$0.75$3.00$2.50

Example  Auction 4: Hiking. Joe should win. But he doesn’t have enough budget, so Campmor wins. Pays some amount < 0.75 (depends on other bidders and system minimums). Advertiser Bid on “skiing” “hiking” Daily Budget Remaining Budget Joe’s outdoor store $0.50$1.00$2.00$0.50 Campmor$1.00$0.75$3.00$2.50

Lessons  Auction is not always won by the highest bidder.  Bidders may engage in strategic behavior, allow others to win at a high price so as to exhaust their budget and be removed from competition (bid jamming)

What about the advertiser?  Suppose you are an advertiser  You want to bid on many keywords  How should you allocate your budget among multiple keywords?

Example: Asics  Keywords: Shoe, sneaker, running shoe, gel kayano, gel nimbus, pronating, marathon shoe, race shoe, reebok, I hate asics, sporting goods, footware, racing attire, race, wicked fast, air jordan, laces, shin splints, white sneakers, …  Budget: $100  What to bid on each one?

Who advertises adwords?

Optimization Problem  Suppose that I knew Everyone else’s bids Everyone else’s bids Exactly how many times each keyword will come up Exactly how many times each keyword will come up The number of clicks associated with each position The number of clicks associated with each position Google’s exact advertising mechanism Google’s exact advertising mechanism How should I bid? (Note: Think of bid as cost-per-click (cpc))

Queries vs. Keywords asics Gel kayano I hate asics Red running shoes shoe asics I hate peanut butter Red asics Asics gel nimbush Running shoes

Bidding Problem  Bidding on multiple keywords is an NP-hard problem (knapsack)  Knapsack: Given a set of items with sizes and values and a knapsack of size B, choose a maximum value set of objects that fit in the knapsack.  Proof of difficulty is also demonstrated by the amount that people are willing to pay to solve the problem.

How do we evaluate a bidding strategy  Objective: maximize number of clicks, subject to budget  We compare the performance of our (simple) strategy to the performance of an arbitrarily complicated strategy of an adversary, take the ratio, and minimize over all inputs

Simple Strategies are pretty good  Feldman, Pal, Muthukrishnan, Stein  There exists one amount b, and if I bid b on every keyword, I will get at least ½ the clicks of any strategy  There exists two numbers b1 and and b2, and if I bid either b1 or b2 on every keyword, I will get at least 63% (e/(e-1))of the clicks of any strategy

Theorem is pessimistic

Display advertising  From ALENEX 2009, with Cosmin Negruseri, Mircea Pasio, Barbara Stanley, Cristian Strat

Display Ads  Publishers (e.g. newspapers, travel agents) make on-line “real estate” available for ads  Advertisers purchase impressions (e.g. 100 appearances on CNN money page on 5/3/2009)  Users visit pages, see advertisements  Market manager (e.g. google) has to decide which ad is displayed for each user

Optimization Problem  Publisher is offering certain types of unsold “inventory” (views on pages with certain demographics)  Advertiser is looking to buy some amount of inventory  Market manager needs to quickly decide if advertisers demand can be satisfied.

Algorithmic Ideas  Push/relabel algorithms work by pushing flow over on edge at a time (pushing downhill)  Excess accumulates at vertices  Vertices are relabeled, pulling them uphill.  Flow is pushed over eligible edges (pointing downhill, have residual capacity)

Previous experimental work  None that we are aware of

Synthetic Distributions  Uniform Random  High-Lo (chosen flow with additional edges)  Rope (long paths plus some useless edges)  Zipf (edge between i and j with probability proportional to 1/(ij). Said to model real-world web graphs First three are modifications of generators from [CG97

Experimental Questions  Can we efficiently solve the real problems we have?  On bipartite graphs, do the specialize algorithms help?  Which variants are best?  Which heuristics are needed?  Do these conclusions generalize to other data?

Typical results (hi-lo)

Conclusions for display ads  Bipartite flow algorithms are worth using in practice  FIFO is usually the best variant, and rarely the worst  Heuristics (gap relabelling, exact label updates) are helpful  Data is publicly available

General Conclusions  Internet advertising is an important area  Interesting algorithmic and game-theoretic problems need to be solved.  There are problems for the advertiser and for the auction maker and for the user  Ongoing work – studying the online problem of matching impressions with advertisers (e.g. Feldman, Korula, Muthukrishnan, Makroni 09)