1. 1 Issues on the border of economics and computation נושאים בגבול כלכלה וחישוב Price of Anarchy and Stability Liad Blumrosen ©

2. Today Previously, we introduced some equilibrium notions. –Dominant strategies, Pure/mixed Nash, Correlated, no-regret sequences. Today, we will continue the discussion on Price of Anarchy (POA) and price of Stability (POS). If we reached an equilibrium, will it be near optimal? Far from optimal? We will start the discussion with the analysis of congestion games.

3. Today Reminder: –congestion games –Price of Anarchy (PoA) –Price of Stability (PoS) Congestion games with linear costs: –An upper bound on PoS –An upper bound on PoA PoA of real economic systems (or at least for their theoretical model…) –PoA in sponsored search.

4. Congestion games Definition: congestion games (משחקי גודש) –A set of players 1,…,n –A set of resources M = {1,…,m} –S i is the set of (pure) strategies of player i i.e., s i  S i is a subset of M. –Cost for the players that use resource j  M depends on the number of players using j : c j (n j ) –For s=(s 1,…,s n ), let n j (s) = the number of players using resource j –The total cost c i for player i:

5. Example 1: network cong. game Resources: the edges. Pure strategies: subsets of edges. Travel time on each edges: f(congestion) Player 1 wants to travel A  D –S 1 ={ {AB,BD}, {AC,CD}, {AC,CB,BD} } Player 2 wants to travel A  B –S 1 ={ {AC,CB}, {AB} } A B C D E c(n)=1 c(n)=n/2 c(n)=n 2 c(n)=10 c(n)=4n Consider the strategy profile: s 1 = {AC,CD} s 2 = {AC,CB} c 1 (s)=4+10 c 2 (s)=4+1 c(n)=n

6. Equilibria in congestion game Theorem: In every congestion game there exists a pure Nash equilibrium. –(At least one…) Proof idea: congestion games are exact potential games, with

7. Quality of equilibria We saw: congestion games admit pure Nash equilibria Are these equilibria “good” for the society? Approximately good? We will need to: –Specify some objective function. –Define “approximation”. –Deal with multiplicity of equilibria.

8. Price of anarchy/stability Price of anarchy: Price of stability: Cost of worst Nash eq. Optimal cost Cost of best Nash eq. Optimal cost When talking about cost minimization, POA and POS ≥1 Concepts are not restricted to Nash equilibria (similar concepts available for other types of equilibria)

9. 9 Price of Anarchy Price of anarchy: [Koutsoupias/Papadimitriou 99] quantify inefficiency w.r.t some objective function and some equilibrium concept. –e.g., Nash equilibrium Definition: price of anarchy (POA) of a game (w.r.t. some objective function): optimal obj fn value equilibrium objective fn value the closer to 1 the better Slide is stolen (and modified) from Tim Roughgarden.

10. Examples Optimization goal: social welfare (=sum of payoffs) Optimal cost: 1+1=2 Cost of worst NE = cost of best NE = 6 –One Nash equilibrium. POA = POS = 3 CooperateDefect Cooperate 1, 15, 05, 0 Defect 0, 53,33,3

11. Examples Optimization goal: social welfare Two pure equilibria: (Ballet, Ballet), (Football, Football) Optimal cost: 2+1=3 Cost of worst NE 1+4 = 5 –POA=5/3 Cost of best NE 1+2 = 3 –POS=1 BalletFootball Ballet 2, 15, 55, 5 Football 5, 51,41,4

12. 12 The Price of Anarchy Network w/2 players, each routes 1 unit of traffic from s to t: st 2x 12 5x 5 0

13. 13 The Price of Anarchy Nash Equilibrium: cost = 14+14 = 28 st 2x 12 5x 5 0

14. 14 The Price of Anarchy Nash Equilibrium: To Minimize Cost: Price of anarchy = 28/24 = 7/6. st 2x 12 5x 5 cost = 14+10 = 24 cost = 14+14 = 28 st 2x 12 5x 5 0 0

15. Approximation measurements Several approximation concepts in the design of algorithms: –Approximation ratio (approximation algorithms): what is the price of limited computational resources. –Competitive ratio (online algorithms): what is the price for not knowing the future. –Price of anarchy: the price of lack of coordination –Price of stability: price of selfish decision making with some coordination. Approximation results are one of the main contributions of algorithmic-game-theory to economic thinking.

16. Today Reminder: –congestion games –Price of Anarchy (PoA) –Price of Stability (PoS) Congestion games with linear costs: –An upper bound on PoS –An upper bound on PoA PoA of real economic systems (or at least for their theoretical model…) –PoA in sponsored search.

17. Price of stability in cong. games Meaning: in such games there exists pure Nash equilibria with cost which is at most double the optimal cost. Theorem: in congestion games with linear cost function, POS ≤ 2 –Objective: cost minimization. –Linear cost: c j (n j )=a j n j +b j for some a j,b j ≥0

18. Price of stability – proof (1 of 2) Proof: let Φ = potential function from previous slides. Consider a strategy profile s  S. We first compare: Φ(s) and c(s) = Σ i  N c i (s) Φ(s)≤ c(s) ≤ 2Φ(s)

19. Price of stability – proof (2 of 2) Proof: thus, for every strategy profile s, we have Let s* = argmin s Φ(s). As argued before, s* is a pure Nash equilibrium. Let s opt be the optimal solution, c(s opt ) = min s c(s) Then, c(s*)  POS ≤ c(s*)/c(s opt ) ≤ 2 Φ(s)≤ c(s) ≤ 2Φ(s) ≤ 2Φ(s*)≤ 2Φ(s opt )≤ 2c(s opt )

20. Bounds on PoA We will now show a bound on the PoA in the same setting. We will show it in a wider framework, that helps approaching this problem in a general way. –Approach is not restricted to congestion games. The approach also helps in proving bounds on the PoA for other solution concepts –Mixed Nash, correleated, no-regret.

21. 21 POA in general settings n players, each picks a strategy s i player i incurs a cost C i (s) Important Assumption: objective function is cost(s) :=  i C i (s) Key Definition: A game is (λ,μ)-smooth if, for every pair s,s * outcomes (λ > 0; μ < 1):  i C i (s * i,s -i ) ≤ λ∙cost(s * ) + μ ∙ cost(s) Slide is stolen (and modified) from Tim Roughgarden.

22. 22 Smooth => POA Bound Next: “canonical” way to upper bound POA (via a smoothness argument). notation: s = a Nash eq; s * = optimal Assuming (λ,μ)-smooth: cost(s) =  i C i (s) [defn of cost] ≤  i C i (s * i,s -i ) [s a Nash eq] ≤ λ∙cost(s * ) + μ∙cost(s) [prev. slide] Then: POA (of pure Nash eq) ≤ λ/(1-μ). Slide is stolen (and modified) from Tim Roughgarden.

23. 23 Example: congestion games Congestion games with linear costs. –Previously, we showed PoS ≤ 2. –We now prove: PoA ≤ 2.5 For pure Nash. Tight. (bound for mixed Nash is 2.618) We use smoothness arguments: we will show that congestion games with linear costs are (5/3,1/3)-smooth.  POA ≤ λ/(1-μ) = 5/3 / (1-1/3) = 5/2

24. 24 Example: congestion games Fact: –for all integers y,z ≥ 0 and reals a,b ≥ 0 Consider two strategy profiles s, s* –#users of resource k is n k, n * k, resp. Why? (Two arguments.) Follows from:

25. 25 Smoothness Stronger than POA bound Key point: to derive POA bound, only needed  i C i (s * i,s -i ) ≤ λ ∙ cost(s * ) + μ ∙ cost(s) [(*)] to hold in special case where s = a Nash eq and s * = optimal. Smoothness: requires (*) for every pair s,s * outcomes. –even if s is not a pure Nash equilibrium Slide is stolen (and modified) from Tim Roughgarden.

26. Application: Out-of-Equilibria Definition: a sequence s 1,s 2,...,s T of outcomes is no-regret if: for each player i, each fixed action q i : –average cost player i incurs over sequence no worse than playing action q i every time –simple hedging strategies can be used by players to enforce this (for suff large T) Theorem: in a (λ,μ)-smooth game, average cost of every no-regret sequence at most [λ/(1-μ)] x cost of optimal outcome. 26

27. 27 Why Important? pure Nash mixed Nash correlated eq no regret bound on no-regret sequences implies bound on inefficiency of mixed and correlated equilibria bound applies even to sequences that don’t converge in any sense no regret much weaker than reaching equilibrium

28. Today Reminder: –congestion games –Price of Anarchy (PoA) –Price of Stability (PoS) Congestion games with linear costs: –An upper bound on PoS –An upper bound on PoA PoA of real economic systems (or at least for their theoretical model…) –PoA in sponsored search. What is sponsored search? GSP Bounding the PoA in GSP

29. Keyword Auctions organic search results sponsored search links

30. Selling one Ad Slot $2 $5 $7 $3 Prospective advertisers

31. Selling one Ad Slot $2 $5 $7 $3 Pays $5 per click

32. Auction Model b1b1 b1b1 b2b2 b2b2 b3b3 b3b3 b4b4 b4b4 b5b5 b5b5 b6b6 b6b6 $$$ $$ $ $

33. Bidding 33 A basic campaign for an advertiser includes: Some keywords have bids greater than $50 –E.g., Mesothelioma Search engine provides assistance traffic estimator, keyword suggestions, auto bidding Pay per click List of : keywords + bid per click “hotel Las Vegas” $5 “Nikon camera d60” $30 Budget (for example, daily) I want to spend at most $500 a day

34. Bidding: more detailsmore details 34 When does a keyword match a user search-query? –When bidding $5 per “hotel California”. Will “hotel California song” appear? Broad match –California hotel, hotel California Hilton, cheap hotel California. Exact match: –“ hotel California” with no changes or additions. Negative words: –“ hotel California –song -eagles “ Many more options: –Geography, time, languages, mobile/desktops/laptops, etc.

35. Click Through rate 35 9% 4% 2% 0.5% 0.2% 0.08%

36. Click Through rate 36 α1α1 α2α2 α3α3 α4α4 … … αkαk

37. Example 37 v 1 =10 v 2 =8 v 3 =2 α 1 =0.08 α 2 =0.03 α 3 =0.01 Slot 1 Slot 2 Slot 3 The socially efficient outcome: Total efficiency: 10*0.08 + 8*0.03 + 2*0.01 The efficient allocation is called assortative matching.

38. How would you sell the slots? Yahoo! (that acquired Overture) sold ads in a pay- your-bid auction. Results: Sawtooth 38

39. Pay-your-bid data (14 hours) 39

40. Pay-your-bid data (week) 40

41. GSP 41 The Generalized Second price (GSP) auction –I like the name “next-price auction” better. Used by major search engines –Google, Bing (Microsoft), Yahoo –Multi-Billion dollars each quarter Auction rules (“next-price auction”) –Bidders bid their value per click b i –The ith highest bidder wins the ith slot and pays the (i+1)th highest bid.

42. Example 42 b 1 =10 b 2 =8 b 3 =2 α 1 =0.08 α 2 =0.03 α 3 =0.01 Slot 1 Slot 2 Slot 3 Pays $8 Pays $2 b 4 =1 Pays $1

43. Equilibrium concept 43 Analysis as a full-information game. b 2 =1b 2 =2b 3 =3…. b 1 =1 b 1 =2 b 1 =3 … Payoff are determined by the auction rules. Reason: equilibrium model “stable” bids in repeated- auction scenarios. (advertisers experiment…) Nash equilibrium: a set of bids in GSP where no bidder benefits from changing his bid (given the other bids).

44. (Simplified) Model α j : click-rate of slot j v i : value (per click) of player i b i : bid (declared value) (wlog, #slots=#advertisers) Assumption: b i ≤ v i Since playing b i > v i is dominated strategy. (“conservative bidders”) α1α1 α2α2 α3α3 b1b1 b1b1 b2b2 b2b2 b3b3 b3b3 v1v1 v2v2 v3v3 Slide is stolen from Renato Paes Leme and modified.

45. (Simplified) Model vivi αjαj bibi bibi j = σ(i)i = π(j) u i (b) = α σ(i) ( v i - b π(σ(i) + 1) ) Utility of player i : σ = π -1 Slide is stolen from Renato Paes Leme and modified.

46. Model vivi αjαj bibi bibi j = σ(i)i = π(j) σ = π -1 next highest bid u i (b) = α σ(i) ( v i - b π(σ(i) + 1) ) Utility of player i : Slide is stolen from Renato Paes Leme and modified.

47. Model vivi αjαj bibi bibi j = σ(i)i = π(j) Nash equilibrium: bids b σ = π -1 u i (b i,b -i ) ≥ u i (b’ i,b -i ) Is truth-telling always Nash ? Slide is stolen from Renato Paes Leme and modified.

48. Example Non-truthful α 1 = 1 α 2 = 0.9 b 1 = 2 v 1 = 2 v 2 = 1 b 1 = 1 b 1 = 0.9 u 1 = 1 (2-1)u 1 = 0.9(2-0) Slide is stolen from Renato Paes Leme and modified.

49. Measuring inefficiency vivi αjαj bibi bibi j = σ(i)i = π(j) σ = π -1 Social welfare = ∑ i v i α i ∑ i v i α σ(i) Optimal allocation = Slide is stolen from Renato Paes Leme and modified.

50. Measuring inefficiency Price of Anarchy = max = Opt SW(Nash) Nash We prove today: in GSP with conservative bidders, POA≤2 (One can work harder using similar techniques and show POA ≤ 1.282 ) Unlike congestion games, this is a maximization problem thus fraction is reversed. Still, smaller POA is better. Slide is stolen from Renato Paes Leme and modified.

51. Bounds on PoA We will prove next: In GSP, for all realization of values v and CTR’s α POA≤2 (One can work harder using similar techniques and show POA ≤ 1.282 ) Later in the course we will show PoS(GSP)=1 –If time allows… needs more machinery.

52. Proof (POA≤2) α2α2 α3α3 Opt α1α1 v1v1 v2v2 v3v3 αiαi α σ(j) vjvj v π(i) therefore: α σ(j) αiαi vjvj v π(i) ≥ 1 2 ≥ 1 2 or Intuition: social welfare is not hurt too much; either the values or the CTR are close to each other. α σ(j) αiαi vjvj v π(i) + ≥ 1 Claim: in a Nash equilibrium, the following holds for every i,j Slide is stolen from Renato Paes Leme and modified.

53. αiαi α σ(j) vjvj v π(i) α σ(j) αiαi vjvj v π(i) + ≥ 1 Assume σ(j)>i. (obvious otherwise) It is a combination of 3 relations: α σ(j) ( v j – b π(σ(j)+1) ) ≥ α i ( v j – b π(i) ) [ Nash ] b π(σ(j)+1) ≥ 0 b π(i) ≤ v π(i) [conservative] Sketch of the proof Proof of claim: Slide is stolen from Renato Paes Leme and modified. α σ(j) v j ≥ α i v j – α i v π(i) ) [ now divide by α i v j ] b π(σ(j)+1) b π(i)

54. Sketch of the proof α σ(j) αiαi vjvj v π(i) + ≥ 1 2 NashSW = ∑ i α σ(i) v i + α i v π(i) = v π(i) = ∑ i α i v i ≥ α σ(i) αiαi vivi + ≥ ∑ i α i v i = Opt Slide is stolen from Renato Paes Leme and modified.

55. Efficiency of GSP Conclusion: No matter how an equilibrium is reached, and which equilibrium is reached, as long we are in a steady state, we are in an approximately efficient state. A theoretical tool for analyzing a hard-to- experiment large system involving many different selfish agent. Of course, for a simplified model…

56. Summary Assuming that equilibria have been reached/computed by markets. Are they any good? –Selfish decisions (“anarchy”) lead to market failures? We saw environments where the PoA,PoS is small: equilibria are not far from optimal. –There are opposite examples. One major issue untouched so far: equilibria computation –So an equilibrium exists, and nearly optimal. Can it be computed in reasonable time by reasonable resources?

57. (Simplified) Model v1v1 v2v2 v3v3 α1α1 α2α2 α3α3 b1b1 b1b1 b2b2 b2b2 b3b3 b3b3 pays b 1 per click Slide is stolen from Renato Paes Leme and modified.