1 Robust Price of Anarchy Bounds via Smoothness Arguments Tim Roughgarden Stanford University

2 Price of Anarchy Definition: price of anarchy (POA) of a game (w.r.t. some objective function, eq concept): optimal obj fn value equilibrium objective fn value the closer to 1 the better

3 Price of Anarchy Definition: price of anarchy (POA) of a game (w.r.t. some objective function, eq concept): optimal obj fn value equilibrium objective fn value the closer to 1 the better st 2x 12 5x 5 cost = = 24 cost = = 28 st 2x 12 5x 5 0 0

4 The Need for Robustness Meaning of a POA bound: if the game is at an equilibrium, then outcome is near-optimal.

5 The Need for Robustness Meaning of a POA bound: if the game is at an equilibrium, then outcome is near-optimal. Problem: what if cant reach equilibrium? (pure) equilibrium might not exist might be hard to compute, even centrally –[Fabrikant/Papadimitriou/Talwar], [Daskalakis/ Goldbeg/Papadimitriou], [Chen/Deng/Teng], etc. might be hard to learn in distributed way Worry: are our POA bounds meaningless?

6 Fix #1: Provable Convergence One Good Research Agenda: prove that natural learning dynamics converge (quickly) to equilibrium. [Hart/Mas-Collell], [Even-Dar/Kesselman/Mansour], [Fisher/ Racke/Voecking], [Chien/Sinclair], [Awerbuch et al], [Even- Dar/Mansour/Nadav], [Kleinberg/Piliouras/Tardos], etc. Problem: this best-case scenario has limited reach non-existence/complexity issues lower bounds on convergence time of natural dynamics (e.g., [Skopalik/Voecking STOC 08])

7 Fix #2: Robust POA Bounds High-Level Goal: worst-case bounds that apply even to non-equilibrium outcomes! best-response dynamics, pre-convergence –[Mirrokni/Vetta 04], [Goemans/Mirrokni/Vetta 05], [Awerbuch/Azar/Epstein/Mirrokni/Skopalik 08] correlated equilibria –[Christodoulou/Koutsoupias 05] coarse correlated equilibria aka price of total anarchy aka no-regret players –[Blum/Even-Dar/Ligett 06], [Blum/Hajiaghayi/Ligett/Roth 08]

8 Abstract Setup n players, each picks a strategy s i player i incurs a cost C i (s) Important Assumption: objective function is cost(s) := i C i (s) Key Definition: A game is (λ,μ)-smooth if, for every pair s,s * outcomes (λ > 0; μ < 1): i C i (s * i,s -i ) λcost(s * ) + μcost(s) [(*)]

9 Smooth => POA Bound Next: canonical way to upper bound POA (via a smoothness argument). notation: s = a Nash eq; s * = optimal Assuming (λ,μ)-smooth: cost(s) = i C i (s) [defn of cost] i C i (s * i,s -i ) [s a Nash eq] λcost(s * ) + μcost(s) [(*)] Then: POA (of pure Nash eq) λ/(1-μ).

10 Why Is Smoothness Stronger? Key point: to derive POA bound, only needed i C i (s * i,s -i ) λcost(s * ) + μcost(s) [(*)] to hold in special case where s = a Nash eq and s * = optimal. Smoothness: requires (*) for every pair s,s * outcomes. –even if s is not a pure Nash equilibrium

11 Some Smoothness Bounds atomic (unweighted) selfish routing [Awerbuch/Azar/Epstein 05], [Christodoulou/Koutsoupias 05], [Aland/Dumrauf/Gairing/Monien/Schoppmann 06], [Roughgarden 09] nonatomic selfish routing [Roughgarden/Tardos 00],[Perakis 04] [Correa/Schulz/Stier Moses 05] weighted congestion games [Aland/Dumrauf/Gairing/Monien/Schoppmann 06], [Bhawalkar/Gairing/Roughgarden 10] submodular maximization games [Vetta 02], [Marden/Roughgarden 10] coordination mechanisms [Cole/Gkatzelis/Mirrokni 10]

Application: Out-of-Equilibria Definition: a sequence s 1,s 2,...,s T of outcomes is no-regret if: for each player i, each fixed action q i : –average cost player i incurs over sequence no worse than playing action q i every time –simple hedging strategies can be used by players to enforce this (for suff large T) Theorem: [Roughgarden STOC 09] in a (λ,μ)- smooth game, average cost of every no- regret sequence at most [λ/(1-μ)] x cost of optimal outcome. 12

13 Smooth => POTA Bound notation: s 1,s 2,...,s T = no regret; s * = optimal Assuming (λ,μ)-smooth: t cost(s t ) = t i C i (s t ) [defn of cost]

14 Smooth => POTA Bound notation: s 1,s 2,...,s T = no regret; s * = optimal Assuming (λ,μ)-smooth: t cost(s t ) = t i C i (s t ) [defn of cost] = t i [C i (s * i,s t -i ) + i,t ] [ i,t := C i (s t )- C i (s * i,s t -i )]

15 Smooth => POTA Bound notation: s 1,s 2,...,s T = no regret; s * = optimal Assuming (λ,μ)-smooth: t cost(s t ) = t i C i (s t ) [defn of cost] = t i [C i (s * i,s t -i ) + i,t ] [ i,t := C i (s t )- C i (s * i,s t -i )] t [λcost(s * ) + μcost(s t )] + i t i,t [(*)]

16 Smooth => POTA Bound notation: s 1,s 2,...,s T = no regret; s * = optimal Assuming (λ,μ)-smooth: t cost(s t ) = t i C i (s t ) [defn of cost] = t i [C i (s * i,s t -i ) + i,t ] [ i,t := C i (s t )- C i (s * i,s t -i )] t [λcost(s * ) + μcost(s t )] + i t i,t [(*)] No regret: t i,t 0 for each i. To finish proof: divide through by T.

17 Why Important? pure Nash mixed Nash correlated eq no regret bound on no-regret sequences implies bound on inefficiency of mixed and correlated equilibria bound applies even to sequences that dont converge in any sense no regret much weaker than reaching equilibrium e.g., if every player uses multiplicative weights then get o(1) regret in poly-time

Tight Game Classes Theorem: [Roughgarden STOC 09] for every set C, unweighted congestion games with cost functions restricted to C are tight: maximum [pure POA] = minimum [λ/(1-μ)] congestion games w/cost functions in C (λ,μ): all such games are (λ,μ)-smooth 18

Tight Game Classes Theorem: [Roughgarden STOC 09] for every set C, unweighted congestion games with cost functions restricted to C are tight: maximum [pure POA] = minimum [λ/(1-μ)] weighted congestion games [Bhawalkar/ Gairing/Roughgarden ESA 10] and submodular maximization games [Marden/Roughgarden CDC 10] are also tight in this sense congestion games w/cost functions in C (λ,μ): all such games are (λ,μ)-smooth 19

20 Further Applications pure Nash mixed Nash correlated eq no regret best- response dynamics approximate Nash Theorem: in a (λ,μ)-smooth game, everything in these sets costs (essentially) λ/(1-μ) x OPT.

21 More Applications? Theorem: [Nadav/Roughgarden 10] Consider a (λ,μ)-smooth game for optimal choices of λ,μ. Then precisely the aggregate coarse correlated equilibria have cost λ/(1-μ) x OPT. essentially, bound holds if and only if the average (rather than per-player) regret is non-positive Proof: convex duality.

When Is a POA Bound a Smoothness Proof? Need to show: for every pair s,s * outcomes: i C i (s * i,s -i ) λcost(s * ) + μcost(s) Generally sufficient: prove POA bound for pure Nash equilibria such that: invoke best-response condition once/player hypothetical deviation by i independent of s -i Non-example: network creation games [Fabrikant et al], [Albers et al], [Demaine et al], etc. 22

Application: POA of Non- Truthful Mechanisms Mechanism Design: design protocol with desirable outcome despite selfish participants example: Vickrey (second-price) auction focus thus far on "truthful" mechanisms Non-truthful mechanisms: motivated by simplicity (e.g., sponsored search auctions) low communication complexity low computational complexity 23

Application: POA of Non- Truthful Mechanisms Fact: plausible outcomes of non-truthful mechanisms = Bayes-Nash equilibria POA results for simple non-truthful auctions in [Christodoulou/Kovacs/Schapira ICALP 08] and [Borodin/Lucier SODA 10] first bound POA of Nash equilibria, then "by magic" same bound holds for Bayes-Nash Fact: these are automatic consequences of smoothness. 24

POA of Non-Truthful Mechanisms (continued) [Paes Leme/Tardos FOCS 10] POA upper bounds for welfare of Generalized Sponsored Search auction different bounds for pure (1.618), mixed (4), and Bayes-Nash eq (8) (without smoothness) Open Questions: compute best smoothness bound prove a lower bound separating the best- possible pure vs. Bayes-Nash POA 25

POA of Non-Truthful Mechanisms (continued) [Bhawalkar/Roughgarden 10] POA upper bounds for welfare of subadditive combinatorial auctions with item bidding –generalizes [Christodoulou/Kovacs/Schapira 08] pure Nash: non-smooth upper bound of 2 Bayes-Nash: lower bound of 2.01, upper bound of 2 ln m [m = # goods] Open Questions: is Bayes-Nash POA = O(1)? 26

Local Smoothness and Splittable Congestion Games Local smoothness: require smoothness condition only for outcomes that are close –assume strategy sets = convex subset of R n can only decrease optimal value of λ/(1-μ) [Harks 08]: local smoothness gives improved POA bounds for splittable congestion games [Roughgarden/Schoppmann 10]: matching lower bounds => first tight bounds in this model [RS10]: local smoothness bounds extend to correlated eq but not to no-regret outcomes! 27

Splittable Congestion Games: Open Questions Open Question #1: determine optimal POA bounds for no-regret sequences. Open Question #2: is (the distribution of) every no-regret sequence a convex combination of pure Nash equilibria? Open Question #3: prove something non-trivial about the POA of symmetric splittable congestion games. 28