Price of Anarchy Georgios Piliouras

Games (i.e. Multi-Body Interactions) Interacting entities Pursuing their own goals Lack of centralized control Prediction?

Games n players Set of strategies S i for each player i Possible states (strategy profiles) S=×S i Utility u i :S→ R Social Welfare Q:S→ R Extend to allow probabilities Δ(S i ), Δ(S) u i (Δ(S))=E(u i (S)) Q(Δ(S))=E(Q(S)) (review)

Zero-Sum Games & Equilibria 0, 0-1, 11, -1 0, 0-1, 1 1, -10, 0 Rock Paper Scissors Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy. 1/3 Existence, Uniqueness of Payoffs [von Neumann 1928] (review)

General Games & Equilibria? 1, 0 -1, 11, -1 0, 0-1, 1 1, -10, 0 Rock Paper Scissors Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy. Borel conjectured the non-existence of eq. in general

Prediction in Games Idea 1 Nash Equilibrium (1950): A strategy tuple (i.e. one for each agent) s.t. no agent can deviate profitably. For finite games, it always exists when we allow agents to randomize Proof on the board

Equilibria & Prediction 0, 0-1, 11, -1 0, 0-1, 1 1, -10, 0 NE 0, 0-1, 11, -1 0, 0-1, 1 1, -10, 0

Implicit assumptions The players all will do their utmost to maximize their expected payoff as described by the game. The players are flawless in execution. The players have sufficient intelligence to deduce the solution. The players know (can compute) the planned equilibrium strategy of all of the other players. The players believe that a deviation in their own strategy will not cause deviations by any other players. There is common knowledge that all players meet these conditions, including this one. Uniqueness

Games & Equilibria 0, 0-1, 11, -1 0, 0-1, 1 1, -10, 0 Rock Paper Scissors Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy. 1/3

Equilibria & Prediction 20, 200, 1 1, 01, 1 Stag Hare Multiple Nash: Which one to choose?

Prediction in Games Idea 2a Koutsoupias and Papadimitriou (1999) If there exist several Nash Equilibria, then be pessimistic and output the worst possible one. (worst case analysis) Worst in terms of what? Social Welfare Examples of Social Welfare: Sum of utilities, maxmin utility, median utility

Metrics of Social Welfare Examples Sum of latencies (sec) maxmin utility ($) Throughput bottleneck (bit/sec)

Prediction in Games Idea 2b Koutsoupias and Papadimitriou (1999) If there exist several Nash Equilibria, then be pessimistic and output the worst possible one. (worst case analysis) Normalization Social Cost (worst Equilibrium) Social Cost (OPT) Price of Anarchy = ≥ 1

PoA = ≥ 1 Social Cost (worst Equilibrium) Social Cost (OPT) x 10 0 A B C D 10 agents A D PoA = 4/3 PoA ≤ 5/2, for all networks delay (x) = x [Koutouspias, Christodoulou 05] Price of Anarchy

Equilibria & Prediction 0, 0-1, 11, -1 0, 0-1, 1 1, -10, 0 NE 0, 0-1, 11, -1 0, 0-1, 1 1, -10, 0 NE PoA

Advantages of PoA Approach Simplicity Widely Applicable (conditions?) Allows for cross-domain comparisons (e.g. routing game vs facility location game) Analytically tractable? Several variants: Price of Stability, Price of Total Anarchy, Price of X,… YES, citations

BREAK Q: Any other ways to make predictions in multi-body problems? How do you do it in real life situations?

Recap + Plan Games + Worst Case Analysis + Normalization PoA = To do: – PoA Analysis (when welfare = sum utility) – Beyond Nash equilibria Social Cost (worst Equilibrium) Social Cost (OPT) PoA

Congestion Games n players and m resources (“edges”) Each strategy corresponds to a set of resources (“paths”) Each edge has a cost function c e (x) that determines the cost as a function on the # of players using it. Cost experienced by a player = sum of edge costs xxxx 2x xx Cost(red)=6 Cost(green)=8

Potential Games A potential game is a game that exhibits a function Φ : S→ R s.t. for every s ∈ S and every agent i, u i (s i,s -i ) - u i (s) = Φ (s i,s -i ) - Φ (s) Every congestion game is a potential game: Why? This implies that any such game has a pure NE. Why?

PoA ≤ 5/2 for linear latencies [Koutouspias, Christodoulou 05], [Roughgarden 09] Definition: A game is (λ,μ)-smooth if  i C i (s * i,s -i ) ≤ λcost(s * ) + μ cost(s) for all s,s * Then: POA (of pure Nash eq) ≤ λ/(1-μ) Proof: Let s arbitrary Nash eq. cost(s) =  i C i (s) [definition of social cost] ≤  i C i (s * i,s -i ) [s a Nash eq] ≤ λcost(s * ) + μ cost(s) [(λ,μ)-smooth]

PoA ≤ 5/2 for linear latencies [Koutouspias, Christodoulou 05], [Roughgarden 09] Technical lemma: A linear congestion game is (5/3,1/3)-smooth. Proof : Step 0: Matlab simulations to get a hint about what is the best possible (λ,μ) s.t. game is (λ,μ)-smooth. Step 1: Verify hypothesis (On the board).

Tight Example N agents, 2N elements (x 1, x 2,…, x N ) (y 1, y 2,…, y N ) c(x)=x for all of them Each agent i has 2 strategies : (x i,y i ) or (x i, y i-1, y i+1 ) … x1x1 … y1y1 xNxN yNyN x2x2 y2y2

BREAK 2 Q: What about PoA of mixed NE?

Recap + Plan Games + Worst Case Analysis + Normalization PoA = To do: – PoA Analysis (when welfare = sum utility) – Beyond Nash equilibria Social Cost (worst Equilibrium) Social Cost (OPT) PoA

0, 0-1, 11, -1 0, 0-1, 1 1, -10, 0 Rock PaperScissors Rock Paper Scissors 1/3 Other Equilibrium Notions Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy.

0, 0-1, 11, -1 0, 0-1, 1 1, -10, 0 Nash: A probability distribution over outcomes, that is a product of mixed strategies s.t. no player has a profitable deviating strategy. Choose any of the green outcomes uniformly (prob. 1/9) Rock PaperScissors Rock Paper Scissors 1/3 Other Equilibrium Notions

0, 0-1, 11, -1 0, 0-1, 1 1, -10, 0 Nash: A probability distribution over outcomes, s.t. no player has a profitable deviating strategy. Rock PaperScissors Rock Paper Scissors 1/3 Coarse Correlated Equilibria (CCE): Other Equilibrium Notions

A probability distribution over outcomes, s.t. no player has a profitable deviating strategy. Rock PaperScissors Rock Paper Scissors Coarse Correlated Equilibria (CCE): 0, 0-1, 11, -1 0, 0-1, 1 1, -10, 0 Other Equilibrium Notions

A probability distribution over outcomes, s.t. no player has a profitable deviating strategy. Rock PaperScissors Rock Paper Scissors Coarse Correlated Equilibria (CCE): 0, 0-1, 11, -1 0, 0-1, 1 1, -10, 0 Choose any of the green outcomes uniformly (prob. 1/6) Other Equilibrium Notions

A probability distribution over outcomes, s.t. no player has a profitable deviating strategy even if he can condition the advice from the dist.. Rock PaperScissors Rock Paper Scissors Correlated Equilibria (CE): 0, 0-1, 11, -1 0, 0-1, 1 1, -10, 0 Choose any of the green outcomes uniformly (prob. 1/6) Other Equilibrium Notions Is this a CE? NO

Other Equilibrium Notions Pure NE CE CCE

Smoothness bounds extend to CCE Definition: A game is (λ,μ)-smooth if  i C i (s * i,s -i ) ≤ λcost(s * ) + μ cost(s)for all s,s * Then: POA (of pure Nash eq) ≤ λ/(1-μ) Proof: Let s arbitrary Nash eq. cost(s) =  i C i (s) [definition of social cost] ≤  i C i (s * i,s -i ) [s a Nash eq] ≤ λcost(s * ) + μ cost(s) [(λ,μ)-smooth]

Smoothness bounds extend to CCE Definition: A game is (λ,μ)-smooth if  i C i (s * i,s -i ) ≤ λcost(s * ) + μ cost(s)for all s,s * Then: POA (of pure Nash eq) ≤ λ/(1-μ) Proof: Let s arbitrary CCE. E[cost(s)] = E[  i C i (s)] [definition of social cost] ≤ E[  i C i (s * i,s -i ) ] [s a CCE ] ≤ λ E[ cost(s * ) ] + μ E[ cost(s) ]

Criticism of PoA Analysis What happens in we add 10^10 to the utilities of each agent? Tightness is achieved over classes. Holds only for sum of utilities Sensitive to noise

Open Questions Choose your favorite class of games. Attempt (λ,μ)-smoothness analysis. – Possible problems Technique gives trivial upper bounds Still need to identify lower bounds What about uncertainty? Other (hidden) assumptions? [Balcan,Blum,Mansur’09] [Balcan,Constantin,Ehrlich‘11]

Recap Nash always exists (fixed point) but not unique PoA addresses non-uniqueness (λ,μ)-smoothness general technique for proving PoA bounds, extends to other notions, and can provide tight bounds

Thank You