Inefficiency of Equilibria: POA, POS, SPOA Based on Slides by Amir Epstein and by Svetlana Olonetsky Modified/Corrupted by Michal Feldman and Amos Fiat

Inefficiency of equilibria Outcome of rational behavior might be inefficient How to measure inefficiency? –E.g., prisoner’s dilemma Define an objective function –Social welfare (= sum of players’ payoffs): utilitarian –Maximize min i u i (egalitarian) –… 0,53,3 1,15,0

Inefficiency of equilibria To measure inefficiency we need to specify: –Objective function –Definition of approximately optimal –Definition of an equilibrium –If multiple equilibria exist, which one do we consider?

4 More Equilibrium Concepts pure Nash mixed Nash correlated eq ??? no regret ??? best- response dynamics Strong Nash

Common measures Price of anarchy (POA)=cost of worst NE / cost of OPT Price of stability (POS)=cost of best NE / cost of OPT Approximation ratio: Measures price of limited computational resources Competitive ratio: Measures price of not knowing future Price of anarchy: Measures price of lack of coordination Pure /Mixed /Strong Corr POA ≤ Pure POA ≤ Mixed POA ≤ Strong POA Corr POS ≥ Pure POS ≥ Mixed POA ≥ Strong POA

Price of anarchy Example: in prisoner’s dilemma, POA = POA = 3 –But can be as large as desired Wish to find games in which POS or POA are bounded –NE “approximates” OPT –Might explains Internet efficiency. Suppose we define POA and POS w.r.t. NE in pure strategies –we first need to prove existence of pure NE 0,53,3 1,15,0 Prisoner’s dilemma

Max-cut game Given undirected graph G = (V,E) players are nodes v in V An edge (u,v) means u “hates” v (and vice versa) Strategy of node i: s i  {Black,White} Utility of node i: # neighbors of different color Lemma: for every graph G, corresponding game has a pure NE

Proof 1 Claim: OPT of max-cut defines a NE Proof: –Define strategies of players by cut (i.e., one side is Black, other side is White) –Suppose a player i wishes to switch strategies: i’s benefit from switching = improvement in value of the cut –Contradicting optimality of cut u i =1 u i =2

Proof 2 Algorithm greedy-find-cut (GFC): –Start with arbitrary partition of nodes into two sets –If exists node with more neighbors in other side, move it to other side (repeat until no such node exists) Claim 1: GFC provides 2-approx. to max-cut, and runs in polynomial time Proof: –Poly time: GFC terminates within at most |E| steps (since every step improves the value of the solution in at least 1, and |E| is a trivial upper bound to solution) –2-approx.: Each node ends up with more neighbors in other side than in own side, so at least |E|/2 edges are in cut (since #edges in cut > #edges not in cut)

Proof 2 (cont’d) Claim 2: cut obtained by GFC defines a NE Proof: obvious, as each player stops only if his strategy is the best response to the other players’ strategies Conclusion: max-cut game admits a NE in pure strategies

11 Wardrop Equilibria: Traffic Flow: the Mathematical Model a directed graph G = (V,E) k source-destination pairs (s 1,t 1 ), …, (s k,t k ) a rate (amount) r i of traffic from s i to t i for each edge e, a cost function c e () s1s1 t1t1 c(x)=x Flow = ½ c(x)=1 Example: (k,r=1)

12 Routings of Traffic Traffic and Flows: f P = amount of traffic routed on s i -t i path P flow vector f routing of traffic Selfish routing: what are the equilibria? st

13 Wardrop Flows Special case, assumptions: agents small relative to network (nonatomic game) want to minimize cost of their path Def: A flow is at [Pure] Nash equilibrium (or is a Nash flow) if all flow is routed on min-cost paths [given current edge congestion] x st 1 Flow =.5 st 1 Flow = 0 Flow = 1 x Example:

14 History + Generalizations model, defn of Nash flows by [Wardrop 52] Nash flows exist, are (essentially) unique –due to [Beckmann et al. 56] –general nonatomic games: [Schmeidler 73] congestion game (payoffs fn of # of players) –defined for atomic games by [Rosenthal 73] –previous focus: Nash eq in pure strategies exist potential game (equilibria as optima) –defined by [Monderer/Shapley 96]

15 The Cost of a Flow Def: the cost C(f) of flow f = sum of all costs incurred by traffic (avg cost × traffic rate) st x 1 ½ ½ Cost = ½½ +½1 = ¾

16 The Cost of a Flow Def: the cost C(f) of flow f = sum of all costs incurred by traffic (avg cost × traffic rate) Formally: if c P (f) = sum of costs of edges of P (w.r.t. the flow f), then: C(f) =  P f P c P (f) st st x 1 ½ ½ Cost = ½½ +½1 = ¾

17 Inefficiency of Nash Flows Note: Nash flows do not minimize the cost observed informally by [Pigou 1920] Cost of Nash flow = = 1 Cost of optimal (min-cost) flow = ½½ +½1 = ¾ Price of anarchy := Nash/OPT ratio = 4/3 st x ½ ½

18 Braess’s Paradox Initial Network: st x1 ½ x 1 ½ ½ ½ cost = 1.5

19 Braess’s Paradox Initial Network: Augmented Network: st x1 ½ x 1 ½ ½ ½ cost = 1.5 st x1 ½ x 1 ½ ½ ½ 0 Now what?

20 Braess’s Paradox Initial Network: Augmented Network: st x1 ½ x 1 ½ ½ ½ cost = 1.5 cost = 2 st x 1 x 1 0

21 Braess’s Paradox Initial Network: Augmented Network: All traffic incurs more cost! [Braess 68] see also [Cohen/Horowitz 91], [Roughgarden 01] st x1 ½ x 1 ½ ½ ½ cost = 1.5 cost = 2 st x 1 x 1 0

Special Case of routing: Equal Machine Load Balancing = Parallel Links Two nodes m parallel (related) links n jobs (communication requests) User cost (delay) is proportional to link load Global cost (maximum delay) is the maximum link load

Price of Anarchy Price of Anarchy: The worst possible ratio between: -Objective function in Nash Equilibrium and -Optimal Objective function Objective function: total user cost, total user utility, maximal/minimal cost, utility, etc., etc.

Identical machines Main results (objective function – maximum load) -For m identical links, identical jobs (pure) R=1 -For m identical links (pure) R=2-1/(m+1) -For m identical links (mixed) Lower bound – easy : uniformly choose machine with prob. 1/m Upper bound – assume opt = 1, opt = max expected ≤ 2 in NE (otherwise not NE, NE = expected max ≤ log m / loglog m due to Hoeffding concentration inequality

Related Work (Cont’) Main results -For 2 related links R= For m related links (pure) -For m related links (mixed) -For m links restricted assignment (pure) -For m links restricted assignment (mixed)

2 2 1 Identical machines Highest load machine (#1), lowest weight job on #1 (1) Lowest weight job on highest load machine ≤ ½ HL (3) Every other machine has load ≥ ½ HL 21.5

m (=3) machines n (=4) jobs v i – speed of machine i w j – weight of job j v 1 = 4 v 2 = 2 v 3 = 1 1 (4) 2 (4) 2 (2) 1 (2) L 1 = 1 L 2 = 3 L 3 = 2 Related machines L i – load on machine i

Price of Anarchy: Lower Bound k! / (k-i)! GiGi k-i k! 1 k GkGk k k-1 k(k-1) k-2 G0G0 G1G1 G2G2 v=2 k-i v=1 v=2 k w=2 k-i w=2 k v=2 w=2 v=2 k-1 w=2 k-1

Price of Anarchy: Lower Bound GiGi k-i k! 1 k GkGk k k-1 k(k-1) k-2 G0G0 G1G1 G2G2 k! ~ m k ~ log m / log log m k! / (k-i)! v=2 k-i v=1 v=2 k w=2 k-i w=2 k v=2 w=2 v=2 k-1 w=2 k-1

1 1 Its a Nash Equilibrium GiGi k-i k! 1 k GkGk k k-1 k(k-1) k-2 G0G0 G1G1 G2G2 k! / (k-i)! 2 v=2 k-i v=1 v=2 k w=2 k-i w=2 k v=2 w=2 v=2 k-1 w=2 k-1

1 Its a Nash Equilibrium GiGi k-i k! 1 k GkGk k k-1 k(k-1) k-2 G0G0 G1G1 G2G2 k! / (k-i)! 2 4 v=2 k-i v=1 v=2 k w=2 k-i w=2 k v=2 w=2 v=2 k-1 w=2 k-1

1 The social optimum k! / (k-i)! GiGi k-i k! 1 k GkGk k k-1 k(k-1) k-2 G0G0 G1G1 G2G2 2 1 v=2 k-i v=1 v=2 k w=2 k-i w=2 k v=2 w=2 v=2 k-1 w=2 k-1

The social optimum k! / (k-i)! GiGi k-i k! 1 k k k-1 k(k-1) k-2 G0G0 G1G1 G2G2 2 v=2 k-i v=1 v=2 k w=2 k-i w=2 k v=2 w=2 v=2 k-1 w=2 k-1 1 GkGk

1 2 The social optimum k! / (k-i)! GiGi k-i k! 1 k k k-1 k(k-1) k-2 G0G0 G1G1 G2G v=2 k-i v=1 v=2 k w=2 k-i w=2 k v=2 w=2 v=2 k-1 w=2 k-1 GkGk

Related Machines: Price of Anarchy upper bound Normalize so that Opt = 1 Sort machines by speed The fastest machine (#1) has load Z, no machine has load greater than Z+1 (otherwise some job would jump to machine #1) We want to give an upper bound on Z

Related Machines: Price of Anarchy upper bound Normalize so that Opt = 1 The fastest machine (#1) has load Z, but Opt is 1, consider all the machines that Opt uses to run these jobs. These machines must have load ≥ Z-1 (otherwise job would jump from #1 to this machine) There must be at least Z such machines, as they need to do work ≥ Z

Related Machines: Price of Anarchy upper bound Take the set of all machines up to the last machine that opt uses to service the jobs on machine #1. The jobs on this set of machines have to use Z(Z-1) other machines under opt. Continue, the bottom line is that n ≥ Z!, or that Z ≤ log m / log log m

Restricted Assignment to Machines m0m0 m0m0 m0m0 m0m0 m0m0 m1m1 m0m0 m1m1 m1m1 m1m1 m1m1 m1m1 m2m2 m2m2 m2m2 m3m3 NASH Group 1 m0m0 m0m0 m0m0 m0m0 m0m0 m1m1 m0m0 m1m1 m1m1 m1m1 m1m1 m1m1 m2m2 m2m2 m2m2 m3m3 Group 2Group 3 Group 1 Group 2 Group 3 OPT l=3

Restricted Assignment to Machines m0m0 m0m0 m0m0 m0m0 m0m0 m1m1 m0m0 m1m1 m1m1 m1m1 m1m1 m1m1 m2m2 m2m2 m2m2 m3m3 NASH Group 1 m0m0 m0m0 m0m0 m0m0 m0m0 m1m1 m0m0 m1m1 m1m1 m1m1 m1m1 m1m1 m2m2 m2m2 m2m2 m3m3 Group 2Group 3 Group 1 Group 2 Group 3 OPT l=3

Price of anarchy for unrelated machines POA for unrelated machines is unbounded  1 1  Job 1 Job 2 Machine 1Machine 2 11 Machine 1 Machine 2  Machine 1 Social optimumNash equilibrium makespan=  makespan=  PoA=1/ 

Allowing Coordination in Equilibrium Strong Equilibrium [Aumann’59] –No coalition can deviate and strictly improve the utility of all of its members very robust concept may be a better prediction of rational behavior most games do not admit Strong Eq. –usually applied to pure Eq with pure deviations

Example 1: Prisoner’s Dilemma  0,5 5,0  cooperate defect Unique Nash Eq. Strong Eq. ? Prisoner’s dilemma does not admit any Strong Eq.

Strong Price of Anarchy Determining SPoA requires two parts: –Proving existence of Strong Eq –Bounding the worst ratio SE  NE  SPoA ≤ PoA Price of Anarchy (PoA) [KP00]: Strong Price of Anarchy (SPoA):

k-Strong Equilibrium A joint action s  S is not resilient to a pure deviation of a coalition  if there is a pure action profile  of  such that c i (s - ,  )<c i (s) for any i   –e.g., (defect,defect) in Prisoner’s dilemma A pure Nash Eq s  S is resilient to pure deviation of coalitions of size k if there is no coalition  of size at most k such that s is not resilient to a pure deviation by  A k-Strong Equilibrium is a pure Nash Eq that is resilient to pure deviation of coalitions of size at most k S=S 1 x…xS n

Strong Equilibrium Hierarchy 1-SE 2-SE n-SE = NE =SE [Aumann]

Related Work Existence of Strong Equilibrium –monotone decreasing congestion games [Holzman+Lev- tov 1997, 2003] –monotone increasing congestion games + correlated SE [Rosenfeld+Tennenholtz 2006] Related solution concepts –Coalition-proof Eq. [Bernheim 1987] –Group-strategyproof mechanisms [Moulin+Shenker 2001] –Coalitions with transferable utilities [Hayrapetyan et al 2006] SE CPE NE

Existence of Strong Equilibrium in load balancing games Is every Nash Eq. on identical machines also a Strong Eq ? –NO ! (for m ≥ 3) s s’ Coalition: 5,5,3,3

Strong Eq. Existence Theorem: in any load balancing game, the lex. minimal joint action s is a k-SE for any k

Recall Lexicographic Order Definition: a vector (l 1,…l m ) is smaller than (l 1 ’,…,l m ’) lexicographically if for some i, l i < l i ’ and l k = l k ’ for all k<I Definition: A joint action s is smaller than s’ lex. (s  s’) if the vector of machine loads L(s), sorted in non-decreasing order, is smaller lex. than L(s’) s s’ s  s’

Proof of SE existence Lemma: suppose L(s) and L(s’) differ only in the loads of machines in a set M’  M. if for each M i  M’, L i (s) < max k {L k (s’) | M k  M’}, then s  s’

Proof of SE Existence Suppose in contradiction that s (lex. minimal) is not a SE, and let  be the smallest coalition (deviating to s’). Claim: the same set of machines are chosen by  in s and in s’ (denote it M(  )) –If a job migrates TO some machine, another job migrates FROM it else contradicting s is NE –If a job migrates FROM some machine, another job migrates TO it else contradicting minimality of  Since all jobs in  must benefit, all loads of M(  ) in s’ must be smaller than max load of M(  ) in s –Contradicting minimality of s

Price of Anarchy (PoA) Recall: for unrelated machines, PoA may be unbounded  1 1  Job 1 Job 2 Machine 1Machine 2 Objective: min makespan Social optimum Nash equilibrium PoA=1/  11 M1M1 makespan=  M2M2  makespan=  M1M1 M2M2 Strong equilibrium  SPoA=1

Strong Price of Anarchy Theorem: for any job scheduling game with m unrelated machines and n jobs, SPoA ≤ m

Proof for SpoA ≤ m Claim 1: L 1 (s) ≤ OPT –else: coalition of all jobs to OPT M1M1 MmMm MiMi M i-1 M1M1 MmMm MiMi OPT L 1 (s) OPT L 1 (s)

Proof for SpoA ≤ m Claim 1: L 1 (s) ≤ OPT –else: coalition of all jobs to OPT Claim 2:  i L i (s)-L i-1 (s) ≤ OPT –else: consider s’, where all jobs on machines i..m go to OPT. For all J   c J (s) > L i-1 (s) + OPT c J (s’) ≤ L i-1 (s) + OPT (since all J   together add at most OPT) M1M1 MmMm MiMi M i-1 M1M1 MmMm MiMi > OPT OPT L m (s) ≤ m OPT L i-1 (s)L 1 (s) L i (s)

Lower Bound (m machines) Theorem: there exists a job scheduling game with m unrelated machines for which SPoA ≥ m Proof : M1M1 M2M2 M3M3 M4M4 MmMm J1J1 11 J2J2 12 J3J3 13 J4J4 14 JmJm 1m   OPT = 1 makespan=m SE

Identical Machines Theorem: there exists a job scheduling game with m identical machines and n jobs, such that 12m-1m J1J1 JmJm J m+1 J 2m 1 1/m 1 m-2 m-1m OPT SE 1+1/m 2

Results - machines Objective function – maximum load -For m identical links, identical jobs (pure) R=1 -For m identical links (pure) POA= SPOA = 2-1/(m+1), -For m related links (pure) -For m links restricted assignment (pure) -For m unrelated machines,

Homework assignment Fill in tables: –(pure POA, pure SPOA, pure POS, pure SPOS) * (identical, related, unrelated) –(mixed POA, mixed SPOA, mixed POS, mixed SPOS) * (identical, related, unrelated) Why? Send me with pptx presentation