1. Load Balancing, Multicast routing, Price of Anarchy and Strong Equilibrium Computational game theory Spring 2008 Michal Feldman

2. Load Balancing: Model Set of machines M = {M 1,…,M m } Set of jobs N = {1,…,n} Weight of job J  N on machine M i  M : w i (J) Action space of job J: S J = M (job J selects machine s J ) B i s = {J | s J = M i } set of players on machine M i L i (s) =  J  Bis w i (J) Load player J observes: c J (s) = L i (s), where s J = M i Objective function: makespan (OPT = min s makespan(s))

3. Load Balancing Model: Unrelated Machines Set of machines M = {M 1,…,M m } Set of jobs N = {1,…,n} Unrelated machines model: Job (player) i has load w ij on machine j Strategy: select a machine Cost of a job = total load on selected machine Objective: minimize makespan (max load) Special cases: – Identical machines: w ij =w ij’ for all j,j’ – Related machines: each machine j has a speed s j, and each job i has load l i, and w ij =l i /s j M1M1 M2M2 J1J1 57 J2J2 23 J3J3 41 5 4 3 L 1 (s)=9 M1M1 M2M2 L 2 (s)=3 jobs machines

4. (pure) equilibrium existence Potential function – Identical machines: sum of squares (why?) – Unrelated machines: Does sum of squares work? No ! Before migration: 10, after migration: 9, so cost decreased Yet, sum of squares increased from 10 2 +5 2 to 9 2 +9 2 10 5 1 4

5. Lexicographic order Definition: a vector (l 1,…l m ) is smaller than (l 1 ’,…,l m ’) lexicographically if for some i, l i < l i ’ and l k = l k ’ for all k<I Definition: A joint action s is smaller than s’ lex. (s  s’) if the vector of machine loads L(s), sorted in non- decreasing order, is smaller lex. than L(s’) s s’ s  s’

6. (Pure) NE Existence Lemma: if a job i improves its cost by migration, then the lexicographic order decreases Proof sketch: – a job migrating from blue machine to red machine – Only the load on these two machines change (blue decreases, red increases) – But if the migrating job improves, red (in post-migration) must be lower than blue (in pre-migration) – Thus after migration, both blue and red are lower than blue prior to migration – Thus profile decreases lexicographically Conclusion 1: load balancing game admit a Nash equilibrium in pure strategies Conclusion 2: price of stability of any load balancing game is 1

7. Price of Anarchy for identical machines Theorem: in any load balancing game on identical machines, it holds that Proof: – Let s be a NE and let s* be OPT – Let i’ be a machine with highest cost in s, and let j’ be job with lowest weight on machine i’ – wlog, at least 2 jobs on machine i’ (why?), thus w j’≤ ½ cost(s) – Since s is a NE, for any machine i≠I’ (job j’ stays) l i ≥ l i’ – w j’ ≥ cost(s) – ½ cost(s) = ½ cost(s)

8. Convergence time of best response for identical machines Max-weight best response policy: – activate jobs, always activating job of max-weight among unsatisfied jobs – activated job migrates to its best machines (i.e., performs a best-response) Theorem: for any load balancing game on identical machines, the max-weight best response policy converges to a NE, after each agent was activated at most once (from any initial profile)

9. Convergence time of best response for identical machines Proof sketch: – Claim: once a job was activated, it never gets unsatisfied again – Proof of claim is based on two observations (for identical machines): Job is satisfied IFF assigned to machine with minimal load (other than itself) Best response never decreases the minimal load among the machines (why?) – Thus, a job can become unsatisfied only if another job migrated to its own machine – Thus, sufficient to show that a migration of a job of lower weight into one’s machine cannot make it unsatisfied – Proof in class.. Note: order is crucial. Under “min-weight best response policy”, there may be instances with an exponential number of steps

10. Price of anarchy for unrelated machines POA for unrelated machines is unbounded  1 1  Job 1 Job 2 Machine 1Machine 2 11 Machine 1 Machine 2  Machine 1 Social optimumNash equilibrium makespan=  makespan=  PoA=1/ 

11. Allowing Coordination in Equilibrium Strong Equilibrium [Aumann’59] – No coalition can deviate and strictly improve the utility of all of its members very robust concept may be a better prediction of rational behavior most games do not admit Strong Eq. – usually applied to pure Eq with pure deviations

12. Example 1: Prisoner’s Dilemma  0,5 5,0  cooperate defect Unique Nash Eq. Strong Eq. ? Prisoner’s dilemma does not admit any Strong Eq.

13. Strong Price of Anarchy Determining SPoA requires two parts: – Proving existence of Strong Eq – Bounding the worst ratio SE  NE  SPoA ≤ PoA Price of Anarchy (PoA) [KP00]: Strong Price of Anarchy (SPoA):

14. k-Strong Equilibrium A joint action s  S is not resilient to a pure deviation of a coalition  if there is a pure action profile  of  such that c i (s - ,  )<c i (s) for any i   – e.g., (defect,defect) in Prisoner’s dilemma A pure Nash Eq s  S is resilient to pure deviation of coalitions of size k if there is no coalition  of size at most k such that s is not resilient to a pure deviation by  A k-Strong Equilibrium is a pure Nash Eq that is resilient to pure deviation of coalitions of size at most k S=S 1 x…xS n

15. Strong Equilibrium Hierarchy 1-SE 2-SE n-SE = NE =SE [Aumann]

16. Related Work Existence of Strong Equilibrium – monotone decreasing congestion games [Holzman+Lev-tov 1997, 2003] – monotone increasing congestion games + correlated SE [Rosenfeld+Tennenholtz 2006] Related solution concepts – Coalition-proof Eq. [Bernheim 1987] – Group-strategyproof mechanisms [Moulin+Shenker 2001] – Coalitions with transferable utilities [Hayrapetyan et al 2006] SE CPE NE

17. Existence of Strong Equilibrium in load balancing games Is every Nash Eq. on identical machines also a Strong Eq ? – NO ! (for m ≥ 3) 5 5 44 33 1077 s 55 44 3 3 69 9 s’ Coalition: 5,5,3,3

18. Strong Eq. Existence Theorem: in any load balancing game, the lex. minimal joint action s is a k-SE for any k

19. Recall Lexicographic Order Definition: a vector (l 1,…l m ) is smaller than (l 1 ’,…,l m ’) lexicographically if for some i, l i < l i ’ and l k = l k ’ for all k<I Definition: A joint action s is smaller than s’ lex. (s  s’) if the vector of machine loads L(s), sorted in non- decreasing order, is smaller lex. than L(s’) s s’ s  s’

20. Proof of SE existence Lemma: suppose L(s) and L(s’) differ only in the loads of machines in a set M’  M. if for each M i  M’, L i (s) < max k {L k (s’) | M k  M’}, then s  s’

21. Proof of SE existence Recall: s is a NE Suppose in contradiction that s is not a SE Let  be the smallest coalition that can deviate (any job in  deviates) Let M( ,s) be the set of machines that the coalition  chooses in s – For every M i  M( ,s) that a job wishes to migrate to, another job wishes to migrate from (otherwise, contradicting s NE) – For every M i  M( ,s) that a job wishes to migrate from, another job wishes to migrate to (otherwise, contradicting  minimal) – Therefore, only machines in M’=M( ,s)=M( ,s’) change their loads Since each job benefits from the deviation, the new load on each machine M i  M’ must be < max k {L k (s) | M k  M’ } By previous lemma, s’  s, contradicting minimality of s

22. Proof of SE Existence Suppose in contradiction that s (lex. minimal) is not a SE, and let  be the smallest coalition (deviating to s’). Claim: the same set of machines are chosen by  in s and in s’ (denote it M(  )) – If a job migrates TO some machine, another job migrates FROM it else contradicting s is NE – If a job migrates FROM some machine, another job migrates TO it else contradicting minimality of  Since all jobs in  must benefit, all loads of M(  ) in s’ must be smaller than max load of M(  ) in s – Contradicting minimality of s

23. Price of Anarchy (PoA) Recall: for unrelated machines, PoA may be unbounded  1 1  Job 1 Job 2 Machine 1Machine 2 Objective: min makespan Social optimum Nash equilibrium PoA=1/  11 M1M1 makespan=  M2M2  makespan=  M1M1 M2M2 Strong equilibrium  SPoA=1

24. Strong Price of Anarchy Theorem: for any job scheduling game with m unrelated machines and n jobs, SPoA ≤ m

25. Proof for SpoA ≤ m Claim 1: L 1 (s) ≤ OPT – else: coalition of all jobs to OPT M1M1 MmMm MiMi M i-1 M1M1 MmMm MiMi OPT L 1 (s) OPT L 1 (s)

26. Proof for SpoA ≤ m Claim 1: L 1 (s) ≤ OPT – else: coalition of all jobs to OPT Claim 2:  i L i (s)-L i-1 (s) ≤ OPT – else: consider s’, where all jobs on machines i..m go to OPT. For all J   c J (s) > L i-1 (s) + OPT c J (s’) ≤ L i-1 (s) + OPT (since all J   together add at most OPT) M1M1 MmMm MiMi M i-1 M1M1 MmMm MiMi > OPT OPT L m (s) ≤ m OPT L i-1 (s)L 1 (s) L i (s)

27. Lower Bound (m machines) Theorem: there exists a job scheduling game with m unrelated machines for which SPoA ≥ m Proof : M1M1 M2M2 M3M3 M4M4 MmMm J1J1 11 J2J2 12 J3J3 13 J4J4 14 JmJm 1m   OPT = 1 makespan=m SE

28. Identical Machines Theorem: there exists a job scheduling game with m identical machines and n jobs, such that 12m-1m J1J1 JmJm J m+1 J 2m 1 1/m 1 m-2 m-1m OPT SE 1+1/m 2

29. 2 Machines: Strong Price of Anarchy Let s be a Strong Eq. If for every job J on M 2 : w 1 (J) ≥ w 2 (J) L 2 ≤ 2 OPT  OPT ≥  J w min (J) / m Otherwise let J on M 2 : w 1 (J) ≤ w 2 (J) L 2 ≤ L 1 + OPT  OPT ≥ max J w min (J) L1L1 L2L2 s OPT

30. 2 Machines: Strong Price of Anarchy L 1 ≤ L 2 < OPT – IMPOSSIBLE!!! L 1 ≤ OPT ≤ L 2 – L 2 ≤ OPT + L 1 ≤ 2 OPT OPT < L 1 ≤ L 2 – Not a Strong Eq. – Deviation by all players THEOREM: SPoA ≤ 2 L1L1 L2L2 s OPT

31. Mixed Deviations and Mixed Strong Eq Nash Eq – unilateral deviations – pure and mixed deviations are equivalent Strong Eq – coordinated deviation – Pure and mixed deviations are not equivalent – Given a mixed deviation, there might be no single pure deviation which is good J1J1 M1M1 M2M2 J3J3 J2J2 Unique Nash Eq J1J1 J2J2 ¾¼ c J1 =c J2 =15/8 c J1 =c J2 =2 mixed deviation J1J1 M1M1 M2M2 J3J3 J2J2 ½½

32. Mixed Deviations and Mixed Equilibrium However, in many cases, allowing mixed deviations by a coalition eliminates all Nash Eq. Theorem: for m≥5 identical machines, and n>3m unit jobs, there is no 4-Strong Eq when mixed deviations are allowed – Based on a lemma that shows that the support of any two “mixing” jobs must be disjoint

33. Strong equilibrium in multicast routing Theorem : There exists a multicast routing game that does not posses a strong equilibrium. Proof: s t1t1 t2t2 2 1 1 ½-3ε 2+2ε 1-2ε 1+3ε Unique NE: c 1 (S) = c 2 (S) = 2/2+1=2 deviation: c i (S) < 2 No SE in game

34. Single-Commodity Games Theorem : Every single-commodity fair connection game possesses a SE Proof: we show that a profile S in which all players choose the shortest path from s to t is a SE st S Cost(S’\S) ≥ Cost(S\S’)  c i (S’) ≥ c i (S) S’

35. Strong Price of Anarchy Theorem : The strong price of Anarchy of a multicast routing game with n players is at most H(n). Proof: Let S be a SE, and S Γ be the induced profile of players in Γ, and let S* be OPT For k=n,…,1, since S is SE, there exists a player “k”   k ={1,…,k} that does not benefit from coal. deviation. i.e., Potential function:

36. Proof (cont’d) We got for every k: Summing over all players:

37. Lower Bound OPT: all users use indirect edge, c(OPT)=1+  Unique NE and SE: each user uses direct edge to t i, c(NE)=c(SE)=H(n)  PoA = SPoA = PoS = SPoS = H(n) t1t1 t n-2 t3t3 t2t2 t n-1 tntn s 1 1+ 