1. Algorithmic Issues in Strategic Distributed Systems FOURTH PART

2. Suggested readings Algorithmic Game Theory, Edited by Noam Nisan, Tim Roughgarden, Eva Tardos, and Vijay V. Vazirani, Cambridge University Press. Blog by Noam Nisan http://agtb.wordpress.com/

3. Two Research Traditions Theory of Algorithms: computational issues What can be feasibly computed? How long does it take to compute a solution? Which is the quality of a computed solution? Centralized or distributed computational models Game Theory: interaction between self-interested individuals What is the outcome of the interaction? Which social goals are compatible with selfishness?

4. Different Assumptions Theory of Algorithms (in distributed systems): Processors are obedient, faulty (i.e., crash), adversarial (i.e., Byzantine), or they compete without being strategic (e.g., concurrent systems) Large systems, limited computational resources Game Theory: Players are strategic (selfish) Small systems, unlimited computational resources

5. The Internet World Users often selfish Have their own individual goals Own network components Internet scale Massive systems Limited communication/computational resources  Both strategic and computational issues!

6. Fundamental question How the computational aspects of a strategic distributed system should be addressed? Theory of Algorithms Game Theory Algorithmic Game Theory +=

7. Basics of Game Theory A game consists of: A set of players (or agents) A specification of the information available to each player A set of rules of encounter: Who should act when, and what are the possible actions (strategies) A specification of payoffs for each possible outcome (combination of strategies) of the game Game Theory attempts to predict the final outcomes (or solutions) of the game by taking into account the individual behavior of the players

8. Solution concept How do we establish that an outcome is a solution? Among the possible outcomes of a game, those enjoying the following property play a fundamental role: Equilibrium solution: strategy combination in which players are not willing to change their state. But this is quite informal: what does it rationally mean that a player does not want to change his state? In the Homo Economicus model, this makes sense when he has selected a strategy that maximizes his individual wealth, knowing that other players are also doing the same.

9. Roadmap We will focus on two prominent types of equilibria: Nash Equilibria (NE) and Dominant Strategy Equilibria (DSE) Computational Aspects of Nash Equilibria Can a NE be feasibly computed, once it exists? What about the “quality” of a NE? Case study: Network Flow Games (i.e., selfish routing in Internet), Network Design Games, Network Creation Games (Algorithmic) Mechanism Design Which social goals can be (efficiently) implemented in a strategic distributed system? Strategy-proof mechanisms in DSE: Vickrey-Clarke-Groves (VCG)-mechanisms and one-parameter mechanisms Case study: Shortest Path, Minimum Spanning Tree, Single- source Shortest-path Tree, Single-Minded Combinatorial Auctions

10. FIRST PART: (Nash) Equilibria

11. (Some) Types of games Cooperative/Non-cooperative Symmetric/Asymmetric (for 2-player games) Zero sum/Non-zero sum Simultaneous/Sequential Perfect information/Imperfect information One-shot/Repeated

12. Games in Normal-Form A set of N rational players For each player i, a strategy set S i A payoff matrix: for each strategy combination (s 1, s 2, …, s N ), where s i  S i, a corresponding payoff vector (p 1, p 2, …, p N )  |S 1 |  |S 2 |  …  |S N | payoff matrix We start by considering simultaneous, perfect- information and non-cooperative games. These games are usually represented explicitly by listing all possible strategies and corresponding payoffs of all players (this is the so-called normal–form); more formally, we have:

13. A famous game: the Prisoner’s Dilemma Prisoner I Prisoner II Don’t Implicate Implicate Don’t Implicate 1, 16, 0 Implicate0, 65, 5 Strategy Set Strategy Set Payoffs (for this game, these are years in jail, so they should be seen as a cost that a player wants to minimize) Non-cooperative, symmetric, non-zero sum, simultaneous, perfect information, one-shot, 2-player game

14. Prisoner I’s decision Prisoner I’s decision: If II chooses Don’t Implicate then it is best to Implicate If II chooses Implicate then it is best to Implicate It is best to Implicate for I, regardless of what II does: Dominant Strategy Prisoner I Prisoner II Don’t ImplicateImplicate Don’t Implicate1, 16, 0 Implicate0, 65, 5

15. Prisoner II’s decision Prisoner II’s decision: If I chooses Don’t Implicate then it is best to Implicate If I chooses Implicate then it is best to Implicate It is best to Implicate for II, regardless of what I does: Dominant Strategy Prisoner I Prisoner II Don’t ImplicateImplicate Don’t Implicate1, 16, 0 Implicate0, 65, 5

16. Hence… It is best for both to implicate regardless of what the other one does Implicate is a Dominant Strategy for both (Implicate, Implicate) becomes the Dominant Strategy Equilibrium Note: If they might collude, then it’s beneficial for both to Not Implicate, but it’s not an equilibrium as both have incentive to deviate Prisoner I Prisoner II Don’t Implicate Implicate Don’t Implicate 1, 16, 0 Implicate0, 65, 5

17. Dominant Strategy Equilibrium Dominant Strategy Equilibrium: is a strategy combination s * = (s 1 *, s 2 *, …, s N * ), such that s i * is a dominant strategy for each i, namely, for any possible alternative strategy profile s= (s 1, s 2, …, s i, …, s N ): if p i is a utility, then p i (s 1, s 2,…, s i *,…, s N ) ≥ p i (s 1, s 2,…, s i,…, s N ) if p i is a cost, then p i (s 1, s 2, …, s i *, …, s N ) ≤ p i (s 1, s 2, …, s i, …, s N ) Dominant Strategy is the best response to any strategy of other players If a game has a DSE, then players will immediately converge to it Of course, not all games (only very few in the practice!) have a dominant strategy equilibrium

18. A more relaxed solution concept: Nash Equilibrium [1951] Nash Equilibrium: is a strategy combination s * = (s 1 *, s 2 *, …, s N * ) such that for each i, s i * is a best response to (s 1 *, …,s i-1 *,s i+1 *,…, s N * ), namely, for any possible alternative strategy s i of player i if p i is a utility, then p i (s 1 *, s 2 *,…, s i *,…, s N * ) ≥ p i (s 1 *, s 2 *,…, s i,…, s N * ) if p i is a cost, then p i (s 1 *, s 2 *, …, s i *, …, s N * ) ≤ p i (s 1 *, s 2 *, …, s i, …, s N * )

19. Nash Equilibrium In a NE no agent can unilaterally deviate from his strategy given others’ strategies as fixed Each agent has to take into consideration the strategies of the other agents If a game has one or more NE, players need not to converge to it Dominant Strategy Equilibrium  Nash Equilibrium (but the converse is not true)

20. Nash Equilibrium: The Battle of the Sexes (coordination game) (Stadium, Stadium) is a NE: Best responses to each other (Cinema, Cinema) is a NE: Best responses to each other  but they are not Dominant Strategy Equilibria … are we really sure they will eventually go out together???? Man Woman StadiumCinema Stadium2, 10, 0 Cinema0, 01, 2

21. A crucial issue in game theory: the existence of a NE Unfortunately, for pure strategies games (as those seen so far, in which each player, for each possible situation of the game, selects his action deterministically), it is easy to see that we cannot have a general result of existence In other words, there may be no, one, or many NE, depending on the game

22. A conflictual game: Head or Tail Player I (row) prefers to do what Player II does, while Player II prefer to do the opposite of what Player I does!  In any configuration, one of the players prefers to change his strategy, and so on and so forth…thus, there are no NE! Player I Player II HeadTail Head1,-1-1,1 Tail-1,11,-1

23. On the existence of a NE However, when a player can select his strategy randomly by using a probability distribution over his set of possible pure strategies (mixed strategy), then the following general result holds: Theorem (Nash, 1951): Any game with a finite set of players and a finite set of strategies has a NE of mixed strategies (i.e., there exists a profile of probability distributions for the players such that the expected payoff of each player cannot be improved by changing unilaterally the selected probability distribution). Head or Tail game: if each player sets p(Head)=p(Tail)=1/2, then the expected payoff of each player is 0, and this is a NE, since no player can improve on this by choosing unilaterally a different randomization!

24. 1.Finding (efficiently) a mixed/pure (if any) NE 2.Establishing the quality of a NE, as compared to a cooperative system, namely a system in which agents can collaborate (recall the Prisoner’s Dilemma) 3.In a repeated game, establishing whether and in how many steps the system will eventually converge to a NE (recall the Battle of the Sexes) 4.Verifying that a strategy profile is a NE, approximating a NE, NE in resource (e.g., time, space, message size) constrained settings, breaking a NE by colluding, etc... Fundamental computational issues concerned with NE (interested in a Thesis, or even in a PhD?)

25. Finding a NE in mixed strategies How do we select the correct probability distribution ? It looks like a problem in the continuous… …but it’s not, actually! It can be shown that such a distribution can be found by selecting for each player a best possible subset of pure strategies (so-called best support), over which the probability distribution can actually be found by solving a system of algebraic equations!  : In the practice, the problem can be solved by a simplex-like technique called the Lemke–Howson algorithm, which however is exponential in the worst case

26. Is finding a NE NP-hard? In pure strategies, yes, for many games of interest What about mixed strategies? W.l.o.g., we restrict ourself to 2- player games: Then, we wonder whether 2-NASH is NP-hard. Recall: a problem  is NP-hard if one can Turing-reduce in polynomial time any NP-complete problem  ’ to it (this means,  ’ can be solved in polynomial time by an oracle machine with an oracle for  ) Recall also: a problem  is in NP (resp., in coNP) if all its "yes"- instances (resp., “no”-instances) can be decided in polynomial time by a Non-Deterministic Turing Machine (NDTM). But 2-NASH can be solved in polynomial-time by a NDTM (by enumerating all the supports); moreover, every instance of 2-NASH is a “yes”-instance (since every game has a NE), and so we could certificate in polynomial-time on a NDTM both “yes” and “no”- instances of any NP-complete problem  if 2-NASH is NP-hard then NP = coNP (hard to believe!)

27. The complexity class PPAD Definition (Papadimitriou, 1994): PPAD (Polynomial Parity Argument – Directed case) is a subclass of TFNP (Total Function Nondeterministic Polynomial), where existence of a solution is guaranteed by a parity argument. Roughly speaking, PPAD contains all problems whose solution space can be set up as the (non-empty) set of all sinks in a suitable directed graph (generated by the input instance), having an exponential number of vertices in the size of the input, though. Breakthrough: 2-NASH is PPAD-complete!!! (Chen & Deng, FOCS’06) Remark: It could very well be that PPAD=P  NP, but several PPAD-complete problems are resisting for decades to poly- time attacks (e.g., finding Brouwer fixed points)

28. Finding a NE in pure strategies By definition, it is easy to see that an entry (p 1,…,p N ) of the payoff matrix is a NE if and only if p i is the maximum ith element of the row (p 1,…,p i-1, {p(s):s  S i },p i+1,…,p N ), for each i=1,…,N. Notice that, with N players, an explicit (i.e., in normal-form) representation of the payoff functions is exponential in N  brute-force (i.e., enumerative) search for pure NE is then exponential in the number of players (even if it is still polynomial in the input size, but the normal-form representation needs not be a minimal-space representation of the input!)  Alternative cheaper methods are sought: for many games of interest, a NE can be found in poly-time w.r.t. to the number of players (e.g., using the powerful potential method)

29. On the quality of a NE How inefficient is a NE in comparison to an idealized situation in which the players would collaborate selflessly (in other words, the distributed system become cooperative), with the common goal of maximizing the overall social welfare, i.e., a social-choice function C which depends on the payoff of all the players (e.g., C is the sum of all the payoffs)? Example: in the Prisoner’s Dilemma (PD) game, the DSE (and NE) incurs a total of 10 years in jail for the players. However, if the prisoners would cooperate by not implicating reciprocally, then they would stay a total of only 2 years in jail!

30. A worst-case perspective: the Price of Anarchy (PoA) Definition (Koutsopias & Papadimitriou, 1999): Given a game G and a social-choice function C, let S be the set of all NE. If the payoff represents a cost (resp., a utility) for a player, let OPT be the outcome of G minimizing (resp., maximizing) C. Then, the Price of Anarchy (PoA) of G w.r.t. C is Example: in the PD game, PoA PD (C)=10/2=5 PoA G (C) =

31. Internet components are made up of heterogeneous nodes and links, and the network architecture is open-based and dynamic Internet users behave selfishly: they generate traffic, and their only goal is to download/upload data as fast as possible! But the more a link is used, the more is slower, and there is no central authority “optimizing” the data flow… So, why does Internet eventually work is such a jungle??? A case study for the existence and quality of a NE: selfish routing on Internet

32. Internet can be modelled by using game theory: it is a (congestion) game in which players users strategies paths over which users can route their traffic Non-atomic Selfish Routing: There is a large number of (selfish) users; All the traffic of a user is routed over a single path simultaneously; Every user controls an infinitesimal fraction of the traffic. The Internet routing game

33. Mathematical model (multicommodity flow network) A directed graph G = (V,E) and a set of N players A set of commodities, i.e., source–sink pairs (s i,t i ), for i=1,..,k (each of the N≥k players is associated with a commodity) Let N i be the amount of players associated with (s i,t i ), for each i=1,..,k; then, the rate of traffic between s i and t i is r i =N i /N, with 0≤ r i ≤1 and  i=1,…,k r i = 1 A set Π i of paths in G between s i and t i for each i=1,..,k, and the corresponding set of all paths Π=U i=1,…,k Π i Strategy for a player: a path joining its commodity Strategy profile: a flow vector f specifying the rate of traffic f P routed on each path P  Π (notice that 0≤ f P ≤1, and that for every i=1,..,k we have  P  Π i f P =r i )

34. For each e  E, the amount of flow absorbed by e w.r.t. f is f e =  P  Π : e  P f P For each edge e, a real-value latency function l e (x) of its absorbed flow x (this is a monotonically non-decreasing function which expresses how e gets congested when a fraction 0≤x≤1 of the total flow f uses e) Cost of a player: the latency of its used path P  Π: c(P)=  e  P l e (f e ) Cost (or average latency) of a flow f (social-choice function): C(f)=  P  Π f P ·c(P)=  P  Π f P ·  e  P l e (f e )=  e  E f e ·l e (f e ) Observation: Notice that the game is not given in normal form! Mathematical model (2)

35. Flows and NE Definition: A flow f* is a Nash flow if no player can improve its cost (i.e., the cost of its used path) by changing unilaterally its path. QUESTION: Given an instance (G,r=(r 1,…,r k ),l=(l e 1,…, l e m )) of the non-atomic selfish routing game, does it admit one or more Nash flows? And in the positive case, what is the PoA of the game?

36. Latency is fixed Latency depends on the congestion (x is the fraction of flow using the edge) st Example: Pigou’s game [1920]  What is the (only) NE of this game? Trivial: all the fraction of flow tends to travel on the upper edge  the cost of the flow is C(f) = 1 · l e 1 (1) +0 · l e 2 (0) = 1 · 1 +0 · 1 = 1  What is the PoA of this NE? The optimal solution is the minimum of C(x)=x·x +(1-x)·1  C(x)=x 2 -x+1  C’(x)=2x-1  OPT for C’(x)=0, i.e., x=1/2  C(OPT)=1/2·1/2+(1-1/2)·1=0.75  PoA(C) = 1/0.75 = 4/3 Total amount of flow: 1 l e 1 (x)=x l e 2 (x)=1

37. Existence of a Nash flow Theorem ( Beckmann et al., 1956): If for each edge e the function x · l e (x) is convex (i.e., its graphic lies below the line segment joining any two points of the graphic) and continuously differentiable (i.e., its derivative exists at each point in its domain and is continuous), then the Nash flow of (G,r,l) exists and is unique, and is equal to the optimal min-cost flow of the following instance: (G,r, λ(x)=[∫ l(t)dt]/x). Remark: The optimal min-cost flow can be computed in polynomial time through convex programming methods. x 0

38. Flows and Price of Anarchy Theorem 1: In a network with linear latency functions, the cost of a Nash flow is at most 4/3 times that of the min-cost flow  every instance of the non-atomic selfish routing satisfying this constraint has PoA ≤ 4/3. Theorem 2: In a network with degree-p polynomials latency functions, the cost of a Nash flow is O(p/log p) times that of the min-cost flow. (Roughgarden & Tardos, JACM’02)

39. A bad example for non-linear latencies Assume p>>1 st xpxp 1 0 1 1-1-  close to 0 A Nash flow (of cost 1) is arbitrarily more expensive than the optimal flow (of cost close to 0)

40. Improving the PoA: the Braess’s paradox  Does it help adding edges to improve the PoA?  NO! Let’s have a look at the Braess Paradox (1968) v w x 1 s t x1 1/2 Cost of each path= x+1=1/2+1 = 1.5 Cost of the flow= 2·(1.5·1/2)=1.5 (notice this a NE and it is also an optimal flow)

41. To reduce the cost of the flow, we try to add a no- latency road between v and w. Intuitively, this should not worse things! v w x 1 s t x1 0 The Braess’s paradox (2)

42. However, each user is tempted to change its route now, since the path s → v → w → t has less cost (indeed, x≤1) v w x1 s t x1 0 If only a single user changes its route, then its cost decreases from 1.5 to approximately 1, i.e.: c(s → v → w → t) = x+0+x ≈ 0.5 + 0.5 = 1 The Braess’s paradox (3) But the problem is that all the users will decide to change!

43.  So, the cost of the flow f that now entirely uses the path s → v → w → t is: C(f) = 1·1+1·0+1·1=2>1.5  Even worse, this is a NE (the cost of the path s → v → w → t is 2, and the cost of the two paths not using (v,w) is also 2)!  The optimal min-cost flow is equal to that we had before adding the new road and so, the PoA is The Braess’s paradox (4) Notice it is 4/3, as in the Pigou’s example, and it is equal to the upper bound we gave for linear latency functions

44. Convergence towards a NE (in pure strategies games) Ok, we know that selfish routing is not so bad at its NE, but are we really sure this point of equilibrium will be eventually reached? Convergence Time: number of moves made by the players to reach a NE from an initial arbitrary state Question: Is the convergence time (polynomially) bounded in the number of players?

45. The potential function method (Rough) Definition: A potential function for a game (if any) is a real-valued function, defined on the set of possible outcomes (combination of strategies) of the game, such that the equilibria of the game are precisely the local optima of the potential function. Theorem: In any finite game admitting a potential function, best response dynamics (i.e., each player at each step greedily takes a move which maximizes its personal utility) always converges to a NE of pure strategies. But how many steps are needed to reach a NE? It depends on the combinatorial structure of the players' strategy space…

46. Convergence towards the Nash flow Positive result: If players obey to a best response dynamics (i.e., each player at each step greedily selects a strategy which maximizes its personal utility) then the non-atomic selfish routing game will converge to a NE. Moreover, for many instances (i.e., for prominent graph topologies and/or commodity specifications), the convergence time is polynomial.  Negative result: However, there exist instances of the non-atomic selfish routing game for which the convergence time is exponential (under some mild assumptions).