1. Game Theory: introduction and applications to computer networks Game Theory: introduction and applications to computer networks Introduction Giovanni Neglia INRIA – EPI Maestro 27 January 2014 Part of the slides are based on a previous course with D. Figueiredo (UFRJ) and H. Zhang (Suffolk University)

2. Mixed strategies equilibria r Same idea of equilibrium m each player plays a mixed strategy (equalizing strategy), that equalizes the opponent payoffs m how to calculate it? AB A5, 0-1, 4 B3, 22, 1 Rose Colin

3. Mixed strategies equilibria r Same idea of equilibrium m each player plays a mixed strategy, that equalizes the opponent payoffs m how to calculate it? AB A-0-4 B-2 Rose Colin Rose considers Colin’s game 4 1 1/5 4/5

4. Mixed strategies equilibria r Same idea of equilibrium m each player plays a mixed strategy, that equalizes the opponent payoffs m how to calculate it? AB A5 B32 Rose Colin Colin considers Rose’s game 3/5 2/5

5. Mixed strategies equilibria r Same idea of equilibrium m each player plays a mixed strategy, that equalizes the opponent payoffs m how to calculate it? AB A5, 0-1, 4 B3, 22, 1 Rose Colin Rose playing (1/5,4/5) Colin playing (3/5,2/5) is an equilibrium Rose gains 13/5 Colin gains 8/5

6. Good news: Nash’s theorem [1950] r Every two-person games has at least one equilibrium either in pure strategies or in mixed strategies m Proved using fixed point theorem m generalized to N person game r This equilibrium concept called Nash equilibrium in his honor m A vector of strategies (a profile) is a Nash Equilibrium (NE) if no player can unilaterally change its strategy and increase its payoff

7. A useful property r Given a finite game, a profile is a mixed NE of the game if and only if for every player i, every pure strategy used by i with non-null probability is a best response to other players mixed strategies in the profile m see Osborne and Rubinstein, A course in game theory, Lemma 33.2

8. Bad news: what do we lose? r equivalence r interchangeability r identity of equalizing strategies with prudential strategies r main cause m at equilibrium every player is considering the opponent’s payoffs ignoring its payoffs. r New problematic aspect m group rationality versus individual rationality (cooperation versus competition) m absent in zero-sum games  we lose the idea of the solution

9. Game of Chicken 2 2 r Game of Chicken (aka. Hawk-Dove Game) m driver who swerves looses swervestay swerve0, 0-1, 5 stay5, -1-10, -10 Driver 1 Driver 2 Drivers want to do opposite of one another Two equilibria: not equivalent not interchangeable! playing an equilibrium strategy does not lead to equilibrium

10. The Prisoner’s Dilemma r One of the most studied and used games m proposed in 1950 r Two suspects arrested for joint crime m each suspect when interrogated separately, has option to confess NCC 2, 210, 1 C1, 105, 5 Suspect 1 Suspect 2 payoff is years in jail (smaller is better) single NE better outcome

11. Pareto Optimal NCC 2, 210, 1 C1, 105, 5 Suspect 1 Suspect 2 r Def: outcome o* is Pareto Optimal if no other outcome would give to all the players a payoff not smaller and a payoff higher to at least one of them r Pareto Principle: to be acceptable as a solution of a game, an outcome should be Pareto Optimal o the NE of the Prisoner’s dilemma is not! r Conflict between group rationality (Pareto principle) and individual rationality (dominance principle) Pareto Optimal

12. Payoff polygon r All the points in the convex hull of the pure strategy payoffs correspond to payoffs obtainable by mixed strategies r The north-east boundary contains the Pareto optimal points AB A5, 0-1, 4 B3, 22, 1 Rose Colin A,A B,A A,B B,B NE Rose’s payoff Colin’s payoff

13. Another possible approach to equilibria  NE  equalizing strategies r What about prudential strategies?

14. Prudential strategies r Each player tries to minimize its maximum loss (then it plays in its own game) AB A5, 0-1, 4 B3, 22, 1 Rose Colin

15. Prudential strategies r Rose assumes that Colin would like to minimize her gain r Rose plays in Rose’s game r Saddle point in BB r B is Rose’s prudential strategy and guarantees to Rose at least 2 (Rose’s security level) AB A5 B32 Rose Colin

16. Prudential strategies r Colin assumes that Rose would like to minimize his gain (maximize his loss) r Colin plays in Colin’s game r mixed strategy equilibrium, r (3/5,2/5) is Colin’s prudential strategy and guarantees Colin a gain not smaller than 8/5 AB A0-4 B-2 Rose Colin

17. Prudential strategies r Prudential strategies m Rose plays B, Colin plays A w. prob. 3/5, B w. 2/5 m Rose gains 13/5 (>2), Colin gains 8/5 r Is it stable? m No, if Colin thinks that Rose plays B, he would be better off by playing A (Colin’s counter-prudential strategy) AB A5, 0-1, 4 B3, 22, 1 Rose Colin

18. Prudential strategies r are not the solution neither: m do not lead to equilibria m do not solve the group rationality versus individual rationality conflict r dual basic problem: m look at your payoff, ignoring the payoffs of the opponents

19. Exercises r Find NE and Pareto optimal outcomes: NCC 2, 210, 1 C1, 105, 5 AB A2, 33, 2 B1, 00, 1 swervestay swerve0, 0-1, 5 stay5, -1-10, -10 AB A2, 41, 0 B3, 10, 4

20. Performance Evaluation Routing as a Potential game Giovanni Neglia INRIA – EPI Maestro

21. Routing games r Possible in the Internet? 12 2 2 2 2 2 2 2 2 2 2 ? Traffic (cars#) Delay

22. Overlay networks Internet Overlay Underlay

23. Routing games r Users can ignore ISP choices 3 4 1 2 3 4 1 route allowed by the overlay underlay route An Overlay for routing: Resilient Overlay Routing

24. Traffic demand r unit traffic demands between pair of nodes 2 3 4 1 a b c d e f 1,3 f 2,3

25. Delay costs r Social cost: C S = Σ  ε E f  *c  (f  ) r User cost: m C 1,3 (f)= Σ  ε R 1,3 c  (f  ) 2 3 4 1 a b c d e R 1,3 = {a,b}, R 2,3 ={b} f a =f 1,3, f b = f 1,3 + f 2,3, f c =f d =0 f 1,3 f 2,3 c  (f  ),  ε E={a,b,c,d,e}, Non-negative, non decreasing functions

26. Pigou’s example r Two possible roads between 1 and 2 m a) a longer highway (almost constant transit time) ‏ m b) shorter but traffic sensitive city road r 2 Selfish users (choose the road in order to minimize their delay) 12 transit_time a =2 hour transit_time b =x hours ab a-2, -2-2, -1 b-1, -2-2, -2 Rose Colin

27. Pigou’s example r Two possible roads between 1 and 2 m a) a longer highway (almost constant transit time) ‏ m b) shorter but traffic sensitive city road r 2 Selfish users (choose the road in order to minimize their delay) m There is 1 (pure-strategy) NE where they all choose the city road... m even if the optimal allocation is not worse for the single user!  What if transit_time a =2+ ε?  In what follows we only consider pure strategy NE 12 transit_time a =2 hour transit_time b =x hours fbfb Social cost 02 4 3 1

28. What is the cost of user selfishness for the community? r Loss of Efficiency (LoE) m given a NE with social cost C S (f NE ) m and the traffic allocation with minimum social cost C S (f Opt ) m LoE = C S (f NE ) / C S (f Opt )

29. Pigou’s example r The LoE of (b,b) is 4/3 r The LoE of (b,a) and (a,b) is 1 12 transit_time a =2 hour transit_time b =x hours ab a-2, -2-2, -1 b-1, -2-2, -2 Rose Colin

30. Braess's paradox  User cost: 3 +ε r Social cost: C NE = 6 +2ε (=C Opt ) 12 c(x)=x 3 4 c(x)=2+ε

31. Braess's paradox 12 transit_time a =3+ε hours c(x)=x 3 4 c(x)=2+ε c(x)=0

32. Braess's paradox r User cost: 4  Social cost: C NE = 8 > 6 +ε (C Opt ) r LoE = 8/(6+ ε ) -> 4/3 12 transit_time a =3+ε hours c(x)=x 3 4 c(x)=2+ε c(x)=0 ε ->0

33. Routing games 1. Is there always a (pure strategy) NE? 2. Can we always find a NE with a “small” Loss of Efficiency (LoE)?

34. Always an equilibrium? r Best Response dynamics m Start from a given routing and let each player play its Best Response strategy m What if after some time there is no change?

35. BR dynamics 1. Users costs: (3 +ε, 3 +ε) 2. Blue plays BR, costs: (3, 4 +ε ) 3. Pink plays BR, costs: (4, 4) 4. Nothing changes….  Social cost: C NE = 6 +2ε (=C Opt ) 12 c(x)=x 3 4 c(x)=2+ε

36. Always an equilibrium? r Best Response dynamics m Start from a given routing and let each player play its Best Response strategy m What if after some time there is no change? m Are we sure to stop?

37. Games with no saddle-point r There are games with no saddle-point! r An example? RPS min R P S max RPS min R 01 P 10 S 10 max 111 maximin <> minimax maximin minimax

38. Always an equilibrium? r Best Response dynamics m Start from a given routing and let each player play its Best Response strategy m What if after some time there is no change? m Are we sure to stop? In some cases we can define a potential function that keeps decreasing at each BR until a minimum is reached. Is the social cost a good candidate?

39. Potential for routing games r Potential : P =Σ  ε E P  (f  )=Σ  ε E Σ t=1,…f  c  (t) 2 3 4 1 a b c d e R 1,3 = {a,b}, R 2,3 ={b} f a =f 1,3, f b = f 1,3 + f 2,3, f c =f d =0 f 1,3 f 2,3 c  (f  ),  ε E={a,b,c,d,e}, Non-negative, non decreasing functions

40. Potential decreases at every BR 1. User costs: (3 +ε, 3 +ε), P=6+2ε 2. Blue plays BR, costs: (3, 4 +ε ), P= 6+ε 3. Pink plays BR, costs: (4, 4), P=6 4. Nothing changes…. 12 c(x)=x 3 4 c(x)=2+ε

41. Potential decreases at every BR 12 c(x)=x 3 4 c(x)=2+ε From route R to route R’ r f’  =f  +1 if  in R’-R, f’  =f  -1 if  in R-R’ r P  -P’  =-c  (f  +1) if  in R’-R, r P  -P’  =c  (f  ) if  in R-R’  P-P’=Σ  ε R c  (f  )-Σ  ε R’ c  (f’  )= =user difference cost between R and R’>0

42. BR dynamics converges to an equilibrium r The potential decreases at every step r There is a finite number of possible potential values r After a finite number of steps a potential local minimum is reached r The final routes identify a (pure strategy) NE