1 Module J Game theory in Wireless Networks Julien Freudiger, Márk Félegyházi, Jean-Pierre Hubaux

2 Upcoming networks vs. mechanisms XXXXX XXXX XXXXXXX XXXXXXXX XXXXXXX XXXXXXX XXXXXX XXXX Naming and addressing Discouraging greedy op. Small operators, community networks Cellular operators in shared spectrum Mesh networks Hybrid ad hoc networks Autonomous ad hoc networks Security associations Securing neighbor discovery Secure routing Privacy Enforcing PKT FWing Enforcing fair MAC Vehicular networks Sensor networks RFID networks Part I Part IIIPart II Upcoming wireless networks Security and cooperation mechanisms

3 Brief introduction to Game Theory Discipline aiming at modeling situations in which actors have to make decisions which have mutual, possibly conflicting, consequences Classical applications: economics, but also politics and biology Example: should a company invest in a new plant, or enter a new market, considering that the competition may make similar moves? Most widespread kind of game: non-cooperative (meaning that the players do not attempt to find an agreement about their possible moves)

4 Game 1: Guess what? 1.Choose an integer number in [0,100] 2.The winner is: gets closest to the 2/3 of the average 3.Take a piece of paper and write your name (e.g., Julien FREUDIGER) your number (e.g., 99) 4.Price: 4 extra points in Quiz I if several winners, price is divided equally 5.We play the game again at the end of the course ! Remark : Assume truthful game – rational student => write best guess – no (rational) student discussed solution with friends

5 Classification of games Non-cooperativeCooperative StaticDynamic (repeated) Strategic-formExtensive-form Perfect informationImperfect information Complete informationIncomplete information Cooperative Imperfect information Incomplete information Perfect information: each player can observe the action of each other player. Complete information: each player knows the identity of other players and, for each of them, the payoff resulting of each strategy.

6 Cooperation in self-organized wireless networks S1S1 S2S2 D1D1 D2D2 Usually, the devices are assumed to be cooperative. But what if they are not?

7 Static (or “single-stage”) games

8 Example 1: The Forwarder’s Dilemma ? ? Blue Green

9 E1: From a problem to a game users controlling the devices are rational = try to maximize their benefit game formulation: G = (P,S,U) – P: set of players – S: set of strategy functions – U: set of payoff functions strategic-form representation Reward for packet reaching the destination: 1 Cost of packet forwarding: c (0 < c << 1) (1-c, 1-c)(-c, 1) (1, -c)(0, 0) Blue Green Forward Drop Forward Drop

10 Solving the Forwarder’s Dilemma (1/2) Strict dominance: strictly best strategy, for any strategy of the other player(s) where: payoff function of player i strategies of all players except player i In Example 1, strategy Drop strictly dominates strategy Forward (1-c, 1-c)(-c, 1) (1, -c)(0, 0) Blue Green Forward Drop Forward Drop Strategy strictly dominates if

11 Solving the Forwarder’s Dilemma (2/2) Solution by iterative strict dominance: (1-c, 1-c)(-c, 1) (1, -c)(0, 0) Blue Green Forward Drop Forward Drop Drop strictly dominates Forward Dilemma Forward would result in a better outcome BUT }

12 Repeated Iterative Strict Dominance (3, 9)(0, 6)(0, 4)(1, 2) (1, 3)(3, 6)(6, 1)(6, 3) (4, 2)(4, 1)(2, 2)(8, 3) (3, 7)(4, 5)(2, 6)(4, 7) Blue Green A B X Y V W C D Strict dominance: strictly best strategy, for any strategy of the other player(s)

13 Example 2: The Joint Packet Forwarding Game ? Blue Green Source Dest ? No strictly dominated strategies ! Reward for packet reaching the destination: 1 Cost of packet forwarding: c (0 < c << 1) (1-c, 1-c)(-c, 0) (0, 0) Blue Green Forward Drop Forward Drop

14 E2: Weak dominance ? Blue Green Source Dest ? Weak dominance: strictly better strategy for at least one opponent strategy with strict inequality for at least one s -i Iterative weak dominance (1-c, 1-c)(-c, 0) (0, 0) Blue Green Forward Drop Forward Drop BUT The result of the iterative weak dominance is not unique in general ! Strategy s i is weakly dominates strategy s’ i if:

15 Repeated Iterative Weak Dominance (3, 9)(0, 6)(0, 4)(1, 2) (1, 3)(3, 6)(6, 1)(6, 3) (4, 2)(4, 1)(2, 2)(8, 2) (3, 7)(4, 5)(2, 6)(4, 7) Blue Green A B X Y V W C D Weak dominance: strictly better strategy for at least one opponent strategy

16 Nash equilibrium (1/2) Nash Equilibrium: no player can increase its utility by deviating unilaterally (1-c, 1-c)(-c, 1) (1, -c)(0, 0) Blue Green Forward Drop Forward Drop E1: The Forwarder’s Dilemma E2: The Joint Packet Forwarding game (1-c, 1-c)(-c, 0) (0, 0) Blue Green Forward Drop Forward Drop

17 Finding Nash Equilibria (3, 9)(0, 6)(0, 4)(1, 2) (1, 3)(3, 6)(6, 1)(6, 3) (4, 2)(4, 1)(2, 2)(8, 2) (3, 7)(4, 5)(2, 6)(4, 7) Blue Green A B X Y V W C D Nash Equilibrium: no player can increase its utility by deviating unilaterally

18 Nash equilibrium (2/2) where: utility function of player i strategy of player i The best response of player i to the profile of strategies s -i is a strategy s i such that: Nash Equilibrium = Mutual best responses Caution! Many games have more than one Nash equilibrium Strategy profile s * constitutes a Nash equilibrium if, for each player i,

19 Example 3: The Multiple Access game Reward for successful transmission: 1 Cost of transmission: c (0 < c << 1) There is no strictly dominating strategy (0, 0)(0, 1-c) (1-c, 0)(-c, -c) Blue Green Quiet Transmit Quiet Transmit There are two Nash equilibria Time-division channel

20 E3: Mixed strategy Nash equilibrium objectives – Blue: choose p to maximize u blue – Green: choose q to maximize u green p: probability of transmit for Blue q: probability of transmit for Green is a Nash equilibrium

21 Example 4: The Jamming game transmitter: reward for successful transmission: 1 loss for jammed transmission: -1 jammer: reward for successful jamming: 1 loss for missed jamming: -1 There is no pure-strategy Nash equilibrium two channels: C 1 and C 2 (-1, 1)(1, -1) (-1, 1) Blue Green C1C1 C2C2 C1C1 C2C2 transmitter jammer is a Nash equilibrium p: probability of transmit on C 1 for Blue q: probability of transmit on C 1 for Green

22 Theorem by Nash, 1950 Theorem: Every finite strategic-form game has a mixed-strategy Nash equilibrium.

23 Efficiency of Nash equilibria E2: The Joint Packet Forwarding game (1-c, 1-c)(-c, 0) (0, 0) Blue Green Forward Drop Forward Drop How to choose between several Nash equilibria ? Pareto-optimality: A strategy profile is Pareto-optimal if it is not possible to increase the payoff of any player without decreasing the payoff of another player.

24 Efficiency (3, 9)(0, 6)(0, 4)(1, 2) (1, 3)(3, 6)(6, 1)(6, 3) (4, 2)(4, 1)(2, 2)(8, 2) (3, 7)(4, 5)(2, 6)(4, 7) Blue Green A B X Y V W C D * * * * Pareto-optimality: It is not possible to increase the payoff of any player without decreasing the payoff of another player.

25 How to study Nash equilibria ? Properties of Nash equilibria to investigate: existence uniqueness efficiency (Pareto-optimality) emergence (dynamic games, agreements)

26 Dynamic games

27 Extensive-form games usually to model sequential decisions game represented by a tree Example 3 modified: the Sequential Multiple Access game: Blue plays first, then Green plays. Green Blue T Q T Q T Q (-c,-c) (1-c,0)(0,1-c) (0,0) Reward for successful transmission: 1 Cost of transmission: c (0 < c << 1) Green Time-division channel

28 Strategies in dynamic games The strategy defines the moves for a player for every node in the game, even for those nodes that are not reached if the strategy is played. Green Blue T Q T Q T Q (-c,-c) (1-c,0)(0,1-c) (0,0) Green strategies for Blue: T, Q strategies for Green: TT, TQ, QT and QQ If they have to decide independently: three Nash equilibria (T,QT), (T,QQ) and (Q,TT)

29 Extensive to Normal form Blue vs. Green TTTQQTQQ T(-c,-c) (1-c,0) Q(0,1-c)(0,0)(0,1-c)(0,0)

30 Backward induction Solve the game by reducing from the final stage Eliminates Nash equilibria that are increadible threats Green Blue T Q T Q T Q (-c,-c) (1-c,0)(0,1-c) (0,0) Green incredible threat: (Q, TT)

31 Subgame perfection Extends the notion of Nash equilibrium Green Blue T Q T Q T Q (-c,-c) (1-c,0)(0,1-c) (0,0) Green Subgame perfect equilibria: (T, QT) and (T, QQ) One-deviation property: A strategy s i conforms to the one-deviation property if there does not exist any node of the tree, in which a player i can gain by deviating from s i and apply it otherwise. Subgame perfect equilibrium: A strategy profile s constitutes a subgame perfect equilibrium if the one-deviation property holds for every strategy s i in s. Finding subgame perfect equilibria using backward induction

32 Repeated games

33 Repeated games repeated interaction between the players (in stages) move: decision in one interaction strategy: defines how to choose the next move, given the previous moves history: the ordered set of moves in previous stages – most prominent games are history-1 games (players consider only the previous stage) initial move: the first move with no history finite-horizon vs. infinite-horizon games stages denoted by t (or k)

34 Payoffs: Objectives in the repeated game finite-horizon vs. infinite-horizon games myopic vs. long-sighted repeated game myopic: long-sighted finite: long-sighted infinite: payoff with discounting: is the discounting factor

35 Strategies in the repeated game usually, history-1 strategies, based on different inputs: – others’ behavior: – others’ and own behavior: – payoff: Example strategies in the Forwarder’s Dilemma: Blue (t)initial move FDstrategy name Green (t+1)FFFAllC FFDTit-For-Tat (TFT) DDDAllD FDFAnti-TFT

36 The Repeated Forwarder’s Dilemma (1-c, 1-c)(-c, 1) (1, -c)(0, 0) Blue Green Forward Drop Forward Drop ? ? Blue Green stage payoff

37 Analysis of the Repeated Forwarder’s Dilemma (1/3) Blue strategyGreen strategy AllD TFT AllDAllC TFT infinite game with discounting: Blue utilityGreen utility 00 1-c 1/(1-  )-c/(1-  ) (1-c)/(1-  )

38 Analysis of the Repeated Forwarder’s Dilemma (2/3) Blue strategyGreen strategy AllD TFT AllDAllC TFT AllC receives a high payoff with itself and TFT, but AllD exploits AllC AllD performs poor with itself TFT performs well with AllC and itself, and TFT retaliates the defection of AllD TFT is the best strategy if  is high ! Blue utilityGreen utility 00 1-c 1/(1-  )-c/(1-  ) (1-c)/(1-  )

39 Analysis of the Repeated Forwarder’s Dilemma (3/3) Theorem: In the Repeated Forwarder’s Dilemma, if both players play AllD, it is a Nash equilibrium. Theorem: In the Repeated Forwarder’s Dilemma, both players playing TFT is a Nash equilibrium as well. Blue strategyGreen strategyBlue utilityGreen utility AllD 00 TFT (1-c)/(1-  ) The Nash equilibrium s Blue = TFT and s Green = TFT is Pareto-optimal (but s Blue = AllD and s Green = AllD is not) !

40 Experiment: Tournament by Axelrod, 1984 any strategy can be submitted (history-X) strategies play the Repeated Prisoner’s Dilemma (Repeated Forwarder’s Dilemma) in pairs number of rounds is finite but unknown TFT was the winner second round: TFT was the winner again

41 Mechanism Design

42 Game Theory: Analysis vs. Mechanism Design analysis mechanism design behavior system

43 Algorithmic Mechanism Design algorithmic mechanism design behavior (truthful) system (protocols) Questions: distributed implementation complexity convergence speed

44 Example 5: Stable matching in cellular networks (CSM) A B C

45 E5: Cellular stable matching (CSM) N base stations and M mobiles (M>N) Each BS and mobile has preference lists Unstable: If BS A and B serve mobile a and b, respectively, although a prefers B and B also prefers a. Simplified version: N=M Stable marriage problem

46 E5’: Cellular stable matching simplified (CSM’), (1/2) N=M A B C b a c

47 E5’: Cellular stable matching simplified (2/2) preference matrix 1,32,23,1 1,32,2 3,11,3 BS mobiles A B a b c C

48 E5’: Existence of CSM Theorem (Gale-Shapley, 1962): There always exists a stable matching Proof (constructive): The Gale-Shapley algorithm

49 E5’: CSM – The Gale-Shapley algorithm A B C a b c A B C a b c network-centric user-centric ,32,23,1 1,32,2 3,11,3 BS mobiles A B a b c C

50 E5’: Optimality of CSM Optimal: Each participant (in a group) is at least as well off as in another matching The Gale-Shapley algorithm is male (BS)-optimal

51 E5: Back to CSM The Gale-Shapley algorithm generalized – quota at each BS (q=2) 1,32,24,36,13,25,1 6,11,35,13,34,12,2 3,16,21,25,34,3 BS mobiles A B a b c C d e f A B C a b c d e f

52 Stable Matching Applications: – WiFi networks – Load-balancing for processors – Students to schools – Job-hunting

53 Truthful Stable Matching Theorem (Roth, 1982): – The Gale-Shapley algorithm is truthful for males Theorem (Gale-Sotomayor, 1985): – Women can cheat such that they get a better partner 1,32,23,1 1,32,2 3,11,3 BS mobiles A B a b c C – Accept only your favorite peer A B C a b c

54 E6: Stable matching in ad hoc networks (ASM) M mobiles Each mobile has preference lists Unstable: If mobile a and c communicate with mobile b and d, respectively, although a prefers c and c prefers a. Stable roommate problem

55 E6’: ASM simplified preference matrix -1,32,23,1 -1,32,2 3,1-1,3 2,23,1- mobiles a b a b c c a b c d d d a b c d

56 Discussion on game theory Rationality Utility function and cost Pricing and mechanism design (to promote desirable solutions) Infinite-horizon games and discounting Reputation Cooperative games Imperfect / incomplete information

57 Who is malicious? Who is selfish?  Both security and game theory backgrounds are useful in many cases !! Harm everyone: viruses,… Selective harm: DoS,…Spammer Cyber-gangster: phishing attacks, trojan horses,… Big brother Greedy operator Selfish mobile station

58 Conclusion Game theory can help modeling greedy behavior in wireless networks Discipline still in its infancy Alternative solutions – Ignore the problem – Build protocols in tamper-resistant hardware

59 Game 2: Guess what again? 1.Choose an integer number in [0,100] 2.The winner is: gets closest to the 2/3 of the average 3.Take a piece of paper and write your name (e.g., Julien FREUDIGER) your number 4.Price: 2 extra points in Quiz I if several winners, price is divided equally Remarks about the game: truthful: – each (rational) student writes his best guess – no (rational) student should have discussed the solution with friends