1. 1 Spectrum Sharing Games of Network Operators and Cognitive Radios Jean-Pierre Hubaux EPFL Work done in collaboration with M. H. Manshaei, M. Felegyhazi, J. Freudiger, and P. Marbach

2. 2 1.Spectrum allocation and usage 2.Introduction to game theory 3.Spectrum sharing games – Network operators Asymmetric network operators Border games of cellular operators Operators in shared spectrum – Unlicensed bands Asymmetric wireless systems WiFi operators – Cognitive radios Opportunistic spectrum sharing Auction based spectrum sharing Multi-Cell OFDM spectrum sharing 4.Conclusion Contents

3. 3 ©Shared spectrum company report, August 2005 In Europe, cellular operators have spent nearly 100 billion Euros to buy spectrum for the 3 rd generation 1. Spectrum allocation and usage Locations: New York city; Riverbend Park, Great Falls, VA; Tysons Corner, VA NSF Roof, Arlington, VA; NRAO, Greenbank, WV; SSC Roof, Vienna, VA

4. 4 Real-time airwaves auction model Proposal by Google, inspired by their online advertising auction Filed to FCC in May 2007 Would allow commercial operators (including small ones) to bid for access to spectrum controlled by the actual licensee Supposed to improve spectrum use and create a robust market for innovative digital services

5. 5 2. Brief introduction to Game Theory Discipline aiming at modeling situations in which actors have to make decisions which have mutual, possibly conflicting, consequences Classical applications: economics, but also politics and biology Example: should a company invest in a new plant, or enter a new market, considering that the competition may make similar moves? Most widespread kind of game: non-cooperative (meaning that the players do not attempt to find an agreement about their possible moves)

6. 6 Classification of games Non-cooperativeCooperative StaticDynamic (repeated) Strategic-formExtensive-form Perfect informationImperfect information Complete informationIncomplete information Cooperative Imperfect information Incomplete information Perfect info: each player knows the identity of other players and, for each of them, the payoff resulting of each strategy. Complete info: each player can observe the action of each other player.

7. 7 Static (or “single-stage”) games

8. 8 Example 1: The Forwarder’s Dilemma ? ? Blue Green

9. 9 E1: From a problem to a game users controlling the devices are rational = try to maximize their benefit game formulation: G = (P,S,U) – P: set of players – S: set of strategy functions – U: set of utility functions strategic-form representation Reward for packet reaching the destination: 1 Cost of packet forwarding: c (0 < c << 1) (1-c, 1-c)(-c, 1) (1, -c)(0, 0) Blue Green Forward Drop Forward Drop

10. 10 Solving the Forwarder’s Dilemma (1/2) Strict dominance: strictly best strategy, for any strategy of the other player(s) where: utility function of player i strategies of all players except player i In Example 1, strategy Drop strictly dominates strategy Forward (1-c, 1-c)(-c, 1) (1, -c)(0, 0) Blue Green Forward Drop Forward Drop Strategy strictly dominates if

11. 11 Solving the Forwarder’s Dilemma (2/2) Solution by iterative strict dominance: (1-c, 1-c)(-c, 1) (1, -c)(0, 0) Blue Green Forward Drop Forward Drop Result: Tragedy of the commons ! (Hardin, 1968) Drop strictly dominates Forward Dilemma Forward would result in a better outcome BUT }

12. 12 Example 2: The Joint Packet Forwarding Game ? Blue Green Source Dest ? No strictly dominated strategies ! Reward for packet reaching the destination: 1 Cost of packet forwarding: c (0 < c << 1) (1-c, 1-c)(-c, 0) (0, 0) Blue Green Forward Drop Forward Drop

13. 13 E2: Weak dominance ? Blue Green Source Dest ? Weak dominance: strictly better strategy for at least one opponent strategy with strict inequality for at least one s -i Iterative weak dominance (1-c, 1-c)(-c, 0) (0, 0) Blue Green Forward Drop Forward Drop BUT The result of the iterative weak dominance is not unique in general ! Strategy s’ i is weakly dominated by strategy s i if

14. 14 Nash equilibrium (1/2) Nash Equilibrium: no player can increase its utility by deviating unilaterally (1-c, 1-c)(-c, 1) (1, -c)(0, 0) Blue Green Forward Drop Forward Drop E1: The Forwarder’s Dilemma E2: The Joint Packet Forwarding game (1-c, 1-c)(-c, 0) (0, 0) Blue Green Forward Drop Forward Drop

15. 15 Nash equilibrium (2/2) where: utility function of player i strategy of player i The best response of player i to the profile of strategies s -i is a strategy s i such that: Nash Equilibrium = Mutual best responses Caution! Many games have more than one Nash equilibrium Strategy profile s * constitutes a Nash equilibrium if, for each player i,

16. 16 Example 3: The Multiple Access game Reward for successful transmission: 1 Cost of transmission: c (0 < c << 1) There is no strictly dominating strategy (0, 0)(0, 1-c) (1-c, 0)(-c, -c) Blue Green Quiet Transmit Quiet Transmit There are two Nash equilibria Time-division channel

17. 17 E3: Mixed strategy Nash equilibrium objectives – Blue: choose p to maximize u blue – Green: choose q to maximize u green p: probability of transmit for Blue q: probability of transmit for Green is a Nash equilibrium

18. 18 Example 4: The Jamming game transmitter: reward for successful transmission: 1 loss for jammed transmission: -1 jammer: reward for successful jamming: 1 loss for missed jamming: -1 There is no pure-strategy Nash equilibrium two channels: C 1 and C 2 (-1, 1)(1, -1) (-1, 1) Blue Green C1C1 C2C2 C1C1 C2C2 transmitter jammer is a Nash equilibrium p: probability of transmit on C 1 for Blue q: probability of transmit on C 1 for Green

19. 19 Theorem by Nash, 1950 Theorem: Every finite strategic-form game has a mixed-strategy Nash equilibrium.

20. 20 Efficiency of Nash equilibria E2: The Joint Packet Forwarding game (1-c, 1-c)(-c, 0) (0, 0) Blue Green Forward Drop Forward Drop How to choose between several Nash equilibria ? Pareto-optimality: A strategy profile is Pareto-optimal if it is not possible to increase the payoff of any player without decreasing the payoff of another player.

21. 21 How to study Nash equilibria ? Properties of Nash equilibria to investigate: existence uniqueness efficiency (Pareto-optimality) emergence (dynamic games, agreements)

22. 22 Repeated games

23. 23 Repeated games repeated interaction between the players (in stages) move: decision in one interaction strategy: defines how to choose the next move, given the previous moves history: the ordered set of moves in previous stages – most prominent games are history-1 games (players consider only the previous stage) initial move: the first move with no history finite-horizon vs. infinite-horizon games stages denoted by t (or k)

24. 24 Strategies in the repeated game usually, history-1 strategies, based on different inputs: – others’ behavior: – others’ and own behavior: – utility: Example strategies in the Forwarder’s Dilemma: Blue (t)initial move FDstrategy name Green (t+1)FFFAllC FFDTit-For-Tat (TFT) DDDAllD FDFAnti-TFT

25. 25 The Repeated Forwarder’s Dilemma (1-c, 1-c)(-c, 1) (1, -c)(0, 0) Blue Green Forward Drop Forward Drop ? ? Blue Green stage payoff

26. 26 Analysis of the Repeated Forwarder’s Dilemma Blue strategyGreen strategy AllD TFT AllDAllC TFT infinite game with discounting: Blue utilityGreen utility 00 1-c 1/(1-ω)-c/(1-ω) (1-c)/(1-ω)

27. 27 Conclusion on game theory Game theory can help modeling greedy behavior in wireless networks Discipline still in its infancy Alternative solutions – Ignore the problem – Build protocols in tamper-resistant hardware

28. 28 http://secowinet.epfl.ch For the tutorial on game theory: M. Felegyhazi and J.-P. Hubaux, Game Theory in Wireless Networks: A Tutorial Technical report – 2006 [LCA-REPORT-2006-002]

29. 29 Book structure 1. Existing networks 2. Upcoming networks 3. Trust 4. Naming and addressing 5. Security associations 6. Secure neighbor discovery 7. Secure routing 8. Privacy protection 9. Selfishness at MAC layer 10. Selfishness in PKT FWing 11. Operators in shared spectrum 12. Behavior enforcement Appendix A: Security and crypto Appendix B: Game theory SecurityCooperation

30. 30 Who is malicious? Who is selfish? There is no watertight boundary between malice and selfishness  Both security and game theory approaches can be useful There is no watertight boundary between malice and selfishness  Both security and game theory approaches can be useful Harm everyone: viruses,… Selective harm: DoS,…Spammer Cyber-gangster: phishing attacks, trojan horses,… Big brother Greedy operator Selfish mobile station

31. 31 From discrete to continuous Warfare-inspired Manichaeism: The more subtle case of commercial applications: Bad guys (they) Attacker Good guys (we) System (or country) to be defended 0 1 Undesirable behavior Desirable behavior 0 1 Security often needs incentives Incentives usually must be secured

32. 32 Another book: Cognitive Wireless Networks: Concepts, Methodologies and Visions - Inspiring the Age of Enlightenment of Wireless Communications – Edited by Frank H. P. Fitzek and Marcos D. Katz Part I:Introductory Chapter Part II:Cooperative Networks: Social, Operational and Communicational Aspects Part III:Cognitive Networks Part IV:Marrying Cooperation and Cognition in Wireless Networks Part V:Methodologies and Tools Part VI:Visions, Prospects and Emerging Technologies

33. 33 List of Contributors Aachen University, Germany AIST, Japan Budapest University of Technology and Economics, Hungary Create-NET, Italy DoCoMo Euro-Labs, Germany EPFL, Switzerland Harvard University, USA Konica Italia, Italy KTH, Sweden MIT, USA Motorola Labs, France Oxford University, UK Stevens Institute of Technology, USA Technical University of Catalonia, Spain Technical University of Denmark, Denmark Universität Duisburg Essen, Germany Universität Karlsruhe, Germany University of Aalborg, Denmark University of Bologna, Italy University of California at Berkeley, USA University of California, San Diego, USA University of Dresden, Germany University of Paderborn, Germany University of Padova, Italy University of Piraeus, Greece VTT, Finland WINLAB and Rutgers University, USA

34. 34 Spectrum Sharing Games in Unlicensed Band Licensed Band Cellular Operators WAN-WiFi competition/cooperation (University of Texas at Austin) Cellular operators near national borders (EPFL) Cellular operators in shared spectrum (EPFL) Unlicensed Band Devices Heterogeneous wireless systems (University of California at Berkeley) Wifi Operators (Bell LAB, MIT) Cognitive Radios Opportunistic spectrum sharing (UCSB) Auction based spectrum sharing (Northwestern) Multi-Cell OFDM spectrum sharing (University of Maryland) 3. Spectrum sharing games

35. 35 3.1 Spectrum Sharing Games of Network Operators 1.A. Zemlianov and G. de Veciana, “Cooperation and Decision Making in Wireless Multi-provider Setting,” INFOCOM 2005 2.M. Felegyhazi et al., “Border Games in Cellular Networks,” INFOCOM 2007 3.M. Felegyhazi and J.-P. Hubaux, “Wireless Operators in a Shared Spectrum,” INFOCOM 2006

36. 36 3.1.1 WAN-WiFi competition/cooperation (University of Texas at Austin)

37. 37 Switch decision Switch from WLAN to WAN Switch from WAN to WLAN N k h : number of users connected to hotspot h k S m w : number of users connected to base station w m c h : cost of switching to a hotspot AP c w : cost of switching to a WAN AP t: decision time

38. 38 Results Given any initial configuration of agents’ choices, the system converges to an equilibrium configuration as t  ∞ This equilibrium is not necessarily unique The class of payoff functions that are congestion dependent provide much better performance to users on average than the simple proximity-based decision strategy The results can notably help operators with both wireless WAN infrastructure (e.g., WiMAX) and a set of WiFi hotspots to optimize their network

39. 39 3.1.2 Border games of cellular networks (EPFL) Example: Mobile users in Geneva airport have access to two service providers in the same frequency. Other examples: - Slovenia and Croatia - Finland and Russia - USA and Mexico (San Diego) - USA and Canada (Detroit) - Jordan and Israel - Hong-Kong and China

40. 40 Two CDMA operators: A and B Adjust the pilot signals Power control game (no power cost): – players = operators – strategies = pilot powers – payoffs = attracted users (best SINR) where: – pilot processing gain  – pilot signal power of BS A  – path loss between A and v – own-cell interference factor  – other-to-own-cell interference factor  – traffic signal power assigned to w by BS A – set of users attached to BS A Signal-to-interference-plus-noise ratio: Own-cell interference Other-to-own-cell interference Border games of cellular operators

41. 41 Results (1): Incentives for operators to be strategic

42. 42 Results (2): Existence of a unique maximum payoff when both operators are strategic

43. 43 Unique and Pareto-optimal Nash equilibrium Higher pilot power than in the standard P s = 2W 10 users in total standard Nash equilibrium Results (3): Best response calculation for two players

44. 44 Other results 1.Price of anarchy and price of conformance to evaluate NE 2.Extend payoff function to consider the cost of high pilot power  leads to a Prisoner’s dilemma situation U – fair payoff (half of the users) D – payoff difference by selfish behavior C* - cost for higher pilot power

45. 45 3.1.3 Operators in a shared spectrum (EPFL) two operators: A and B set of base stations: B A and B B base stations are placed on the vertices of a grid each base station of A has the same radio range r A (relaxed later), same for B base stations emit pilot signals on the same channel, with the radio ranges: r A, r B full coverage by combination of the two operator’s coverage maximum power limit P MAX → R MAX if devices have omnidirectional antennas

46. 46 System model A set of users uniformly distributed in the area Free roaming Users attach to the base station with the best pilot signal Operators want to cover the largest area with their pilot signal where the channel gain:

47. 47 Power control game static game G = (P, S, U) Players → operators Strategy → pilot signal radio range Utility: coverage area of their own pilot signal minus the interference area where γ i is the cooperation parameter of player i: cooperativeness agreement power price

48. 48 Repeated game and punisher strategy Punisher strategy: Play RMIN in the first time step. Then for each time step: – play RMIN, if the other player played RMIN – play RMAX for the next ki time steps, if the other player played anything else

49. 49 What is the right cooperation model ? Non-cooperative games: no trust and no agreement between players Cooperative games: players talk to each other and try to find an agreement Cooperation is often assumed at the physical layer (e.g. for beamforming) Non-cooperative behavior is often (but not always) assumed at the MAC, network, and transport layers

50. 50 A glimpse at IEEE JSAC Cooperative Communications and Networking – Vol. 25 (2007), issue 2 Adaptive, Spectrum Agile and Cognitive Wireless Networks – Vol. 25 (2007), issue 3 Non-Cooperative Behavior in Networking – to appear in Q3 2007 Game Theory in Communication Systems – submission due date: August 1, 2007

51. 51 Panel at Mobicom 2007 (Montreal, September 13) Bonobos Vs Chimps: Cooperative and Non-Cooperative Behavior in Wireless Networks Panelists: Jean-Pierre Hubaux, EPFL (organizer and moderator) Ramesh Johari, Stanford University P. R. Kumar, UIUC Joseph Mitola, MITRE Corp. Heather Zheng, UCSB Bonobo Chimpanzee www.ncbi.nlm.nih.gov www.bio.davidson.edu

52. 52 Conclusion Potential of the area fuelled by: – The emergence of cognitive radios – The willingness to revisit the way spectrum is allocated – The rise of WiFi community networks Limitations of game theory modeling for wireless networks – Information for the players: games are in reality of incomplete and imperfect information – Expression of payoffs (benefits and costs) – Cooperative Vs non-cooperative games – Reputation of players http://winet-coop.epfl.ch