1. Game Theory & Cognitive Radio part A Hamid Mala

2. 2/28 Presentation Objectives 1. Basic concepts of game theory 2. Modeling interactive Cognitive Radios as a game 3. Describe how/when game theory applies to cognitive radio. 4. Highlight some valuable game models.

3. 3/28 Interactive Cognitive Radios Adaptations of one radio can impact adaptations of others Interactive Decisions Difficult to Predict Performance

4. 4/28 Interactive Cognitive Radios Scenario: Distributed SINR maximizing power control in a single cluster. Final state : All nodes transmit at maximum power. (1) the resulting SINRs are unfairly distributed (the closest node will have a far superior SINR to the furthest node) (2) battery life would be greatly shortened. Power SINR

5. 5/28 traditional analysis techniques Dynamical systems theory optimization theory contraction mappings Markov chain theory

6. 6/28 Research in a nutshell Applying game theory and game models (potential and supermodular) to the analysis of cognitive radio interactions – Provides a natural method for modeling cognitive radio interactions – Significantly speeds up and simplifies the analysis process – Permits analysis without well defined decision processes

7. Game Theory Definition, Key Concepts

8. 8/28 Exaple Same color winner opposite color winner $ = card number of winner

9. 9/28 Exaple Same color winner opposite color winner $ = card number of winner

10. 10/28 Exaple (2,-2)(-8,8) (-1,1)(7,-7) Matrix representation Girl’s strategiesBoy’s strategiesPay-off function

11. 11/28 Games A game is a model (mathematical representation) of an interactive decision situation. Its purpose is to create a formal framework that captures the relevant information in such a way that is suitable for analysis. Different situations indicate the use of different game models. 1. A set of 2 or more players, N 2. A set of actions for each player, A i 3. A set of utility functions, {u i }, that describe the players’ preferences over the outcome space Normal Form Game Model

12. 12/28 An action vector from which no player can profitably unilaterally deviate. An action tuple a is a NE if for every i  N for all b i  A i. Definition Nash Equilibrium

13. 13/28 Friend Foe Friend Foe 500,5000,1000 0,0 1000,0 (Friend, Friend)??No (Friend, Foe)?? Yes (Foe, Friend)??Yes (Foe, Foe)??Yes Friend or Foe Example

14. Modeling and Analysis Review

15. 15/28 Modeling a Network as a Game NetworkGame Nodes Power Levels Algorithms Players Actions Utility Functions Structure of game is taken from the algorithm and the environment [Laboratoire de Radiocommunications et de Traitement du Signal] voidupdate_power(void) { /*Adjusting power level*/ intk; }

16. 16/28 Modeling Review The interactions in a cognitive radio network can be represented by the tuple: Timings: – Synchronous – Round-robin – Random – Asynchronous Dynamical System

17. 17/28 1. Steady state characterization 2. Steady state optimality 3. Convergence 4. Stability 5. Scalability a1a1 a2a2 NE1 NE2 NE3 a1a1 a2a2 NE1 NE2 NE3 a1a1 a2a2 NE1 NE2 NE3 a1a1 a2a2 NE1 NE2 NE3 a3a3 Steady State Characterization Is it possible to predict behavior in the system? How many different outcomes are possible? Optimality Are these outcomes desirable? Do these outcomes maximize the system target parameters? Convergence How do initial conditions impact the system steady state? What processes will lead to steady state conditions? How long does it take to reach the steady state? Stability How does system variations impact the system? Do the steady states change? Is convergence affected? Scalability As the number of devices increases, How is the system impacted? Do previously optimal steady states remain optimal? Key Issues in Analysis

18. 18/28 How Game Theory Addresses These Issues Steady-state characterization – Nash Equilibrium existence – Identification requires side information Steady-state optimality – In some special games Convergence – in some cases Stability, scalability – No general techniques – Requires side information

19. 19/28 Nash Equilibrium Identification Time to find all NE can be significant Let t u be the time to evaluate a utility function. Search Time: Example: – 4 player game, each player has 5 actions. – NE characterization requires 4x625 = 2,500 t u Desirable to introduce side information.

20. 20/28 Example(1) : The Cognitive Radios’ Dilemma Example : The Cognitive Radios’ Dilemma Frequency domain representation of waveforms The Cognitive Radios’ Dilemma in Matrix NE=? Two cognitive radios Each radio can implement two different waveforms low-power narrowband higher power wideband

21. 21/28 Repeated Games and Convergence Repeated Game Model – Consists of a sequence of stage games which are repeated a finite or infinite number of times. – Most common stage game: normal form game. Finite Improvement Path (FIP) – From any initial starting action vector, every sequence of round robin better responses converges. Weak FIP – From any initial starting action vector, there exists a sequence of round robin better responses that converge.

22. 22/28 Better Response Dynamic During each stage game, player(s) choose an action that increases their payoff, presuming other players’ actions are fixed. Converges if stage game has FIP. a b A B 1,-1 -1,1 0,2 2,2

23. 23/28 Best Response Dynamic During each stage game, player(s) choose the action that maximizes their payoff, presuming other players’ actions are fixed. converge if stage game has weak FIP. a b A B 1,-1 -1,1 1,-1 -1,1 C 0,2 1,2 c 2,1 2,0 2,2

24. 24/28 Supermodular Games Key Properties – Best Response (Myopic) Dynamic Converges – Nash Equilibrium Generally Exists Why We Care – Low level of network complexity How to Identify

25. 25/28 Supermodulaar Games NE Existence: have at least one NE. NE Identification: all NE for a game form a lattice. While this does not particularly aid in the process of initially identifying NE, from every pair of identified Convergence: have weak FIP, so a sequence of best responses will converge to a NE. Stability: if the radios make a limited number of errors or if the radios are instead playing a best response to a weighted average of observations from the recent past, play will converge.

26. 26/28 Example : outer loop power control Parameters – Single Cluster – P i = P j = [0, P max ]  i,j  N – Utility target SINR Supermodular – best response convergence

27. 27/28 Summary When we use game theory to model and analyse interactive CRs, it should address : – steady state existense and identification – convergence – stability – desirability of steady states Supermodular games : to some extent

28. Questions?

29. Game Theory & Cognitive Radio part B Mahdi Sadjadieh

30. 30/28 Overview Potential Game Model Type of Potential Game Example of Exact Potential Game FIP and Potential Games How Potential Games handle the shortcomings Physical Layer Model Parameters and Potential Game

31. 31/28 Potential Game Model Identification NE Properties (assuming compact spaces) – NE Existence: All potential games have a NE – NE Characterization: Maximizers of V are NE Convergence – Better response algorithms converge. Stability – Maximizers of V are stable Design note: – If V is designed so that its maximizers are coincident with your design objective function, then NE are also optimal.  Existence of a potential function V such that

32. 32/28 Potential Games Existence of a function (called the potential function, V), that reflects the change in utility seen by a unilaterally deviating player. E1 E2 E3 E4 G  PG  GOPG (Gilles) OPG  G  PG (finite A)

33. 33/28 Potential Games

34. 34/28 Ordinal Potential Game Identification Lack of weak improvement cycles [Voorneveld_97] FIP and no action tuples such that Better response equivalence to an exact potential game [Neel_04] Not an OPG An OPG

35. 35/28 Ordinal Potential Game Identification Lack of weak improvement cycles [Voorneveld_97] FIP and no action tuples such that Better response equivalence to an exact potential game [Neel_04] Not an OPG An OPG

36. 36/28 Other Exact Potential Game Identification Techniques Linear Combination of Exact Potential Game Forms [Fachini_97] – If and are EPG, then is an EPG Evaluation of second order derivative [Monderer_96]

37. 37/28 Exact Potential Game Forms Many exact potential games can be recognized by the form of the utility function

38. 38/28 Example Identification Single cluster target SINR Better Response Equivalent Dummy gameBSI gameSelf-motivated game

39. 39/28 FIP and Potential Games GOPG implies FIP ([Monderer_96]) FIP implies GOPG for finite games ([Milchtaich_96]) Thus we have a non-exhaustive search method for identifying when a CRN game model has FIP. Thus we can apply FIP convergence (and noise) results to finite potential games.

40. 40/28 Steady-states As noted previously, FIP implies existence of NE

41. 41/28 Optimality If u i are designed so that maximizers of V are coincident with your design objective function, then NE are also optimal. (*) Can also introduce cost function to utilities to move NE. In theory, can make any action tuple the NE – May introduce additional NE – For complicated NC, might as well completely redesign u i V a

42. 42/28 Convergence in Infinite Potential Games  -improvement path – Given  >0, an  -improvement path is a path such that for all k  1, u i (a k )>u i (a k-1 )+  where i is the unique deviator at step k. Approximate Finite Improvement Property (AFIP) – A normal form game, , is said to have the approximate finite improvement property if for every  >0 there exists an such that the length of all  -improvement paths in  are less than or equal to L. [Monderer_96] shows that exact potential games have AFIP, we showed that AFIP implies a generalized  -potential game.

43. 43/28 Convergence Implications

44. 44/28 How potential games handle the shortcomings Steady-states – Finite game NE can be found from maximizers of V. Optimality – Can adjust exact potential games with additive cost function (that is also an exact potential game) – Sometimes little better than redesigning utility functions Game convergence – Potential game assures us of FIP (and weak FIP) – D V satisfy Zangwill’s (if closed) Noise/Stability – Isolated maximizers of V have a Lyapunov function for decision rules in D V Remaining issue: – Can we design a CRN such that it is a potential game for the convergence, stability, and steady-state identification properties – AND ensure steady-states are desirable?

45. More Examples

46. 46/28 Physical Layer Model Parameters

47. 47/28 Assume that there is a radio network wherein each radio can alter their power. Assume each radio reacts to some separable function of SINR, e.g. log ratio Each radio would also like to minimize power consumption SINR Power Control Games Decentralized Power Control Using a dB Metric Thus game is a potential game and convergence is assured and we can quickly find steady states.

48. 48/28 Example Power Control Game Parameters – Single Cluster – DS-SS multiple access – P i = P j = [0, P max ]  i,j  N – Utility target BER Also a potential game.

49. 49/28 Snapshot inner + outer loop power control Parameters – Single Cluster – DS-SS multiple access – P i = P j = [0, P max ]  i,j  N – Utility target SINR Supermodular – best response convergence

50. 50/28 Game Models, Convergence, and Complexity Determining the kind of game required to accurately model a RRM algorithm yields information about what updating processes are appropriate and thus indicates expected network complexity. In [Neel04] the following relation between power control algorithms, game models, and network complexity was observed.

51. 51/28 Summary Distributed dynamic resource allocations have the potential to provide performance gains with reduced overhead, but introduce a potentially problematic interactive decision process. Game theory is not always applicable. Can generally be applied to distributed radio resource management schemes.

52. Questions?

53. 53/28 Example:Exact Poential Game return

54. 54/28 return

55. 55/28 Example : Ordinal Poential Game return

56. 56/28 Example : Generalized Ordinal Poential Game return

57. 57/28 Exact Potential Game Forms Many exact potential games can be recognized by the form of the utility function