1. Equilibrium Concepts in Two Player Games Kevin Byrnes Department of Applied Mathematics & Statistics

2. Nash Equilibria A set of strategies (x*,y*) in a two player game is a Nash equilibrium point if: f(x*,y*) >=f(x,y*) for all x in S 1 g(x*,y*)>=g(x*,y) for all y in S 2

3. Nash Equilibria More generally, a set of strategies (x 1 *,…,x n *) in an n player game is a Nash equilibrium point if: f 1 (x 1 *,…,x n *)>=f 1 (x,...,x n *) for all x in S 1 f n (x 1 *,…,x n *)>=f n (x 1 *,…,x) for all x in S n

4. Nash Equilibria The key properties of a Nash equilibrium are that it is enforceable and guaranteed to exist under certain reasonable assumptions.

5. Nash Equilibria Enforceability comes from the fact that every player is unilaterally maximizing his payoff, ie no player can single-handedly deviate to do better. To see why it must exist, consider the following sketch of a proof:

6. Nash Equilibria Lemma 1: If f 1,…,f n are concave, then the best response sets are convex Lemma 2: If f 1,…,f n are also continuous and the strategy spaces are compact, then the best response sets form an upper hemi continuous correspondence

7. Nash Equilibria Existence Theorem (Nash): Assuming that the f i are continuous and concave, and that the strategy spaces S i are compact, then a Nash equilibrium exists. Proof: By Lemmas 1 and 2 the BR i (.) are upper hemi continuous correspondences on compact sets. By Kakutani’s Fixed Point Thm. we then know that we must have a fixed point.

8. Nash Equilibria Now let us consider a special type of game, namely a bimatrix game (ie both players have a finite number of pure strategies and payoff matrices) Let A be player 1’s payoff matrix, so A ij =f(s i,s j ), where s i is a pure strategy in S 1, and s j is a pure strategy in S 2. Let B be player 2’s payoff matrix, so B ij =g(s i,s j )

9. Nash Equilibria Now let x i denote the probability with which player 1 plays s i, and let y j denote the probability with which player 2 plays s j. Then a Nash equilibrium is a pair of strategies (x*,y*) such that x* satisfies (i) and y* satisfies (ii), where we define (i) and (ii) as:

10. Nash Equilibria (i) Max x x T Ay* Subject to: x 1 +…+x m =1 x>=0 (ii) Max y x* T By Subject to: y 1 +…+y n =1 y>=0

11. Nash Equilibria By our existence proof, we know that such equilibria exist, the question is, how can we find them?

12. Nash Equilibria By our existence proof, we know that such equilibria exist, the question is, how can we find them? It turns out that Vorob’ev, Kuhn, Lemke, and Howson (inter alia) have proposed algorithms for finding Nash equilibrium points for the special case of bimatrix games. Computing such equilibria may be expensive, however. Thus we shall now focus on a key geometric result of Mangasarian that tells us which equilibrium points we really need to generate.

13. The Geometry of Equilibria For a bimatrix game, finding an equilibrium point is equivalent to simultaneously solving problems (i) and (ii). But each of these is just an LP.

14. The Geometry of Equilibria For a bimatrix game, finding an equilibrium point is equivalent to simultaneously solving problems (i) and (ii). But each of these is just an LP. Now recall that to solve a single LP, we just need to look at the extreme points of the feasible region, could we be so lucky here?

15. The Geometry of Equilibria Before proceeding, it is useful to note that if a pure strategy Nash equilibrium exists in a bimatrix game, it may be found in a straightforward fashion. Consider the following game: UpDownLeft(4,2)(2,3) Right(6,-1)(0,0)

16. The Geometry of Equilibria A yellow box indicates the row player’s best response to a given column strategy. A red box indicates the column player’s best response to a given row strategy. UpDownLeft(4,2)(2,3) Right(6,-1)(0,0)

17. The Geometry of Equilibria A yellow box indicates the row player’s best response to a given column strategy. A red box indicates the column player’s best response to a given row strategy. UpDownLeft(4,2)(2,3) Right(6,-1)(0,0)

18. The Geometry of Equilibria A Nash equilibrium exists at the intersection of any of these two best responses. UpDownLeft(4,2)(2,3) Right(6,-1)(0,0)

19. The Geometry of Equilibria First recall problems (i) and (ii): (i) Max x x T Ay* Subject to: e T x=1 x>=0 (ii) Max y x* T By Subject to: d T y=1 y>=0 Where e and d are the appropriate vectors of all ‘1’s

20. The Geometry of Equilibria Now let w* equal x* T Ay*, and let z* equal x* T By* for a specific (x*,y*) solution of (i) and (ii). Then we have the following: Equivalence Theorem: A necessary and sufficient condition that (x*,y*,w*,z*) be a solution of (i) and (ii) is that it is a solution of the programming problem: (iii) Max x,y,w,z {x T (A+B)y-w-z|(x,z) is in S, and (y,w) is in T}

21. The Geometry of Equilibria Where: S={(x,z)|B T x-zd =0} T={(y,w)|Ay-we =0} Note that S and T are both convex polyhedral sets.

22. The Geometry of Equilibria Now observe that: (iv) x* T (A+B)y*-w*-z*=0 In fact, by the Equivalence Theorem, any set of (x*,y*,w*,z*) that satisfy (iv) with (x*,z*) in S and (y*,w*) in T satisfy (iii).

23. The Geometry of Equilibria Now we shall define an extreme equilibrium point (x*,y*,w*,z*) as a point that satisfies (iv), and for which (x*,z*) is a vertex of S, and (y*,w*) is a vertex of T. Observe that by definition, there exist only a finite number of extreme equilibrium points, as S and T only have a finite number of extreme points.

24. The Geometry of Equilibria Lemma (Mangasarian): All equilibrium points of a bimatrix game may be expressed as convex combinations of some extreme equilibrium points. Proof: Let (x*,y*,w*,z*) be a solution of (iii). Now if we set y=y*, and w=w*, then (iii) reduces to a linear programming problem in x* and z*. This implies, by the extreme point characterization of all solutions of an LP, that all solutions (x,y*,w*,z) must be convex combinations of some subset U of S.

25. The Geometry of Equilibria Thus each vertex (x,z) of U is a solution of our modified (iii), and so satisfies (iv): (v) x T (A+B)y*-w*-z=0 In a similar fashion, we see that (y*,w*) must have been a convex combination of vertices in V, a subset of T. So (v) is equal to: (vi) x T (Ay*-w*e)+y* T (B T x-zd)=0, for (x,z) in U

26. The Geometry of Equilibria Now note that x>=0, y>=0, and Ay*-w*e =0, y>=0, and Ay*-w*e<=0 and B T x-zd<=0, since (y*,w*) is in V and (x,z) is in U. So (vi) implies: (vii) y* T (B T -zd)=0 for (x,z) in U

27. The Geometry of Equilibria Now note that x>=0, y>=0, and Ay*-w*e =0, y>=0, and Ay*-w*e<=0 and B T x-zd<=0, since (y*,w*) is in V and (x,z) is in U. So (vi) implies: (vii) y* T (B T -zd)=0 for (x,z) in U Since (y*,w*) is a convex combination of points in V, (vii) implies that: (viii) y T (B T x-zd)=0 for (x,z) in U and some (y,w) in V

28. The Geometry of Equilibria Similarly, we have (can show) that: (ix) x T (Ay-we)=0 for (y,w) in V and some (x,z) in U This gives us that: x T (Ay-we)+y T (B T x-zd)=0 ie: x T (A+B)y-w-z=0 for some (y,w) in V and some (x,z) in U for some (y,w) in V and some (x,z) in U Which proves the claim.

29. NonInferior Equilibria A set of strategies (x*,y*) in a two player game is a noninferior equilibrium point if: There does not exist x,y in S 1 XS 2 such that: f(x,y)>f(x*,y*) and g(x,y) >= g(x*,y*) or f(x,y) >= f(x*,y*) and g(x,y)>g(x*,y*)

30. NonInferior Equilibria An example of noninferior equilibria: Suppose that we are given the following (x,y) pairs and their associated payoffs: f(x,y)g(x,y) (x 1,y 1 ) 262 (x 2,y 2 ) 1212 (x 3,y 3 ) -612 (x 4,y 4 ) 00

31. NonInferior Equilibria The noninferior points are (x 1,y 1 ) and (x 2,y 2 ) f(x,y)g(x,y) (x 1,y 1 ) 262 (x 2,y 2 ) 1212 (x 3,y 3 ) -612 (x 4,y 4 ) 00

32. NonInferior Equilibria The noninferior points are (x 1,y 1 ) and (x 2,y 2 ) f(x,y)g(x,y) (x 1,y 1 )262 (x 2,y 2 )12 (x 3,y 3 )-612 (x 4,y 4 )00

33. NonInferior Equilibria For a bimatrix game, a noninferior set of strategies (x*,y*) is a pair that satisfies the multiobjective problem: (x) Max x x T Ay*, Max y x* T By Subject to: x 1 +…+x m =1 y 1 +…+y n =1 x>=0y>=0

34. NonInferior Equilibria Not every Nash equilibrium is noninferior however, for example, the Prisoners’ Dilemma Confess Don’t Confess Confess(-2,-2)(0,-10) (-10,0)(-.5,-.5)

35. NonInferior Equilibria Here the unique NE is (Confess, Confess), but its payoff (-2,- 2) is strictly inferior to (-.5,-.5) that of (Don’t Confess, Don’t Confess) Confess Don’t Confess Confess(-2,-2)(0,-10) (-10,0)(-.5,-.5)

36. NonInferior Equilibria Here the unique NE is (Confess, Confess), but its payoff (-2,- 2) is strictly inferior to (-.5,-.5) that of (Don’t Confess, Don’t Confess) Confess Don’t Confess Confess(-2,-2)(0,-10) (-10,0)(-.5,-.5)

37. NonInferior Equilibria The previous example demonstrates that noninferiority may be a better solution concept than Nash equilibria, as it is a true maximizer. The downside is that a noninferior equilibrium may note be enforceable. In our previous example, both players could benefit by unilaterally deviating from the noninferior equilibrium of (Don’t Confess, Don’t Confess) This begs the question, can we perturb the payoffs in a ‘nice’ manner to impose enforcability?

38. Future Directions 1) Can we find an appropriate penalty to transform the noninferior equilibria of some or all games into enforceable ones? 2) Suppose that a given player knows only one of A or B, is there an evolutionary strategy that maximizes his expected payoff if the game is repeated infinitely often? What about finitely often?