Comp/Math 553: Algorithmic Game Theory Lecture 05 Fall 2016 Yang Cai

Overview so far Recap: Games, rationality, solution concepts Existence Theorems for Nash equilibrium: Nash’s theorem for general games (via Brouwer) von Neumann’s theorem for 2-player ZS games (via LP duality) This lecture: algorithms for Nash equilibrium beyond two player zero-sum games (which can be solved using linear programming) Overarching goal: How complex computationally is it to find the Nash equilibrium of the game, for the game theorist and the players themselves?

Algorithms for Nash Equilibria Support Enumeration Algorithms Algorithms for Symmetric Games The Lemke-Howson Algorithm

Algorithms for Nash Equilibria Support Enumeration Algorithms Algorithms for Symmetric Games The Lemke-Howson Algorithm

Support Enumeration Algorithms How better would my life be if I knew the support of the Nash equilibrium? … and the game is 2-player? Setting: Let (R, C) be an m by n game, and suppose a friend revealed to us the supports and respectively of the Row and Column players’ mixed strategies at some equilibrium of the game. Claim: Can find Nash equilibrium (x, y) using linear programming. s.t. and

Support Enumeration Algorithms How better would my life be if I knew the support of the Nash equilibrium? … and the game is 2-player? Runtime: for guessing the support of each player’s strategy at a Nash eq for solving the LP Corollary: Existence of rational equilibria in 2-player games. Proof: Follows from the correctness of the support enumeration algorithm. If there is a Nash equilibrium with supports SR and SC for the two players, then the polytope of the corresponding LP is non-empty. Any vertex of that polytope is a Nash equilibrium, whose coordinates are rational and whose bit complexity is polynomial in the description of the game.

Support Enumeration for n-player games How better would my life be if I knew the support of the Nash equilibrium? … and the game is n-player? Challenge: Even if the support Σp of each player at a Nash equilibrium is given to us, the problem we need to solve is still not an LP. Polynomial of degree (n-1) in variables x11, x12,…, xnk (suppose k strategies per player) So need to solve a system of polynomial equations and inequalities on n k variables, where the involved polynomials have degree n-1. Can be done to accuracy ε in time polynomial in nnk log (1/ε). (using tools from the existential theory of the reals) Overall time: polynomial in nnk log (1/ε) (c.f. input size: n kn = n 2n log k )

Algorithms for Nash Equilibria Support Enumeration Algorithms Algorithms for Symmetric Games The Lemke-Howson Algorithm

number of the other players choosing each strategy in S Symmetries in Games Def: An n-player game is symmetric iff : - all players have the same strategy set: S = {1,…, k} - there exists a function f such that every player’s utility can be written as: up(s) = f (sp ; n1(s-p),…,nk(s-p) ) number of the other players choosing each strategy in S E.g. : - Rock-Paper-Scissors, guess 2/3 of the average - congestion games, with same source destination pairs for each player Description Size: O(min {k nk-1, kn }) Thm [Nash ’51]: Always exists a Nash equilibrium in which every player uses the same mixed strategy.

Existence of a Symmetric Equilibrium Recall Nash’s function: Thought Experiment: restrict Nash’s function on the set: crucial observation: Nash’s function maps points of the above set to itself as every player will perform the same update

Algorithms for n-player symmetric games A symmetric equilibrium x = (x1, x2, …, xk) can be found as follows [Papadimitriou-Roughgarden ’04]: using tools from the existential theory of the reals - guess the support of x : 2k possibilities can be solved approximately in time polynomial in nk log(1/ε) - write down a set of polynomial equations and inequalities corresponding to the equilibrium conditions, for the guessed support - polynomial equations and inequalities of degree n in k variables (cf kn for general games) polynomial in the size of the input for k up to about n (n1-δ for all δ )

Symmetrization [Gale-Kuhn-Tucker 1950] x y x 0, 0 R, C R , C CT, RT y / |y|1 w.l.o.g. suppose that R, C have positive entries x 0, 0 R, C x / |x|1 R , C CT, RT 0, 0 y Equilibrium Symmetric Equilibrium In fact we show that Equilibrium Any Equilibrium Proof: On the board.

Symmetrization R , C RT,CT C, R x y Symmetric Equilibrium Equilibrium 0,0 Any Equilibrium In fact […] Hence, essentially as hard to solve symmetric 2-player games as it is to solve general 2-player games Open: - Reduction from 3-player games to symmetric 3-player games?

Algorithms for Nash Equilibria Support Enumeration Algorithms Parenthesis: Symmetric Games The Lemke-Howson Algorithm

Polytopes 101 Convex Polytope in Rn: The intersection of n-dimensional hyperplanes (or half-spaces) of the form aT  x ≥ c, where a is a vector in Rn and c a scalar. Often described compactly in matrix form: A x ≥ b, where A is an m  n matrix and b is a vector in Rn. This inequality specifies the polytope that is the intersection of half-spaces: AiT  x ≥ bi , i=1,…,m (*) Extreme-point of a Convex Polytope: A point x such that (*) is satisfied with at least n (linearly-independent1) inequalities tight. Edge of a Convex Polytope: Subset of the polytope where n-1 (linearly independent) inequalities are tight. 1 a collection of inequalities BiT  x ≥ di , i = 1, …, k are called linearly independent if the vectors B1,…, Bk are linearly independent vectors of Rn

The Lemke-Howson Algorithm (1964) Problem: Find an exact equilibrium of a 2-player game. Since there exists a rational equilibrium this task is feasible. Assumption (w.l.o.g.): The game given in the input is a symmetric game, i.e. Idea of LH: Perform pivoting steps between the corners of a polytope related to the game until a Nash equilibrium is found. Polytope of Interest: Assumption 2 (w.l.o.g): At every corner of the polytope exactly n out of the 2n inequalities are tight. (perturb original payoff entries with exponentially small noise to achieve this; equilibria of the new game are approximate eq. of original game of very high accuracy, and these can be converted to exact equilibria (exercise) )

The Lemke-Howson Algorithm Def: Pure strategy i is represented at a corner z of the polytope if at least one of the following is tight: At corner (0,0,…,0) all pure strategies are present. Call any corner of the polytope where this happens a democracy. Lemma: If a vertex z≠0 of the polytope is a democracy, then is a Nash eq. Proof: At a democracy we have the following implication: Hence:

The Lemke-Howson Algorithm Start at the corner (0,0,…,0). By non-degeneracy there are exactly n edges of the polytope adjacent to the (0,0,…,0) corner. Each of these edges corresponds to un-tightening one of the inequalities. Select an arbitrary pure strategy, say pure strategy n, and un-tighten . This corresponds to an edge of the polytope adjacent to 0. Jump to the other endpoint of this edge. If the obtained vertex z is a democracy, then a Nash equilibrium has been found because z≠0. Otherwise, one of the strategies 1,…, n-1, say strategy j, is represented twice, by both was already tight just became tight I will untighten one of the above. What happens if I require ? Question: A: I am going to return to (0,0,…,0), since I would be walking on the edge of the polytope that brought me here. So let me untighten the other one, requiring .

The Lemke-Howson Algorithm If the obtained vertex is a democracy, then a Nash equilibrium has been found. Otherwise, one of the strategies 1,…, n-1, is represented twice. This strategy is doubly represented because one of its inequalities was tight before the step, and the other one became tight after the step was taken. To proceed, un-tighten the former. This defines a directed walk on the vertices of the polytope, starting at the democracy (0,0,…,0), and with every intermediate vertex having all of 1,…, n-1 represented, and exactly one of them represented twice. The walk proceeds by un-tightening one of the two inequalities of the doubly represented strategy, namely the one that does not bring it back to where it came from. The walk can keep going unless a democracy is encountered. Claim: The walk will settle on a democracy ≠ 0. Proof: Next slide.

Proof that Lemke-Howson Terminates Claim: The walk will settle on a democracy ≠ 0. Proof: Consider the vertices of the polytope that are either democracies or have only 1,…, n-1 represented (and exactly one of them represented twice). Define a graph whose nodes are the aforementioned vertices, and whose edges are defined as follows: a vertex where n is not represented and where j is represented twice has two neighbors corresponding to the vertices reached when un-tightening either zj ≥ 0 or Rj z ≤1 a vertex that is a democracy has one neighbor corresponding to un-tightening whichever of zn≥ 0 or Rn z ≤1 is tight Clearly every node in this graph has degree ≤ 2, hence it comprises paths and cycles. In fact, nodes with degree 1 are democracies and only democracies have degree 1. (0,0,…,0) has degree 1 so it is sitting on a path of this graph, whose other endpoint is another democracy ≠0 Lemke-Howson visits the vertices on the path where (0,0,…,0) sits, by the definition of the graph and the definition of the algorithm.

Lemke-Howson Example 3 0 0 2 2 2 0 3 0

Post Mortem The Lemke-Howson algorithm: - provides an alternative proof that a Nash equilibrium exists in 2-player games; - moreover, it shows that there always exists a rational equilibrium in 2-player games; it works by virtue of a parity argument: it identifies a directed path on the vertices of the polytope; since that path has a source, it must also have a sink. worst-case running time: exponential in the number of strategies [Savani-von Stengel’04]. there are analogs of the Lemke-Howson algorithm for multi-player games working with manifolds instead of polytopes ( [Rosenmuller ’71] and [Wilson ’71])