A Projection Framework for Near- Potential Polynomial Games Nikolai Matni Control and Dynamical Systems, California Institute of Technology IEEE CDC Maui, December 13 th 2012

Motivation – Potential Games Informal definition: local actions have predictable global consequences. Nice properties – Pure-strategy Nash Equilibria (NE) – Simple dynamics converge to these NE Applications to distributed control – Marden, Arslan & Shamma 2010 – Candogan, Menache, Ozdaglar & Parrilo 2009 – Li & Marden, 2011

Motivation – Polynomial Games Would like to consider general class of continuous games – Finite players, continuous action sets. Why? – Goal is control: most systems of interest are analog. – Quantization leads to tradeoffs in granularity, performance and problem dimension. Why not? – Potentially intractable to analyze (Parrilo 2006, Stein et al for recent results). – Can lead to infinite dimensional optimization problems. Solution? – Restrict ourselves to polynomial cost functions and use Sum Of Squares (SOS) methods.

Motivation – Near Potential Games O. Candogan, A. Ozdalgar, P.A. Parrilo, A Projection Framework for Near-Potential Games, CDC 2010 (and subsequent work) Basic idea: if a game is “close” to being a potential game, it behaves “almost as well.” Projection Framework – finite dimensional case – Potential games form a subspace. – Project onto this framework to find closest potential game. – If distance from subspace is small, original game inherits many nice properties. Goal: Extend these ideas to polynomial games.

Outline Motivation – Potential games – Polynomial games – Near-Potential games Preliminaries – Game Theory – Algebraic Geometry/Sum of Squares (SOS) Projection Framework Properties – Static – Dynamic Example Conclusions and Future work

Outline Motivation – Potential games – Polynomial games – Near-Potential games Preliminaries – Game Theory – Algebraic Geometry/Sum of Squares (SOS) Projection Framework Properties – Static – Dynamic Example Conclusions and Future work

Prelims – Polynomial Game A polynomial game is given by: – A finite player set – Strategy spaces, where – Polynomial utility functions, A polynomial game is: – Continuous if for all n, is a closed interval of the real line – Discrete if for all n, – Mixed if some strategy sets are continuous, and some are discrete. – Assume w.l.o.g.

Prelims – Potential Games A polynomial game G is a polynomial potential game if there exists a polynomial potential function such that, for every player n, and every Algebraic characterization (Monderer, Shapley ’96): A continuous game is a potential game iff

Prelims – Misc. Game Theory A strategy is an approximate Nash (or ε) Equilibrium if, for all n, we have that

Prelims – SOS and p(x)≥0 Definition: a real polynomial p(x) admits a Sum Of Squares (SOS) decomposition if Why SOS? – Determining if p(x)≥0, is in general, NP-hard – Determining if p(x) is SOS tested through SDP Lemma [SOS relaxation]: If there exist SOS polynomials such that then

Outline Motivation – Potential games – Polynomial games – Near-Potential games Preliminaries – Game Theory – Algebraic Geometry/Sum of Squares (SOS) Projection Framework Properties – Static – Dynamic Example Conclusions and Future work

Projection Framework – MPD & MDD Need a notion of distance in the space of games Candogan et al. introduced Maximum Pairwise Distance (MPD) Use the continuity of polynomials to define Maximum Differential Difference (MDD) Both capture how different two games are in terms of utility improvements due to unilateral deviations

Projection Framework Task: Given a polynomial game, find a nearby potential polynomial game Formulate as an optimization problem: Constraint ensures we get a Potential Game Objective function minimizes MDD. Intractable!

Projection Framework – Convexify Step 1: rewrite constraint in terms of algebraic characterization Step 2: introduce slack variable γ

Projection Framework – Convexify Step 3: apply Lemma [SOS relaxation] This is a finite dimensional SOS program, solvable in polynomial time. It yields a polynomial potential game satisfying

Projection Framework - Extensions Can extend this idea to mixed/discrete games Lemma [MPD]: If, then Continuous Relaxations: For a mixed or discrete game, set all strategy sets to [-1,1] – Apply previous SOS program and Lemma [MPD] to mixed games or discrete games with – Allows us to apply algebraic characterization, which can reduce number of constraints from O( ) to O(N)

Outline Motivation – Potential games – Polynomial games – Near-Potential games Preliminaries – Game Theory – Algebraic Geometry/Sum of Squares (SOS) Projection Framework Properties – Static – Dynamic Example Conclusions and Future work

Properties – Static Let and be such that. Then for every ε 1 -equilibrium y of, z(y) is an ε-equilibrium of, where For continuous games, D=0, z(y)=y, and local maxima of P are pure (ε=0) NE.

Properties – Static Let and be such that. Then for every ε 1 -equilibrium y of, z(y) is an ε-equilibrium of, where For continuous games, D=0, z(y)=y, and local maxima of P are pure (ε=0) NE.

Properties – Dynamic Definition: ε-better response dynamics – Round robin updates – Player updates only to improve utility by at least ε – Otherwise does not update Suppose there exists such that Then, under ε-better response dynamics, after a finite number of iterations, dynamics will be confined to the ε-equilibria set of, for arbitrary.

Outline Motivation – Potential games – Polynomial games – Near-Potential games Preliminaries – Game Theory – Algebraic Geometry/Sum of Squares (SOS) Projection Framework Properties – Static – Dynamic Example Conclusions and Future work

Example – Distributed Power Consider the N player game defined by – Distributed power minimization interpretation

Example – Distributed Power Run through projection framework to find nearby potential game : satisfying

Example – Distributed Power Potential function concave – can compute global maximum to identify.2-equilibria of G Alternatively, can run.2-better response dynamics to converge to a.2-equilibria of G. Quantify performance through cost function

Example – Distributed Power Compare better-response (x br ) to centralized (optimal x * ) positions Better response comes within ~20% of centralized solution Completely decentralized Arbitrarily scalable Requires no a priori knowledge of base station locations

Conclusions & Future Work Introduce framework for analyzing polynomial games – Defined MDD and a tractable projection framework to find nearby potential games – Related static and dynamic properties of polynomial games to those of nearby potential games – Illustrated these methods on a distributed power problem Future work – Projecting onto weighted polynomial games – Additional static properties (mixed-equilibria) – Efficiency notions (price of anarchy, price of stability, etc.)