S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Computing Bayes-Nash Equilibria through Support Enumeration Methods in Bayesian Two-Player Strategic-Form Games Sofia Ceppi, Nicola Gatti, and Nicola Basilico Dipartimento di Elettronica e Informazione, Politecnico di Milano {ceppi, ngatti,

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Outline State of the Art –What is a Bayesian game –Why to study Bayesian games Original Contributions –Extensions of existing algorithms for Bayesian games –B-PNS algorithm Experimental Evaluation Conclusions and Future Contributions

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Bayesian Games What is a Bayesian Game? –Non-cooperative game –A game wherein information is uncertain Type 2.1 2, 7 9, 4 3, 5 2, 3 a b cd ω 2.1 = 0.3 Type 2.2 2, 7 9, 8 3, 5 1, 3 a b cd ω 2.2 = 0.7 ? ? Player 2 Player 1

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Bayesian Games Why to study Bayesian Games? –Most real world strategic situations present uncertainty and therefore can be modeled as Bayesian games, e.g., Negotiation settings: bilateral bargaining and auctions Security settings: strategic mobile robot patrolling –The literature does not study algorithms for computing Bayes- Nash equilibria in depth [Shoham and Leyton-Brown, 2008]

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano State of the Art Solution concept for Bayesian games is Bayes-Nash equilibrium A Bayesian game is solved by reducing it to a complete-information game and then computing a Nash equilibrium in this game The literature provides a detailed comparison of the algorithms for the computation of Nash equilibria in complete-information games The exact algorithms for two-player complete-information strategic- form games are: –LH: based on linear complementary programming [Lemke- Howson, 1964] –PNS: based on support enumeration [Porter et al., 2004] –SGC: based on mixed integer linear programming [Sandholm et al., 2005]

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Bayesian Game Peculiarities The experimental results provided from the literature for computing Nash equilibria cannot be generalized to Bayesian case. The main reasons are: –Bayesian games can present characteristics (e.g., existence of equilibria with small supports) different from those of complete- information games –The reduction to complete-information games raises several problems in the application of algorithms for computing Nash equilibria [Koller and Megiddo, 1996]

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Original Contributions Extension of the algorithms existing in the literature for the computation of Bayes-Nash equilibrium –PNS B-PNS (the main result) –LH B-LC –SGC B-SGC

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano PNS Algorithm The support S i of an agent i is the set of actions played by i with non- null probability The joint support S is the set of single agents support To derive the B-PNS algorithm we modified all the three parts of PNS algorithm STEP 1: Choosing S (Enumeration Criteria) STEP 1: Choosing S (Enumeration Criteria) STEP 2: Pruning (Conditional Dominance) STEP 2: Pruning (Conditional Dominance) STEP 3: Equilibrium Checking (Feasibility Problem) STEP 3: Equilibrium Checking (Feasibility Problem) not dominated feasible dominated not feasible

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Supports Supports for player 1: S 1 =(a), S 1 =(b),S 1 =(a,b) Supports for type 1 of player 2: S 2.1 =(c), S 2.1 =(d),S 2.1 =(c,d) Supports for type 2 of player 2: S 2.2 =(c), S 2.2 =(d),S 2.2 =(c,d) Joint support: S={S 1,S 2.1,S 2.2 } S={ (a), (d), (c,d) } Goal: enumerate the joint supports and check if they are of equilibrium How to enumerate the joint supports? Type 2.1 2, 7 9, 4 3, 5 2, 3 a b cd ω 2.1 = 0.3 Type 2.2 2, 7 9, 8 3, 5 1, 3 a b cd ω 2.2 = 0.7 Player 2 Player 1

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Step 1: Heuristics Balance –Non Bayesian games: |S 1 |-|S 2 | If S 1 =(a), S 2 =(c) the balance is 0 –We call –In Bayesian games the balance is If S 1 =(a,b), S 2.1 =(c), S 2.2 =(c,d) the balance is 0 –Increasing order of balance Size –The size of a player is the sum of all the actions played with non-null probability by all the types of the player –Increasing order of size

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Step 1: Peak Criterion (1) Open Issue: –Given the values of balance and size, ranking a players supports –Example: Balance = 0 Size = 7 Player 1s types = 3 Actions = {a,b,c,d,e} S 1 = { (a), (a,b,c,d,e), (c) } S 1 = { (a,c), (a,b,c), (c,e) } Peak Criterion –Based on the size of types supports –The peak is the size of the maximum possible support –Decreasing criterion and increasing criterion

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano We use an enumeration tree to order the supports where each node defines the size of all the types. (e.g. |S 1 |, |S 2.1 |,|S 2.2 |) Size = 7Types = 3Available Actions = 5 Step 1: Peak Criterion (2)

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Step 2: Pruning Techniques The problem of checking whether or not an action is strictly conditionally Bayesian dominated by another action can be formulated as a linear feasibility problem In our case, it can be formulated as a fractional knapsack problem and then solved in linear time in the number of variables Type 2.1 2, 7 9, 4 3, 5 2, 3 a b cd ω 2.1 = 0.3 Type 2.2 2, 7 9, 8 3, 5 1, 3 a b cd ω 2.2 = 0.7 Action a is strictly conditionally Bayesian dominated by action a if for every σ -i | S -i Player 1 Player 2 Given S -i = {S 2.1 = (c), S 2.2 = (d)} EU 1 (a) = ω 2.1 · 2 + ω 2.2 · 9 EU 1 (b) = ω 2.1 · 3 + ω 2.2 · 1 EU 1 (a) > EU 1 (b)

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Step 3: B-PNS Feasibility Problem (1) Linear feasibility problem used for checking if a joint support is of equilibrium

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Step 3: B-PNS Feasibility Problem (2) The problem with support S = { (a), (c,d), (d) } is infeasible The problem with support S = { (a,b), (c), (c,d) } is feasible: –the probabilities of the actions are: player 1: p(a) = p(b)= type 1 player 2: p(c) = 1 p(d) = 0 type 2 player 2: p(c) = p(d) = Type 2.1 2, 7 9, 4 3, 5 2, 3 a b cd ω 2.1 = 0.3 Type 2.2 2, 7 9, 8 3, 5 1, 3 a b cd ω 2.2 = 0.7 Player 2 Player 1

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Experimental Evaluation We developed a tool based on GAMUT to generate Bayesian games We compared computational time in: –Different configurations of B-PNS –PNS and B-PNS –B-PNS, B-SGC, and B-LC

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Experimental Results

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Conclusions We focus on the computation of equilibria in Bayesian games This class of game is important since most strategic real-world situations can be modeled as a Bayesian game Computing Nash equilibria in complete-information games is inefficient when the game is Bayesian We extend the algorithms used for the computation of Nash equilibria for the Bayesian games We focus on B-PNS We experimentally evaluate the Bayesian algorithms

S. Ceppi, N. Gatti, and N. Basilico DEI, Politecnico di Milano Future Contributions Improvement of support enumeration methods using algorithms based on local search techniques –Non-Stochastic –Stochastic Application to open problems