Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with David Vickrey

- = Nash Equilibrium Strategy profile : strategy for every player Regret(pi) : pi’s gain by changing strategy i Nash equilibrium:  s.t. each agent has 0 regret Theorem (Nash): Every game has at least one Nash equilibrium - = Best Actual Regret

Finding Nash Equilibria Nash equilibria difficult to compute in games with more than 2 agents Best current game solving package (GAMBIT): 2 hrs 30 min for 6-player 3-action game Game representation exponential in # of agents Algorithms inherently centralized Our approach: Structured game representation Find approximate equilibria Fast, decentralized algorithms

Graphical Games Agent pi’s utility depends on only ki other agents Represented as directed cyclic graph: pi’s utility depends on Parents(i) 1W 2W 3W 1E 2E 3E Example: property development along a road [Koller & Milch; Kearns, Littman & Singh]

Approximate Equilibria e-optimal Nash equilibrium: each agent’s regret  e

Constraint Satisfaction Constraint: each agent has zero regret Constraints are local since regret is local Each involves only node and its parents

Solving the CSP Problem: Strategies are continuous Solution: Discretize strategy space of each player Constraint: regret of agent     in table Produces -approximate equilibria The finer the discretization, the lower the  S T 2 actions S’s strategy T’s strategy .2 .4 .6 .8 1 .33 .66

Variable Elimination s s u t t u Key algorithm for both CSPs and BN inference Idea: Eliminate variables one by one Entry in new table is  iff there exists strategy for eliminated variable T s.t. for all tables containing T the matching entry is s 1 2 u S T U s 1 2 t 3 t 1 2 3 u

Cost Minimization s u .2 .3 .1 s t .3 .1 .5 .2 t u .4 .2 .1 .6 .3 S T Problem: Discretization too sparse  no -equilibrium Solution: replace  in constraint tables with regrets To eliminate T: entry in new table is min over all strategies of T of max over all tables Finds best -equilibrium for given discretization S T U s 1 2 u .2 .3 .1 s 1 2 t 3 .3 .1 .5 .2 t 1 2 3 u .4 .2 .1 .6 .3

Equilibrium quality () # Nodes in Internal Ring Experimental Results Ring of Rings Outer ring size 20 Execution Time (sec) Equilibrium quality () 60 0.035 50 0.03 0.025 40 0.02 30 0.015 20 0.01 10 0.005 2 4 6 8 10 12 14 16 18 20 2 4 6 8 10 12 14 16 18 20 # Nodes in Internal Ring Compare to: 2 hrs 30 min for 6-player 3-action game

Conclusion Finding equilibria is hard: Computationally complex Requires centralized solution Used graphical language, similar to BNs, to represent structure Adapted graphical algorithm to find equilibria Significantly more efficiently In a decentralized way