1. Structured Models for Multi-Agent Interactions
Daphne Koller
Stanford University
Joint work with David Vickrey

2. - = Nash Equilibrium Strategy profile : strategy for every player
Regret(pi) : pi’s gain by changing strategy i
Nash equilibrium:  s.t. each agent has 0 regret
Theorem (Nash): Every game has at least one Nash equilibrium - =
Best
Actual
Regret

3. Finding Nash Equilibria
Nash equilibria difficult to compute in games with more than 2 agents
Best current game solving package (GAMBIT):
2 hrs 30 min for 6-player 3-action game
Game representation exponential in # of agents
Algorithms inherently centralized
Our approach:
Structured game representation
Find approximate equilibria
Fast, decentralized algorithms

4. Graphical Games Agent pi’s utility depends on only ki other agents
Represented as directed cyclic graph:
pi’s utility depends on Parents(i) 1W 2W 3W 1E 2E 3E
Example: property development along a road
[Koller & Milch; Kearns, Littman & Singh]

5. Approximate Equilibria
e-optimal Nash equilibrium:
each agent’s regret  e

6. Constraint Satisfaction
Constraint: each agent has zero regret
Constraints are local since regret is local
Each involves only node and its parents

7. Solving the CSP Problem: Strategies are continuous Solution:
Discretize strategy space of each player
Constraint: regret of agent     in table
Produces -approximate equilibria
The finer the discretization, the lower the  S T
2 actions
S’s strategy
T’s strategy .2 .4 .6 .8 1
.33
.66

8. Variable Elimination s s u t t u
Key algorithm for both CSPs and BN inference
Idea: Eliminate variables one by one
Entry in new table is  iff
there exists strategy for eliminated variable T s.t.
for all tables containing T the matching entry is s 1 2 u S T U s 1 2 t 3 t 1 2 3 u

9. Cost Minimization s u .2 .3 .1 s t .3 .1 .5 .2 t u .4 .2 .1 .6 .3 S T
Problem: Discretization too sparse  no -equilibrium
Solution: replace  in constraint tables with regrets
To eliminate T:
entry in new table is min over all strategies of T
of max over all tables
Finds best -equilibrium for given discretization S T U s 1 2 u .2 .3 .1 s 1 2 t 3 .3 .1 .5 .2 t 1 2 3 u .4 .2 .1 .6 .3

10. Equilibrium quality () # Nodes in Internal Ring
Experimental Results
Ring of Rings
Outer ring size 20
Execution Time (sec)
Equilibrium quality () 60
0.035 50
0.03
0.025 40
0.02 30
0.015 20
0.01 10
0.005 2 4 6 8 10 12 14 16 18 20 2 4 6 8 10 12 14 16 18 20
# Nodes in Internal Ring
Compare to: 2 hrs 30 min for 6-player 3-action game

11. Conclusion Finding equilibria is hard:
Computationally complex
Requires centralized solution
Used graphical language, similar to BNs, to represent structure
Adapted graphical algorithm to find equilibria
Significantly more efficiently
In a decentralized way