Joint work with Sam Ganzfried Game Theory-Based Opponent Modeling in Large Imperfect-Information Games Tuomas Sandholm Carnegie Mellon University Computer Science Department Joint work with Sam Ganzfried

Traditionally two approaches Game theory approach (abstraction+equilibrium finding) Safe in 2-person 0-sum games Doesn’t maximally exploit weaknesses in opponent(s) Opponent modeling Get-taught-and-exploited problem [Sandholm AIJ-07] Needs prohibitively many repetitions to learn in large games (loses too much during learning) Crushed by game theory approach in Texas Hold’em…even with just 2 players and limit betting Same tends to be true of no-regret learning algorithms

Let’s hybridize the two approaches Start playing based on game theory approach As we learn opponent(s) deviate from equilibrium, start adjusting our strategy to exploit their weaknesses

The dream of safe exploitation Wish: Let’s avoid the get-taught-and-exploited problem by exploiting only to an extent that risks what we have won so far Proposition. It is impossible to exploit to any extent (beyond what the best equilibrium strategy would exploit) while preserving the safety guarantee of equilibrium play So we give up some on worst-case safety …

Deviation-Based Best Response (DBBR) algorithm (can be generalized to multi-player non-zero-sum) Dirichlet prior Many ways to determine opponent’s “best” strategy that is consistent with observations L1 or L2 distance to equilibrium strategy Custom weight-shifting algorithm ...

Experiments Performs significantly better in 2-player Limit Texas Hold’em against trivial opponents, and weak opponents from AAAI computer poker competitions, than game-theory-based base strategy Can be turned on only against weak opponents Examples of winrate evolution: