Poki: The Poker Agent Greg Priebe Zak Knudson

Overview Texas Hold’em poker Architecture and Opponent Modeling of Poki Improvements from past Poki Betting strategies w/ analysis of hand strengths, weighting, and probabilities.

Texas Hold’em Each player gets 2 “hole” cards Community cards: –3 “flop” cards –1 “turn” card –1 “river” card Rounds of betting after each set of cards

Poki Program Architecture

Opponent Modeling Weight table No modeling Generic opponent modeling Specific opponent modeling

The Old Way Specific opponent modeling Betting frequency table –Keeps track of actions taken in different contexts –Predicts median hand strength Left out many relevant details

New and Improved Specific opponent modeling Detailed Dynamic learning/adaptation Prelim. Neural net study

Poki’s Neural Net

Some Results Training Data Results from percent Confusion matrix Error prone?

In Field Predictions #holdem1: 24% increase

In Field Performance

Small Bets Won Pro: Old opponent modeling:.09 New opponent modeling:.22

Betting Strategy Pre-flop and Post-flop strategies are significantly different. -Pre: little information available to influence the betting decision. -Post: uses opponent models, private hand, and game context to generate an action.

Pre-flop Betting Strategy * - pocket pair hand (two cards of the same rank) s - suited hand o - offsuit hand

Post-flop: Basic Betting Strategy Compute Poki’s effective hand strength (EHS). Using the game context, betting rules, and formulas to translate the EHS into a probability triple { Pr(fold), Pr(call), Pr(raise) }. Generate a random number and use it to choose an action from the probability distribution.

Hand Strength Probability that a given hand is better than that of an active opponent.

Hand Potential With 2 cards yet to be revealed, we want to know the impact. Positive potential (Ppot) – chance hand improves to win. Negative potential (Npot) – chance hand ends up losing. Calculated by enumerating over all possible hole cards, and over all possible board cards. This is an expensive process.

Hand potential example

Effective Hand Strength Combines hand strength and potential to give Poki’s own relative strength against an opponent. Pr(win) = pr(ahead) * pr(opp doesn’t improve) + Pr(behind) * pr(we improve) = HS * (1 – Npot) + (1 – HS) * Ppot

Weighting the Enumerations Probability of hands played to a particular point will vary. To account for this, Poki maintains a weight table for each opponent.

Conclusions Need opponent modeling Complex system Betting strategies and determining hand strength Reweight helps to determine opponents likely action