1. SARTRE: System Overview A Case-Based Agent for Two-Player Texas Hold'em Jonathan Rubin & Ian Watson University of Auckland Game AI Group http://www.cs.auckland.ac.nz/research/gameai/

2. Overview Introduction Texas Hold'em Approaches to Computer Poker Sartre: System Overview Results Future Work

3. Texas Hold'em Two-player Limit Hold'em – Much different to full-table game Chance events Hidden Information

4. Approaches to Computer Poker Near-Equilibrium Strategy Exploitative Strategy

5. Near-Equilibrium Strategy Nash Equilibrium – Assumes the opponent makes no mistakes – Attempts to minimise its loses against this perfect opponent Near-Equilibrium – As game tree is too large – Plays not to lose

6. Exploitative Strategy – Opponent Modelling – Attempts to punish weaknesses in the opponents strategy – Plays off the equilibrium – Plays to win

7. Sartre: System Overview Similarity Assessment Reasoning for Texas hold'em via Recall of Experience Our entry for the 2009 Computer Poker Competition Case-base was constructed from past CPC games

8. Sartre: System Overview Hand picked by authors Case Features – Previous betting for the hand – Hand Category – Board Category

9. 1. Previous betting for the hand Currently represented as a string – f = fold – c = check/call – r = bet/raise Examples – r – rrc-r – rc-crrc-rc-cr

10. 1. Previous betting for the hand

11. 2. Hand Category Rule-based System

12. 2. Hand Category Two components – Hand Category – Hand Potential Examples – Missed – One-Pair, Two-Pair, Three-of-a-kind – Flush-draw, Straight-draw

13. 3. Board Category Captures information about potential – Flush Draws or, – Straight Draws Information that is likely to be noticed by an good player

14. 3. Board Category Flush Highly Possible

15. 3. Board Category Straight Possible

16. Similarity Currently either all or nothing – If a collection of cards maps to the same category they are assigned a similarity of 1.0, otherwise 0.

17. Case Overview Case Features – 1. Previous betting for the hand – 2. Hand Category – 3. Board Category Solution – f, c, r Outcome – +/- value – + Profit – - Loss

18. Case Overview Solution + Outcome – Recorded from equilibrium approaching bots from previous AAAI Computer Poker Competition Separate case-bases for preflop, flop, turn & river Approx. 250,000 cases in each case-base.

19. Decision Making Retrieved cases can have different decisions Three different versions – 1. Probability Triple – 2. Majority rules – 3. Outcome-based

20. Decision Making Probability Triple – Proportion of times that the solution indicated to fold, call or raise – (f, c, r) Majority Rules – Decision made the most is reused Outcome-Based – Dependant on adjusted average outcome values for each decision – If a call or raise decision was never made, it's outcome is unknown and is given a value of +infinity

21. Duplicate Matches Experimental results derived using duplicate matches – Play N poker hands – Reset each players memory – Reverse the position of each player and deal the same N hands Forward + Reverse Directions Reduces variance

22. Self-Play Experiments Small bets per hand (sb/h) – Assuming a $10/$20 game Sartre-Probability Vs. Sartre-Outcome – Sartre-Probability wins 0.168 sb/h – On average $1.68 profit per hand Sartre-Probability Vs. Sartre-Majority – Sartre-Majority wins 0.039 sb/h – On average $0.39 per hand

23. Self-Play Experiments Chose Sartre – Majority Rules. Results not transitive Makes Sartre more predictable and hence more exploitable by strong opposition

24. 2009 Computer Poker Competition Results Duplicate match structure – 3000 hands in forward & reverse direction Multiple matches against each opponent until statistical significance obtained Sartre placed 7 th out of 13 entrants in limit competition

25. 2009 Computer Poker Competition Results 1MANZANA-0.038 2GGValuta-0.043 3 HyperboreanLimit- Eqm -0.051 4 HyperboreanLimit- BR -0.023 5Rockhopper-0.033 6Slumbot-0.012 7Sartre 8GS5-0.007 9AoBot0.131 10LIDIA0.145 11dcurbhu0.217 12GS5Dynamic0.119 13tommybot0.765 Total0.097

26. 2009 Computer Poker Competition Results Overall profit of +0.097 sb/h Assuming a $10/$20 game – $0.97 per hand profit

27. Future Work Investigate loosening of all-or-nothing similarity CBR and adaptive poker agents – Opponent modelling – Learning Better solution adaptation – Combination of decision + outcome

28. The End!