1. On Von Neumann Poker with Community Cards Reto Spöhel Joint work with Nicla Bernasconi and Julian Lorenz TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A A

2. Poker Poker is a popular type of card game in which players gamble on the superior value of the card combination ("hand") in their possession, by placing a bet into a central pot. The winner is the one who holds the hand with the highest value according to an established hand rankings hierarchy, or otherwise the player who remains "in the hand" after all others have folded.card gamegamblebet pothand rankingsfolded According to Wikipedia,

3. Outline Introduction von Neumann Poker von Neumann Poker with community cards Outlook: the Newman model Conclusion

4. Research on poker The game of poker has been studied from many different perspectives: game-theory [this talk] artifical intelligence (heuristics) machine learning (opponent modeling) behavioural psychology etc. In the game-theoretic approach, one assumes best play for all players involved. allows development and application of mathematical theories neglects many other factors

5. Game-theoretic research on poker almost exclusively for two-player game two main lines of attack: simplified models of real poker e.g., 2 suits with 5 cards each solution by brute-force calculation; lots of computational power needed more abstract models, which hopefully capture essential features of poker [this talk] e.g., hands are numbers u.a.r. from [0,1] hopefully analytically solvable most important model: von Neumann poker

6. Von Neumann Poker P chips are in the pot at the beginning. X and Y are dealt independent hands x,y 2 [0,1] u.a.r. X may make a bet of a or pass (check). If X checks, both hands are revealed (showdown), and the player with the higher hand wins the pot. If X makes a bet, Y can either match the bet (call) or concede the pot to X (fold). If Y folds, X wins the pot (and gets his bet back). If Y calls Xs bet, both hands are revealed and the player with the higher hand wins the pot and the two bets. In the following we assume for simplicity P = a = 1.

7. Von Neumann Poker It seems that X has an advantage, since Y can only react. So how should X play to maximize his expected payoff? Clearly, always checking guarantees him an expected payoff of P/2 = 1/2. Similarly, X cannot hope for an expected payoff of more than P=1, since Y can always fold.

8. Von Neumann Poker At first sight, one might guess that X should bet the better half of his hands, i.e., iff x ¸ 1/2. However, once Y realizes that this is Xs strategy, he will only call with hands y ¸ 2/3, since then he wins P+a=2 chips with probability at least 1/3 he loses a=1 chips with probability at most 2/3 i.e., the pot odds are in his favor. 2/3 y 1/2 0 1 x check bet fold call

9. Von Neumann Poker Xs expected payoff can be found by integrating over the hands of x and y and is: P ¢ 1/8 + P ¢ 1/2 ¢ 2/3 + (P+a) ¢ 1/18 – a ¢ 1/9 = 1/8+1/3 = 11/24 < 1/2 X loses money! check 0 bet-call P bet-fold P -a P+a 2/3 0 x y 1/2 2/3 0 1 check bet fold call x y

10. So X has no advantage over Y and should never bet? NO! With the previous strategy, most of Xs good hands go to waste because Y just folds. However, X can induce Y to call more often by including bluffs in his strategy! von Neumann gave an equilibrium pair of strategies Ys strategy is best response to Xs strategy Xs strategy is best response to Ys strategy X can achieve an expected payoff of 5/9, which is optimal. von Neumann, 1928 Von Neumann Poker

11. von Neumanns solution Heuristic: we make the following ansatz: With a hand of y 1, calling and folding should have the same expected payoff for Y: x 0 ¢ (P + a) – (1 – x 1 ) ¢ a = 0 With a hand of x 0 or x 1, betting and checking should have the same expected payoff for X: y 1 ¢ P + (x 1 – y 1 ) ¢ (P + a) – (1 – y 1 ) ¢ a = x 0 ¢ P y 1 ¢ P + (x 1 – y 1 ) ¢ (P + a) – (1 – x 1 ) ¢ a = x 1 ¢ P 0 1 check value-bet fold call x y bluff-bet y1y1 x1x1 x0x0 solution (P=a=1): x 0 = 1/9 y 1 = 5/9 x 1 = 7/9

12. von Neumanns solution The two resulting strategies are indeed in equilibrium. The expected payoff for X turns out to be 5/9 the value of the game is 5/9 (in zero-sum games, all equilibria have the same value!) Insights: Bluffing is a game-theoretic necessity! You should bluff-bet your worst hands! solution (P=a=1): x 0 = 1/9 y 1 = 5/9 x 1 = 7/9 0 1 check value-bet fold call x y bluff-bet y1y1 x1x1 x0x0

13. Von Neumann Poker Many extensions of the von Neumann model have been studied allow multiple betting rounds, raises, reraises, etc. hands may depend on each other… etc. by scientists and professional poker player alike. Chris Ferguson, PhD 2000 World Series of Poker champion co-author of several papers on von Neumann poker The mathematics of Poker, Bill Chen and Jerrod Ankenman, 2006

14. Our contribution In the classical von Neumann model (and its extensions), no further random influences are present once both players have received their hands. In real poker, community cards are drawn between betting rounds. players may bet with a bad hand, hoping that the right card will show up and turn it into a good hand. players with a good hand tend to bet more aggressively to force other players to fold. We propose the following extension of the von Neumann model that accounts for these features: Before the showdown, throw an unfair coin. With probability q, the lower hand wins!

15. Introducing the flip Introducing the flip: With a hand of y 1, calling and folding should have the same expected payoff for Y: before (q=0): x 0 ¢ (P + a) – (1 – x 1 ) ¢ a = 0 now: [x 0 ¢ (1 – q) + (1 – x 1 ) ¢ q] ¢ (P + a) – [(1 – x 1 ) ¢ (1 – q) + x 0 ¢ q] ¢ a = 0 solution (P=a=1): x 0 = x 0 (q) y 1 = y 1 (q) x 1 = x 1 (q) 0 1 check value-bet fold call x y bluff-bet y1y1 x1x1 x0x0

16. Introducing the flip y1y1 x1x1 x0x0 ? 1/3 =: q 0 5/9 1/9 7/9 We obtain

17. Beyond the critical q What happens for q > q 0 = 1/3? Y will call every bet since even with the worst hand y = 0 he wins P+a=2 chips with probability at least q = 1/3 he loses a=1 chips with probability at most 1 – q = 2/3 Knowing this, X will bet the better half of his hands. 1/2 0 1 check bet call x y for general P and a: q : (1-q) = a : (P+a) q 0 = a/(P+2a)

18. The full picture y1y1 x1x1 x0x0 q 0 = 1/3 As q increases, X makes more value bets and bluffs less. Y is induced by the q to call! For q ¸ q 0, theres no point in bluffing, since Y will always call anyway. X value-bets more often to protect his good hands from being flipped into bad hands.

19. The value of the game 1/2 = 0.5 7/12 = 0.583… 5/9 = 0.555… q 0 = 1/3 value of the game The expected payoff of X is maximal at q = q 0. i.e., when Y has just enough incentive to call every bet of X without X wasting money on bluffs.

20. An improved model Where does the discontinuity at q 0 come from? We fixed the bet size a (arbitrarily) before the game started. What if we allow X to look at his hand and then bet any amount a ¸ 0 he likes? …and with flip probability q, 0 · q < 1/2 X can achieve an expected payoff of 4/7, which is optimal. Newman, 1959 X can achieve an expected payoff of (16-q)/(28-8q), which is optimal. Bernasconi, Lorenz, S., 2007+

21. Newmans solution 1/74/7 value-betbluff-bet bet a hand x, resp. y The red line is Xs bet a(x) X checks for 1/7 · x · 4/7 The green line is Ys calling threshold Y calls iff (y,a) is below the green line obtained with similar, but more involved heuristics than before ( differential equations) (q = 0)

22. Introducing the flip a 0 = 1/2 Similarly to before: q : (1-q) = a : (P+a) a 0 = q/(1-2q) ¢ P even with his worst hand, Y calls every bet of at most a 0 (pot-odds!) knowing this, X never bets an amount between 0 and a 0 neither as a value bet (if the odds are in his favour [x ¸ 1/2], he bets at least a 0 ), nor as a bluff bet (theres no value bet for which to induce more calls). hand x, resp. y (q = 1/3) bet a

23. Introducing the flip (q = 1/3) (q = 0.4) bet a As q increases, X bets his value bets more aggressively. hand x

24. The value of the game value of the game For 0 < q · 1/2, we have value(q) = (16-q)/(28-8q) the value is strictly increasing in q but discontinuous at q=1/2, since trivially value(1/2) = 1/2. What is going on? We allowed arbitrarily high bets a. Moreover, a 0 diverges at q = 1/2. If we limit Xs bankroll, the singularities vanish. 4/7 = 0.571… 31/48 = 0.645…

25. Conclusion We proposed a way of including community cards (random effects after betting) into both the von Neumann and the Newman model… … and gave complete analytical solutions. As expected, we observed increasingly aggressive betting for larger q. In both models, we observe a phase transition when q : (1-q) = a : (P+a) (flip-odds = pot-odds)

26. Questions?