Transcript von Neumann Poker

On Von Neumann Poker
with Community Cards
Reto Spöhel
Joint work with Nicla Bernasconi and Julian Lorenz
Poker

According to Wikipedia,
Poker is a popular type of card game in
which players gamble on the superior value
of the card combination ("hand") in their
possession, by placing a bet into a central
pot. The winner is the one who holds the
hand with the highest value according to an
established hand rankings hierarchy, or
otherwise the player who remains "in the
hand" after all others have folded.
Outline





Introduction
von Neumann Poker
von Neumann Poker with community cards
Outlook: the Newman model
Conclusion
Research on poker


The game of poker has been studied from many different
perspectives:
 game-theory [this talk]
 artifical intelligence (heuristics)
 machine learning (opponent modeling)
 behavioural psychology
 etc.
In the game-theoretic approach, one assumes best play for
all players involved.
 allows development and application of mathematical
theories
 neglects many other factors
Game-theoretic research on poker


almost exclusively for two-player game
two main lines of attack:
 simplified models of „real“ poker
 e.g., 2 suits with 5 cards each
 solution by brute-force calculation; lots of
computational power needed
 more abstract models, which hopefully capture
essential features of poker [this talk]
 e.g., hands are numbers u.a.r. from [0,1]
 hopefully analytically solvable
 most important model: von Neumann poker
Von Neumann Poker




P chips are in the pot at the beginning.
X and Y are dealt independent hands x,y 2 [0,1] u.a.r.
X may make a bet of a or pass („check“).
 If X checks, both hands are revealed („showdown“),
and the player with the higher hand wins the pot.
 If X makes a bet, Y can either match the bet („call“) or
concede the pot to X („fold“).
 If Y folds, X wins the pot (and gets his bet back).
 If Y calls X‘s bet, both hands are revealed and the
player with the higher hand wins the pot and the two
bets.
In the following we assume for simplicity P = a = 1.
Von Neumann Poker

It seems that X has an advantage, since Y can only react.
So how should X play to maximize his expected payoff?
 Clearly, always checking guarantees him an expected
payoff of P/2 = 1/2.
 Similarly, X cannot hope for an expected payoff of more
than P=1, since Y can always fold.
Von Neumann Poker


At first sight, one might guess that X should bet the better
half of his hands, i.e., iff x ¸ 1/2.
However, once Y realizes that this is X‘s strategy, he will
only call with hands y ¸ 2/3, since then
 he wins P+a=2 chips with probability at least 1/3
 he loses a=1 chips with probability at most 2/3
 i.e., „the pot odds are in his favor“.
1
call
bet
2/3
1/2
check
fold
0
x
y
Von Neumann Poker


X‘s expected payoff can be found by integrating over the
hands of x and y and is:
P ¢ 1/8 + P ¢ 1/2 ¢ 2/3 + (P+a) ¢ 1/18 – a ¢ 1/9
= 1/8+1/3 = 11/24 < 1/2
 X loses money!
y
1
-a
bet-call
P+a
call
bet
2/3
0
2/3
1/2
check
bet-fold
check
fold
P
P
0
x
y
0
x
1/2
Von Neumann Poker

So X has no advantage over Y and should never bet?
 NO!
von Neumann, 1928
X can achieve an expected payoff of 5/9, which is optimal.
With the previous strategy, most of X‘s good hands go
to waste because Y just folds.
 However, X can induce Y to call more often by
including bluffs in his strategy!
von Neumann gave an equilibrium pair of strategies
 Y‘s strategy is best response to X‘s strategy
 X‘s strategy is best response to Y‘s strategy


von Neumann‘s solution

Heuristic: we make the following ansatz:
 With a hand of y1, calling and folding should have the
same expected payoff for Y:
x0 ¢ (P + a) – (1 – x1) ¢ a = 0
 With a hand of x0 or x1, betting and checking should
have the same expected payoff for X:
y1 ¢ P + (x1 – y1) ¢ (P + a) – (1 – y1) ¢ a = x0 ¢ P
y1 ¢ P + (x1 – y1) ¢ (P + a) – (1 – x1) ¢ a = x1 ¢ P
1
value-bet
call
x1
y1
check
x0
fold
bluff-bet
0
x
y

 solution (P=a=1):
 x0 = 1/9
 y1 = 5/9
 x1 = 7/9
von Neumann‘s solution



The two resulting strategies are indeed in equilibrium.
The expected payoff for X turns out to be 5/9
 the value of the game is 5/9 (in zero-sum games, all
equilibria have the same value!)
Insights:
 Bluffing is a game-theoretic necessity!
 You should bluff-bet your worst hands!
1
value-bet
call
x1
y1
check
x0
fold
bluff-bet
0
x
y

 solution (P=a=1):
 x0 = 1/9
 y1 = 5/9
 x1 = 7/9
Von Neumann Poker


Many extensions of the von Neumann model have been
studied
 allow multiple betting rounds, raises, reraises, etc.
 hands may depend on each other…
 etc.
by scientists and professional poker player alike.
Chris Ferguson, PhD

2000 World Series
of Poker champion

co-author of
several papers on
von Neumann
poker
The mathematics of
Poker, Bill Chen and
Jerrod Ankenman,
2006
Our contribution



In the classical von Neumann model (and its extensions),
no further random influences are present once both players
have received their hands.
In real poker, community cards are drawn between betting
rounds.
  players may bet with a bad hand, hoping that the
right card will show up and turn it into a good hand.
  players with a good hand tend to bet more
aggressively to force other players to fold.
We propose the following extension of the von Neumann
model that accounts for these features:
 Before the showdown, throw an unfair coin. With
probability q, the lower hand wins!
Introducing the flip

1
Introducing the flip:
 With a hand of y1, calling and folding should have the
same expected payoff for Y:
 before (q=0):
x0 ¢ (P + a) – (1 – x1) ¢ a = 0
 now:
[x0 ¢ (1 – q) + (1 – x1) ¢ q] ¢ (P + a)
– [(1 – x1) ¢ (1 – q) + x0 ¢ q] ¢ a
=0
value-bet
call
x1
y1
check
x0
fold
x
 solution (P=a=1):


bluff-bet
0

y

x0 = x0(q)
y1 = y1(q)
x1 = x1(q)
Introducing the flip

We obtain
7/9
x1
5/9
y1
1/9
?
x0
1/3 =: q0
Beyond the critical q



What happens for q > q0 = 1/3?
Y will call every bet since even with the worst hand y = 0
 he wins P+a=2 chips with probability at least q = 1/3
 he loses a=1 chips with probability at most 1 – q = 2/3
Knowing this, X will bet the better half of his hands.
1

for general P and a:
bet
q : (1-q) = a : (P+a)
call
1/2
 q0 = a/(P+2a)
check
0
x
y
The full picture

x1
y1

x0
q0 = 1/3
As q increases, X makes more
value bets and bluffs less.
 Y is induced by the q to call!
 For q ¸ q0, there‘s no point in
bluffing, since Y will always call
anyway.
X value-bets more often to protect
his good hands from being flipped
into bad hands.
The value of the game
7/12 = 0.583…

5/9 = 0.555…
value of the game
1/2 = 0.5
q0 = 1/3
The expected payoff of X is
maximal at q = q0.
 i.e., when Y has just enough
incentive to call every bet of X
without X wasting money on
bluffs.
An improved model


Where does the discontinuity at q0 come from?
 We fixed the bet size a (arbitrarily) before the game
started.
What if we allow X to look at his hand and then bet any
amount a ¸ 0 he likes?
Newman, 1959
X can achieve an expected payoff of 4/7, which is optimal.

…and with flip probability q, 0 · q < 1/2
Bernasconi, Lorenz, S., 2007+
X can achieve an expected payoff of (16-q)/(28-8q), which is
optimal.
Newman‘s solution

(q = 0)

bet a
„bluff-bet“
The red line is X‘s bet a(x)
 X checks for 1/7 · x · 4/7
The green line is Y‘s calling
threshold
 Y calls iff (y,a) is below the
green line
„value-bet“

1/7
4/7
hand x, resp. y
obtained with similar, but more
involved heuristics than before
( differential equations)
Introducing the flip

Similarly to before:
(q = 1/3)
q : (1-q) = a : (P+a)
bet a


a0 =
1/2
hand x, resp. y
 a0 = q/(1-2q) ¢ P
even with his worst hand, Y calls
every bet of at most a0 (pot-odds!)
knowing this, X never bets an
amount between 0 and a0
 neither as a value bet (if the
odds are in his favour [x ¸ 1/2],
he bets at least a0),
 nor as a bluff bet (there‘s no
value bet for which to induce
more calls).
Introducing the flip

bet a
(q = 0.4)
(q = 1/3)
hand x
As q increases, X bets his
value bets more aggressively.
The value of the game
31/48 = 0.645…

value of the game

For 0 < q · 1/2, we have
value(q) = (16-q)/(28-8q)
 the value is strictly increasing
in q but discontinuous at q=1/2,
since trivially
value(1/2) = 1/2.
What is going on?
4/7 = 0.571…


We allowed arbitrarily high bets a. Moreover, a0 diverges at
q = 1/2.
If we limit X‘s bankroll, the singularities vanish.
Conclusion




We proposed a way of including „community cards“
(random effects after betting) into both the von Neumann
and the Newman model…
… and gave complete analytical solutions.
As expected, we observed increasingly aggressive betting
for larger q.
In both models, we observe a „phase transition“ when
q : (1-q) = a : (P+a)
(“flip-odds = pot-odds”)
Questions?