Transcript Poker Bots 2008

Jeff Berkowitz
[email protected]
POKER BOTS 2008
Tonight’s Talk
 Why is poker an interesting problem?
 A bit about poker strategy
 A simple tournament bot
 A more interesting tournament bot
 Onward and beerward
Before we begin … a word about “tells”
 Somebody always asks about Poker “tells”
 “Computers can’t read tells!” (etc.)
 Answer: tells are critical to success
 … if you’re making a movie.
 Seriously: tells are not a big part of poker.
 Questions about tells usually come from people
who haven’t yet grasped what a deep intellectual
challenge the game is.
Characterizing Games
Nondeterministic
Backgammon
…
Poker
Stock market
…
Open
Information
Limited
Information
Chess
Checkers
Go
…
Battleship
Minesweeper
…
Deterministic
Strategy (1) – Equity Basics
 You hold: 5 4
 Board shows: J T 6 2
 You believe your opponent has JJ or TT
 She bets. Should you call?
 It depends on the pot odds
 4 of the 46 unseen cards give you a likely winner
 Pot odds must be 11 : 1 (or so) to call
Limit Poker
Strategy (2) – Controlling Equity
 You Hold: K T
 Board shows: K 7 5
 Pot contains 8 bets [note: small bets]
 You bet your pair of kings [9 bets in pot]
 Turns out, player on your left holds 9 8
 Your bet helped give him the odds to call
 You should have checked, hoping to raise
 Faced your opponent with calling 2 bets
Limit Poker
Deception (1) – Face Up Poker
 Imagine Hold’em with hold cards face-up
 We’ll say heads-up (2 players) for simplicity
 All equity decisions are certain
 Your turn: simulate a few million hands
 < 1 second on modern CPUs
 Model opponents as equally perfect
 Bet based on the simulation outcome
Limit Poker
Deception (2) – Your Hidden Cards
 Now imagine you get to hide hole cards
 But the other player still plays face-up ;-)
 You play the same “optimal” strategy
 And your opponent knows it (important)
 Your bet  positive expectation
 Negative for them – so they will always fold
 So you must “sandbag” some of your hands
 Now defining “optimal” is harder
Limit Poker
NL Tourney Strategy (1) - Background
 In general, for both limit and no-limit …
 Pre-flop play is simpler, more formulaic
 Because there is less information to go on
 Post-flop play is harder
 More information available
 More time (flop, turn, river)
 More room for deception, application of skill, …
NL Tourney Strategy (2)
 But in no-limit, post-flop play is optional
 All-in on the pre-flop  no post-flop decisions!
 Creates worst possible equity position for others
 Only possible in no-limit
 Especially important [later] in tournaments
 “Advance of the blinds”
 Consequence: “Move-in” or “All-in” players
 Book “Kill Phil” [Rodman & Nelson 2005]
Digression: Poker in Academia
 University of Alberta Poker Research Group
 Their work started long before the fad – 1995
 Numerous papers on the internet
 www.poker.cs.ualberta.ca
 Excellent software (100% pure Java)
 Spinout company – Poker Academy
 The company is actually called “Biotools”
 Product has a plug-in API for bots, “Meerkat”
Demo - KillPhilBot
 Implements “Kill Phil Beginner” strategy
 From ibid (“Kill Phil”, Rodman & Nelson)
 Modified with a slightly smarter postflop
Rules, rules, lotsa rules
 A nontrivial rule-based bot gets hairy
 If this else if this then that else if the other thing…
 Tracing scattered throughout
 Must restart Poker Academy to change the rules
 Solution: separate behaviors from code
 “Rule” (not really) interpretation system
 “In the future, every programmer will have their
own language” (me, 2008)
 Poker Rules Definition Language - PRDL
PRDLBot (“Phil Purdle”)
 Express decision tree in PRDL
 Parse with JJTree / javacc
 Evaluate the parse tree for the action
 No code generation (although there could be)
 Advantages:
 Put tracing in the engine not the rules definition
 Change and reload without restarting PA
 Not fully implemented yet
PRDL Example (1)
stage = PREFLOP // all identifiers refer back to bot core
{
m > 30 // “when we have a huge stack …”
{
holeCards in [KPGROUP_1, QQ, KQs] // use of a list of expressions
{
toCall <= BLIND // or write "numberOfRaises = 0"
{
bettors = 0 // no limpers
{
action [BET, (bankroll / 6)] // another use of a list of expressions
}
: // “otherwise, there are limpers”; push all-in
{
action [BET, bankroll]
}
}
// CONTINUES NEXT SLIDE
PRDL Example (2)
: // otherwise, raised pot
{
numberOfRaises = 1
{
action [BET, max[5 * raise, bankroll / 6]]
}
: // multiple raisers
{
action [BET, bankroll]
}
}
}
: // otherwise big stack, we don't have ultra-strong cards: we're done
{
action CHECK_FOLD // use of an expression that is not a list
}
}
}
: // Post-flop
{
// . . . . . etc . . . . .
}
PRDLBot Tour
Bibliography (1 of 2)
 U of Alberta (CA) Poker Research Group:
 http://poker.cs.ualberta.ca/
 Scroll to Publications. Recommended there:
 http://poker.cs.ualberta.ca/papers/AIJ02.html
 Great online bot programming articles:
 http://www.codingthewheel.com/category/poker
 Poker Academy software:
 http://www.poker-academy.com/
 http://forums.poker-academy.com/viewforum.php?f=3
Bibliography (2 of 2)
 Best overall poker web site / book publishers:
 http://www.twoplustwo.com/
 Recommended books from them:
 See http://twoplustwo.com/books.php
 Best for beginners and for learning basic strategy are
Ed Miller’s books, Gettings Started in Hold’em and
Small Stakes Hold’em: Winning Big with Expert Play.
 Best for learning tournament play are the Harrington
and Robertie book series, Harrington on Hold’em
(volumes 1 through 3, to be studied in order).
Online Poker
 Early years (1990s – 2002)
 IRC, academic, first commercial sites
 Fad (2003 – 2006)
 TV with exposed hole cards, Chris Moneymaker
 Reality (2006 - )
 UIGEA, bot suspicions
 No individual has ever been arrested or
charged with any crime for playing poker
online in the United States.
Cheating at Online Poker
 General approach
 Write bot to Meerkat interfaces
 Test and debug to high level of skill in PA
 Emulate Meerkat on a commercial poker client
 Consider Omaha
 4 hole cards, must play 2
 Like playing 6 Hold’em hands simultaneously
 Often played for a split high/low pot (12 hands)
 Computer advantage is maximized