Poker Bots 2008
Download
Report
Transcript Poker Bots 2008
Jeff Berkowitz
[email protected]
POKER BOTS 2008
Tonight’s Talk
Why is poker an interesting problem?
A bit about poker strategy
A simple tournament bot
A more interesting tournament bot
Onward and beerward
Before we begin … a word about “tells”
Somebody always asks about Poker “tells”
“Computers can’t read tells!” (etc.)
Answer: tells are critical to success
… if you’re making a movie.
Seriously: tells are not a big part of poker.
Questions about tells usually come from people
who haven’t yet grasped what a deep intellectual
challenge the game is.
Characterizing Games
Nondeterministic
Backgammon
…
Poker
Stock market
…
Open
Information
Limited
Information
Chess
Checkers
Go
…
Battleship
Minesweeper
…
Deterministic
Strategy (1) – Equity Basics
You hold: 5 4
Board shows: J T 6 2
You believe your opponent has JJ or TT
She bets. Should you call?
It depends on the pot odds
4 of the 46 unseen cards give you a likely winner
Pot odds must be 11 : 1 (or so) to call
Limit Poker
Strategy (2) – Controlling Equity
You Hold: K T
Board shows: K 7 5
Pot contains 8 bets [note: small bets]
You bet your pair of kings [9 bets in pot]
Turns out, player on your left holds 9 8
Your bet helped give him the odds to call
You should have checked, hoping to raise
Faced your opponent with calling 2 bets
Limit Poker
Deception (1) – Face Up Poker
Imagine Hold’em with hold cards face-up
We’ll say heads-up (2 players) for simplicity
All equity decisions are certain
Your turn: simulate a few million hands
< 1 second on modern CPUs
Model opponents as equally perfect
Bet based on the simulation outcome
Limit Poker
Deception (2) – Your Hidden Cards
Now imagine you get to hide hole cards
But the other player still plays face-up ;-)
You play the same “optimal” strategy
And your opponent knows it (important)
Your bet positive expectation
Negative for them – so they will always fold
So you must “sandbag” some of your hands
Now defining “optimal” is harder
Limit Poker
NL Tourney Strategy (1) - Background
In general, for both limit and no-limit …
Pre-flop play is simpler, more formulaic
Because there is less information to go on
Post-flop play is harder
More information available
More time (flop, turn, river)
More room for deception, application of skill, …
NL Tourney Strategy (2)
But in no-limit, post-flop play is optional
All-in on the pre-flop no post-flop decisions!
Creates worst possible equity position for others
Only possible in no-limit
Especially important [later] in tournaments
“Advance of the blinds”
Consequence: “Move-in” or “All-in” players
Book “Kill Phil” [Rodman & Nelson 2005]
Digression: Poker in Academia
University of Alberta Poker Research Group
Their work started long before the fad – 1995
Numerous papers on the internet
www.poker.cs.ualberta.ca
Excellent software (100% pure Java)
Spinout company – Poker Academy
The company is actually called “Biotools”
Product has a plug-in API for bots, “Meerkat”
Demo - KillPhilBot
Implements “Kill Phil Beginner” strategy
From ibid (“Kill Phil”, Rodman & Nelson)
Modified with a slightly smarter postflop
Rules, rules, lotsa rules
A nontrivial rule-based bot gets hairy
If this else if this then that else if the other thing…
Tracing scattered throughout
Must restart Poker Academy to change the rules
Solution: separate behaviors from code
“Rule” (not really) interpretation system
“In the future, every programmer will have their
own language” (me, 2008)
Poker Rules Definition Language - PRDL
PRDLBot (“Phil Purdle”)
Express decision tree in PRDL
Parse with JJTree / javacc
Evaluate the parse tree for the action
No code generation (although there could be)
Advantages:
Put tracing in the engine not the rules definition
Change and reload without restarting PA
Not fully implemented yet
PRDL Example (1)
stage = PREFLOP // all identifiers refer back to bot core
{
m > 30 // “when we have a huge stack …”
{
holeCards in [KPGROUP_1, QQ, KQs] // use of a list of expressions
{
toCall <= BLIND // or write "numberOfRaises = 0"
{
bettors = 0 // no limpers
{
action [BET, (bankroll / 6)] // another use of a list of expressions
}
: // “otherwise, there are limpers”; push all-in
{
action [BET, bankroll]
}
}
// CONTINUES NEXT SLIDE
PRDL Example (2)
: // otherwise, raised pot
{
numberOfRaises = 1
{
action [BET, max[5 * raise, bankroll / 6]]
}
: // multiple raisers
{
action [BET, bankroll]
}
}
}
: // otherwise big stack, we don't have ultra-strong cards: we're done
{
action CHECK_FOLD // use of an expression that is not a list
}
}
}
: // Post-flop
{
// . . . . . etc . . . . .
}
PRDLBot Tour
Bibliography (1 of 2)
U of Alberta (CA) Poker Research Group:
http://poker.cs.ualberta.ca/
Scroll to Publications. Recommended there:
http://poker.cs.ualberta.ca/papers/AIJ02.html
Great online bot programming articles:
http://www.codingthewheel.com/category/poker
Poker Academy software:
http://www.poker-academy.com/
http://forums.poker-academy.com/viewforum.php?f=3
Bibliography (2 of 2)
Best overall poker web site / book publishers:
http://www.twoplustwo.com/
Recommended books from them:
See http://twoplustwo.com/books.php
Best for beginners and for learning basic strategy are
Ed Miller’s books, Gettings Started in Hold’em and
Small Stakes Hold’em: Winning Big with Expert Play.
Best for learning tournament play are the Harrington
and Robertie book series, Harrington on Hold’em
(volumes 1 through 3, to be studied in order).
Online Poker
Early years (1990s – 2002)
IRC, academic, first commercial sites
Fad (2003 – 2006)
TV with exposed hole cards, Chris Moneymaker
Reality (2006 - )
UIGEA, bot suspicions
No individual has ever been arrested or
charged with any crime for playing poker
online in the United States.
Cheating at Online Poker
General approach
Write bot to Meerkat interfaces
Test and debug to high level of skill in PA
Emulate Meerkat on a commercial poker client
Consider Omaha
4 hole cards, must play 2
Like playing 6 Hold’em hands simultaneously
Often played for a split high/low pot (12 hands)
Computer advantage is maximized