Transcript Artificial Intelligence in Game Design

Artificial Intelligence in
Game Design
Heuristics and Other Ideas in
Board Games
Good and Bad Heuristics
• Heuristic for evaluating board must be accurate
– Directly related to “likelihood” of win
– Inversely related to “distance” from win
• Example: TicTacToe heuristic
H(board) = 2 × # of possible rows/columns/diagonals where X could win in one move +
+ 1 × # of possible rows/columns/diagonals where X could win in two moves +
- 2 × # of possible rows/columns/diagonals where O could win in one move +
- 1 × # of possible rows/columns/diagonals where O could win in two moves
• Gives these two boards same measure
Guaranteed
loss for AI
O
X
X
X
O
X
-2
O
O
O
X
X
O
-2
Good and Bad Heuristics
• Better heuristic must take this into account!
• H(board) =
–
–
–
–
–
if my move next and I have 2 in row  MAXINT
if opponent move next and they have 2 in row  -MAXINT
if my move and opponent has > 1 instance of 2 in row  -MAXINT
if opponent move and I have > 1 instance of 2 in row  MAXINT
else (above function)
• Will work better, but more complex to compute!
Accuracy of
heuristic value
tradeoff
Speed to compute
heuristic value
Linear Heuristic Functions
• Heuristic is some function of individual pieces on board
– Usually weighted in some way
– Overall board value = Σ H(piecei)
i
• Very fast to compute
• Example: Chess
– Pieces have “standard values”
– Heuristic = sum of AI pieces –
sum of player pieces
9
5
3
3
1
Linear Heuristic Functions
• Often based on position of pieces on board
• Examples:
– Games where purpose is to move all
pieces to some goal
• Backgammon
• Sorry
• Can often send other player back to “start”
– Heuristic value = total distance of AI pieces from goal –
total distance of player pieces from goal
Linear Heuristic Functions
• Reversi (Othello)
– Board value of based entirely on piece positions
• Corners very valuable (can’t be flipped)
• Sides somewhat valuable (very difficult to flip)
• Middle little value (will easily be flipped)
– H(board) = C1 * number of pieces in corner +
C2 * number of pieces on side +
C3 * number of pieces in middle +
C1 >> C2 >> C3
• Reversi easiest type of game for AI
– Low branching factor (5 to 15 possible moves)
– Good heuristics
– Single move can greatly change board
• Hard for human player to see
• Easy for MinMax lookahead to see
Nonlinear Heuristics
• Based on relationships between pieces on board
• Simple example:
– Prefer chess pieces to protect one another
– Piece value = piece value * 1.5 if protected by another
– Piece value = piece value * 0.5 if not protected and threatened
by opponent piece
Nonlinear Heuristics
• Drawback: Usually much more expensive to compute
– n pieces  O (n2) relationships between them
• May be able to explore more moves with simpler (linear)
heuristic
• Example:
– Simple linear heuristic takes k ms to evaluate per board
– Complex nonlinear heuristic takes 400k ms to evaluate per
board
– Average of 20 possible next moves per board
• 400 possible next two moves
– On average, could explore two additional moves if use linear
heuristic
Linear vs. Nonlinear Heuristics
• Nonlinear heuristics often detect future changes in board
– Pieces threatened by others might be captured
– Pieces protected by others less likely to be captured
• Often see effect of nonlinear heuristic with additional
levels of linear heuristic
• Example: “knight fork” in chess
– Can look for this using nonlinear
heuristic
Linear vs. Nonlinear Heuristics
• Searching 2 more moves in game tree will show result of knight fork
– Shows state where rook now gone
– Higher value for linear heuristic based on piece values
– MinMax then gives “knight fork” state a high value
High value h
Other moves give
checkmate (value
= MAXINT)
Board with
knight fork
Min level
Black moves king
High value h
Board with knight
threatening rook
Max level
High value h
Knight takes rook
Board with black
rook gone
Horizon Effect
• Major weakness of purely linear heuristics based on piece values
– “Throwing good money after bad”
• May need to recognize when move does not improve board position
• Chess example:
Queen pinned by bishops
Could move rook in way
But rook captured and will
still lose queen
Horizon Effect
Board with
queen pinned
Queen takes bishop
Bishop takes queen
White down 6 points
Board with
queen lost
This looks worse at cutoff
level, but is actually best
in long run!
Move rook in front
Bishop takes rook
Board with
rook lost
White down 5 points
Queen takes bishop
Bishop takes queen
Board with
queen and
rook lost
White now down
11 points!
Trappy Minmax
• Idea: Use minmax to set traps for player
– Branch appears to have short-term benefit for player
– In long run, branch has benefit for AI
Good chance
player might
choose this
branch
AI move
This state looks
ok to player
player move
Ok result for AI
This state looks
good to player
Great result for AI
So worth
considering
making this
move
Trappy Minmax
Takes advantage of player horizon effect
Queen can
capture rook
Some other move
White unchanged
Board with
rook ignored
Queen takes rook
Board with
rook captured
White up 5 points
Knight captures queen
Player did not lookahead
far enough to see this
Board with
queen lost
White now down
4 points!
Factors in Trappy Minmax
• Trappiness: Estimate of how likely player will choose
branch corresponding to trap
– Usually computed based on median or maximum of player
heuristics down best branch
Possible player move
5
Path player
thinks game
will follow if
they make this
move
2
9
6
8
-7
Median = 7
Factors in Trappy Minmax
• Profitability: Score if player follows “trap” branch –
score if player follows their best branch
• Trap Quality: Trappiness * Profitability
• Use trap quality to adjust heuristic measure of a state:
Trappy_heuristic(state) = normal_heuristic(state)
+ f(trap_quality(state))
Data-Driven Approaches
Basing actions on known strategies rather than tree search
• Opening books of initial moves (Chess)
– Often 20-30 moves at start of Grandmaster match
Current
board
• Each book consists of:
– List of moves
– Evaluation of final outcome
Boards in
sequence
• Should we follow this strategy
• Allows faster processing
– No need to search game tree until
end of sequence
– Can just use evaluation as heuristic
End of
sequence
Now start branching
Opening Books
• Choose as own opening strategy
• Must have good final evaluation!
– Make moves according to script
– If opponent follows script, keep following
– If opponent leaves script, start MinMax
• Will probably be in same way that benefits us
• Must also recognize when opponent uses an opening book
– Keep database of moves in opening books
– Match current board to those in database to find whether it is part of
a sequence
– If so, make decision about whether following script is good idea
Other Set Plays
• End Games
– Many games have different strategies when few pieces left
• Forcing checkmate in chess
• Kings vs. kings in checkers
• Getting last pieces home in backgammon
– Recognize based on pieces left
– Follow set strategies
• Set Evaluation Values
– No heuristic evaluation of board – instead,
match board to database to get evaluation
– Works best if can match subboards
• Example: Edge configurations in Othello
• This edge has a known value
Alternative Approaches
• Go
– 19  19 board  branching factor of 361
– Impossible for MinMax
• Only known approaches based
on template matching
– Look in local area for configurations
that match known strategies
– Still very open problem
• Best AI for Go only plays at
amateur level