Transcript Turn-Based Games sources: •http://www.game-research.com/ •www.gamespot.com •Wikipedia.org •Russell & Norvig AI Book; Chapter 5 (and slides) •Jonathan Schaeffer’s AAW 05 presentation •My own Héctor Muñoz-Avila.

Turn-Based Games
sources:
•http://www.game-research.com/
•www.gamespot.com
•Wikipedia.org
•Russell & Norvig AI Book; Chapter 5 (and slides)
•Jonathan Schaeffer’s AAW 05 presentation
•My own
Héctor Muñoz-Avila
Turn-Based Strategy Games
Early strategy games was dominated by turnbased games
•Derivate from board games
•Chess
•The Battle for Normandy (1982)
•Nato Division Commanders (1985)
Turn-based strategy:
•game flow is partitioned in turns or rounds.
• Turns separate analysis by the player from actions
•“harvest, build, destroy” in turns
• Two classes:
•Simultaneous
•Mini-turns
Turn-Based Games Continues to be A
Popular Game Genre
• At least 3 sub-styles are very popular:
– “Civilization”-style games
• Civilization IV came out last week
– Fantasy-style (RPG)
• Heroes of Might and Magic series
– Poker games
• Poker Academy
Some Historical Highlights
• 1952 Turing design a chess algorithm. Around the same
time Claude Shannon also develop a chess program
• 1956 Maniac versus Human
• 1970 Hamurabi. A game about building an economy for a
kingdom
• The Battle for Normandy (1982)
• 1987 Pirates!
• 1990 Civilization
• 1995 HoMM
• 1996 Civilization II
• The best game ever?
• …
• 2005 Civilization IV
• 2006 HoMM V
Side-tracking:
Game Design: Contradicting Principles
• Principle: All actions can be done from a
single screen.
• Classical example: Civ IV
• But: HoMM uses two interfaces: HoMM IV
Coming back: How to Construct Good AI?
• Idea: Lets just use A* and define a good heuristic for the game
Search space: a bipartite tree
After all didn’t we use it with the 9-puzzle game?
• Problems with this idea:
 Adversarial: we need to consider possible moves of
our opponent (s)
Time limit: (think Chess)
Types of Adversarial
TBGs (from AI perspective)
Deterministic
Chance
Perfect
information
Chess, Go, rockpaper-scissors
Backgammon,
monopoly
Imperfect
information
Battleships,
Stratego
Bridge, Poker
Civilization, HoMM
Game tree (2-player, deterministic, turns)
Concepts:
•State: node in search space
•Operator: valid move
•Terminal test: game over
•Utility function: value for
outcome of the game
•MAX: 1st player, maximizing its
own utility
•MIN: 2nd player, minimizing
Max’s utility
Minimax
• Finding perfect play for deterministic games
• Idea: choose move to position with highest minimax
value
= best achievable payoff against best play
• E.g., 2-play game:
Minimax algorithm
Properties of minimax
• Complete? Yes (if tree is finite)
• Optimal? Yes (against an optimal opponent)
• Time complexity? O(bm)
– b: branching factor
– m: # moves in a game
•For chess, b ≈ 35, m ≈100 for "reasonable" games
Therefore, exact solution is infeasible
Minimax algorithm with Imperfect
Decisions
Cutoff-test(state)
evaluationFunction(state)
Cutoff-test(state)
evaluationFunction(state)
Evaluation Function
• Evaluation Function
– Is an estimate of the actual utility
– Typically represented as a linear function:
EF(state) = w1f1(state) + w2f2(state) + … + wnfn(state)
– Example:
Chess
• weight: Piece  Number •Function; state  Number
f1 = #(pawns,w)  #(pawns,b)
(w1) Pawn  1
f2 = #(knight,w)  #(knight,b)
(w2) Knight  3
f3 = #(bishop,w)  #(bishop,b)
(w3) Bishop  3
f4 = #(rook,w)  #(rook,b)
(w4) Rook  5
f5 = #(knight,w)  #(knight,b)
(w5) Queen  9
Example: Evaluation Function
“all things been equal”
White moves,
Who is winning?
Black
Is this consistent with
Evaluation function?
Yes!
Evaluation Function (2)
• Obviously, the quality of the AI player depends on the
evaluation function
• Conditions for evaluation functions:
If n is a terminal node, EF(n) = Utility(n)
Computing EF should not take long
EF should reflect chances of winning
If EF(state) > 3 then is almost-certain that blacks win
Cutting Off Search
•
When to cutoff minimax expansion?
Fixed depth limit
Iterative deepening until times runs out
•
Potential problem with cutting off search: Horizon
problem
Decision made by opponent is damaging but
cannot be “seen” because of cutoff
•
Solution:
Quiescent: states that are unlikely to exhibit wild
swings in the values of the evaluation functions
Example: Horizon Problem
“all things been equal”
White moves,
Who is winning?
Black
Is this consistent with
Evaluation function?
No!
α-β pruning: Motivation
•A good program may search 1000 positions per second
•In a chess tournament, a player gets 150 seconds per move
•Therefore, the program can explore 150,000 positions per move
•With a branching factor of 34, this will mean a look ahead of 3
or 4 moves
• Facts:
4-turns ≈ human novice
8-turns ≈ typical PC, human master
12-turns ≈ Deep Blue, Kasparov
• How to look ahead more than 4 turns? Use α-β pruning
Example:
• Finding perfect play for deterministic games
• Idea: choose move to position with highest minimax
value
= best achievable payoff against best play
• E.g., 2-play game:
α-β pruning
α-β pruning example
α-β pruning example
α-β pruning example
α-β pruning example
Principle of α-β Prunning
• α is the value of the best (i.e., highestvalue) choice found so far at any
choice point along the path for max
 If v  α, max will avoid it
o Therefore, prune that branch
• β is the lowest-value found so far at
any choice point along the path for
min
 If v  α, min will avoid it
o Therefore, prune that branch
The α-β algorithm
The α-β algorithm
Properties of α-β
• Pruning preserves completeness and optimality of original
minimax algorithm
• Good move ordering improves effectiveness of pruning
• With "perfect ordering," time complexity = O(bm/2)
Therefore, doubles depth of search
• Used in PC games today (9 moves look-ahead, Grand
Master level)
Deterministic games in practice
• Checkers: Chinook ended 40-year-reign of human world champion
Marion Tinsley in 1994. Used a precomputed endgame database
defining perfect play for all positions involving 8 or fewer pieces on the
board, a total of 444 billion positions.
• Chess: Deep Blue defeated human world champion Garry Kasparov in a
six-game match in 1997. Deep Blue searches 200 million positions per
second, 24 processors, quiescent identified with help of human grand masters
• Othello: human champions refuse to compete against computers, who
are too good.
• Go: human champions refuse to compete against computers, who are too
bad. In go, b > 300, so most programs use pattern knowledge bases to
suggest plausible moves.
Additional Notes
• The next 5 slides are form David W. Aha (NRL)
presentation at Lehigh University in Fall’04
Example Game: FreeCiv
(Chance, adversarial, imperfect information game)
Civilization II (MicroProse)
• Civilization II (1996-): 850K+ copies sold
– PC Gamer: Game of the Year Award winner
– Many other awards
• Civilization series (1991-): Introduced the civilization-based game genre
FreeCiv (Civ II clone)
•Open source freeware
•Discrete strategy game
•Goal: Defeat opponents,
or build a spaceship
•Resource management
– Economy, diplomacy,
science, cities, buildings,
world wonders
– Units (e.g., for combat)
•Up to 7 opponent civs
•Partial observability
http://www.freeciv.org
FreeCiv Scenario
General description
• Game initialization: Your only unit, a “settler”, is placed randomly on a random
world (see Game Options below). Players cyclically alternate play
• Objective: Obtain highest score, conquer all opponents, or build first spaceship
• Scoring: “Basic” goal is to obtain 1000 points. Game options affect the score.
– Citizens: 2 pts per happy citizen, 1 per content citizen
– Advances: 20 pts per World Wonder, 5 per “futuristic” advance
– Peace: 3 pts per turn of world peace (no wars or combat)
– Pollution: -10pts per square currently polluted
• Top-level tasks (to achieve a high score):
– Develop an economy
– Increase population
– Pursue research advances
– Opponent interactions: Diplomacy and defense/combat
Game Option
Y1
Y2
Y3
World size
Small
Normal
Large
Difficulty level
Warlord (2/6)
Prince (3/6)
King (4/6)
#Opponent civilizations
5
5
7
Level of barbarian activity
Low
Medium
High
FreeCiv Concepts
Concepts in an Initial Knowledge Base
• Resources: Collection and use
o Food, production, trade (money)
• Terrain:
o Resources gained per turn
o Movement requirements
• Units:
o Type (Military, trade, diplomatic, settlers, explorers)
o Health
o Combat: Offense & defense
o Movement constraints (e.g., Land, sea, air)
•
•
•
•
Government Types (e.g., anarchy, despotism, monarchy, democracy)
Research network: Identifies constraints on what can be studied at any time
Buildings (e.g., cost, capabilities)
Cities
o Population Growth
o Happiness
o Pollution
• Civilizations (e.g., military strength, aggressiveness, finances, cities, units)
• Diplomatic states & negotiations
FreeCiv Decisions
Civilization decisions
• Choice of government type (e.g., democracy)
• Distribution of income devoted to research, entertainment, and wealth goals
• Strategic decisions affecting other decisions (e.g., coordinated unit movement for trade)
City decisions
• Production choice (i.e., what to create, including city buildings and units)
• Citizen roles (e.g., laborers, entertainers, or specialists), and laborer placement
– Note: Locations vary in their terrain, which generate different amounts of food,
income, and production capability
Unit decisions
• Task (e.g., where to build a city, whether/where to engage in combat, espionage)
• Movement
Diplomacy decisions
• Whether to sign a proffered peace treaty with another civilization
• Whether to offer a gift
FreeCiv CP Decision Space
Variables
• Civilization-wide variables
o
o
o
o
o
N: Number of civilizations encountered
D: Number of diplomatic states (that you can have with an opponent)
G: Number of government types available to you
R: Number of research advances that can be pursued
I: Number of partitions of income into entertainment, money, & research
• U: #Units
o L: Number of locations a unit can move to in a turn
• C: #Cities
o Z: Number of citizens per city
o S: Citizen status (i.e., laborer, entertainer, doctor)
o B: Number of choices for city production
Decision complexity per turn (for a typical game state)
• O(DNGRI*LU*(SZB)C)
; this ignores both other variables and domain knowledge
o This becomes large with the number of units and cities
o Example: N=3; D=5; G=3; R=4; I=10; U=25; L=4; C=8; Z=10; S=3; B=10
o Size of decision space (i.e., possible next states): 2.5*1065 (in one turn!)
o Comparison: Decision space of chess per turn is well below 140 (e.g., 20 at first move)