Transcript Finite State Machine for Games

Finite State Machine for Games
Spring 2005
Ref: Chenney, CS679 lectures
AI Game Programming Wisdom 2
Outline
AI and Game
Introduction/examples
Design


Intuition
State-based
Implementation
Extending


Stack-based
Fuzzy-state machine
Spring 2005
2
What is AI?
AI is the control of every non-human entity in a game



The other cars in a car game
The opponents and monsters in a shooter
Your units, your enemy’s units and your enemy in a RTS
game
But, typically does not refer to passive things that
just react to the player and never initiate action


That’s physics or game logic
For example, the blocks in Tetris are not AI, nor is a flag
blowing in the wind
Spring 2005
3
AI in the Game Loop
AI is updated as part of the game loop,
after user input, and before rendering
There are issues here:



Which AI goes first?
Does the AI run on every frame? (LOD
problem)
Is the AI synchronized?
Spring 2005
4
AI and Animation
AI determines what to do and the animation does it



AI drives animation, deciding what action the animation
system should be animating
Scenario 1: The AI issues orders like “move from A to B”,
and it’s up to the animation system to do the rest
Scenario 2: The AI controls everything down to the
animation clip to play
Which scenario is best depends on the nature of the
AI system and the nature of the animation system


Is the animation system based on move trees (motion
capture), or physics, or something else
Does the AI look after collision avoidance? Does it do
detailed planning?
Spring 2005
5
AI Update Step
The sensing phase determines
the state of the world


May be very simple - state
changes all come by message
Or complex - figure out what is
visible, where your team is, etc
The thinking phase decides
what to do given the world

Sensing
Game
Engine
Thinking
The core of AI
The acting phase tells the
animation what to do

AI Module
Generally not interesting
Spring 2005
Acting
6
AI by Polling
The AI gets called at a fixed rate
Senses: It looks to see what has changed in
the world. For instance:


Queries what it can see
Checks to see if its animation has finished running
And then acts on it
Why is this generally inefficient?

Different characters might require different polling
rate
Spring 2005
7
Event Driven AI
Event driven AI does everything in response to
events in the world

Events sent by message (basically, a function gets called
when a message arrives, just like a user interface)
Example messages:



A certain amount of time has passed, so update yourself
You have heard a sound
Someone has entered your field of view
Note that messages can completely replace sensing,
but typically do not. Why not?

Real system are a mix - something changes, so you do some
sensing
Spring 2005
8
AI Techniques in Games
Basic problem: Given the state of the world,
what should I do?
A wide range of solutions in games:

Finite state machines, Decision trees, Rule based
systems, Neural networks, Fuzzy logic
A wider range of solutions in the academic
world:


Complex planning systems, logic programming,
genetic algorithms, Bayes-nets
Typically, too slow for games
Spring 2005
9
Goals of Game AI
Several goals:






Goal driven - the AI decides what it should do, and then
figures out how to do it
Reactive - the AI responds immediately to changes in the
world
Knowledge intensive - the AI knows a lot about the world
and how it behaves, and embodies knowledge in its own
behavior
Characteristic - Embodies a believable, consistent character
Fast and easy development
Low CPU and memory usage
These conflict in almost every way
Spring 2005
10
Two Measures of Complexity
Complexity of Execution



How fast does it run as more knowledge is added?
How much memory is required as more
knowledge is added?
Determines the run-time cost of the AI
Complexity of Specification



How hard is it to write the code?
As more “knowledge” is added, how much more
code needs to be added?
Determines the development cost, and risk
Spring 2005
11
Expressiveness
What behaviors can easily be defined, or defined at
all?
Propositional logic:



Statements about specific objects in the world – no variables
Jim is in room7, Jim has the rocket launcher, the rocket
launcher does splash damage
Go to room8 if you are in room7 through door14
Predicate Logic:





Allows general statement – using variables
All rooms have doors
All splash damage weapons can be used around corners
All rocket launchers do splash damage
Go to a room connected to the current room
Spring 2005
12
Finite State Machines (FSMs)
A set of states that the agent can be in
Connected by transitions that are triggered by a
change in the world
Normally represented as a directed graph, with the
edges labeled with the transition event
Ubiquitous in computer game AI
You might have seen them, a long time ago, in
formal language theory (or compilers)



What type of languages can be represented by finite state
machines?
How might this impact a character’s AI?
How does it impact the size of the machine?
Spring 2005
13
Formal Definitions (N. Philips)
"An abstract machine consisting of a set of states
(including the initial state), a set of input events, a
set of output events, and a state transition function.
The function takes the current state and an input
event and returns the new set of output events and
the next state. Some states may be designated as
"terminal states".
The state machine can also be viewed as a function
which maps an ordered sequence of input events into
a corresponding sequence of (sets of) output events.
Finite State Automaton: the machine with no output
Spring 2005
14
FSM with Output: vending
machines
[description]
State table
Spring 2005
15
Vending Machine: state diagram
Spring 2005
16
FSM and Game
Game character behavior can be
modeled (in most cases) as a sequence
of different “mental state”, where
change is driven by the actions of
player/other characters, …
Natural choice for defining AI in games
Spring 2005
17
FSM with No Output
Spring 2005
18
Ex: predator vs. prey
Prey (laalaa)
Sees predator
Idle
(stand,wave,…)
No more threat
Dead
Spring 2005
Flee
(run)
captured
19
Predator (Raptor)
Idle
(stand)
Tdining>5
Tidle > 5
Hungry
(wander)
Dining
Prey captured
Pursuit
(run)
Prey in sight
Tpursuit > 10
Spring 2005
20
Idling LaaLaa
This page illustrates:
hierarchical state,
Non-deterministic state transition
Target arrived
Wander
(set random target)
Stand
Tstand>4
20%
R 30%
50%
Wave
Twave>2
Spring 2005
21
Spring 2005
22
FSM + Cinematography
The current state (and state transition)
may have an important effect on how
camera should be manipulated



Idling: focus on character
Movement: zoom out to show spatial info
State-transition: camera animation …
More complicated camera behavior may
be coded in another FSM
Spring 2005
23
Camera Selection Strategy
It is important to choose a
camera that has the front
of the character
X
Spring 2005
Z
24
Camera Selection (cont)
A good camera:
-v lies in the frustum
-v
Spring 2005
25
Other Concerns
• Adjust zoom (fovy) so that the character occupies a
good (fixed!?) portion of the screen
r
d
x
near
Spring 2005
w
x / 2 near

r
d
x
percent 
 : fovy
w
x/2
 w / 2 percent
tan 

2 near near
r
near d  percent
r


near
d  percent
26
FOVY based on Postures
Once in running mode, zoom out
In stand/wave modes, zoom in
Smooth transition between two fovy
settings (by indirectly controlling the
percent value)
Spring 2005
27
FSM Design
Quake2 Examples
Intuitive thinking:
model the events and
state changes
Quake2 uses 9 different states:
standing, walking, running, dodging, attacking, melee,
seeing the enemy, idle and searching.
Incomplet
e design
Spring 2005
29
Quake: Rocket
Spring 2005
30
Shambler monster
Spring 2005
31
Intuitive Design
Say, a simple teletube baby has three
states: idle, run, and wave
Scenario:


When an idle laalaa sees a butterfly, it
waves to it. When the butterfly flies away,
it returns to idle
When an idle laalaa sees a mouse, it flees
away. When the mouse is no longer in
sight, it returns to idle
Spring 2005
32
Laalaa
flee
mouse
~mouse
How to make sure the design
complete? I.e., all states and
transitions are considered
idle
butterfly
~butterfly
Spring 2005
wave
Is there any systematic way of
developing an FSM?
33
Quake Bot Example (first-cut)
Types of behavior to capture:




Wander randomly if don’t see or hear an enemy
When see enemy, attack
When not see enemy and hear an enemy, chase
enemy
When die, re-spawn (new a bot from start)
Events: see enemy, hear enemy, die
States: wander, attack, chase, spawn
Spring 2005
34
Remark
With 3 events, potentially there should be 23
states:

(E,S,D)=(0,0,0),(1,0,0),(0,1,0), …,(1,1,1)
Some doesn’t make sense

E.g., ESD = 111
Name and create a state for the ones that we
want to consider



Wander (ESD=000)
Chase (ESD=010)
Attack (ESD=1x0), x for dont-care
Die (ESD=xx1)

Spring 2005
35
FSM (first-cut)
Attack
1x0
~E
start
Problem: Can’t go directly from
attack to chase. Why not?
E
Events:
D
Wander
000

S
Chase
010
~S

D
D
Spawn
xx1

States:
E SD



Spring 2005
E: see an enemy
S: hear a sound
D: die
E: enemy in sight
S: sound audible
D: dead
36
FSM (first-cut)
Attack+S
110
~S
Attack
100
~E
start
E
S
D
D
Wander
000
~E
Chase
010
~S
Spawn
xx1
E
Events:

S


D
D
Extra state
needs to be
defined
States:
E SD



Spring 2005
E: see an enemy
S: hear a sound
D: die
E: enemy in sight
S: sound audible
D: dead
37
Quake Bot Example (refined)
Types of behavior to capture:




Wander randomly if don’t see or hear an enemy
When see enemy, attack
When not see enemy and hear an enemy, chase
enemy
When die, respawn
Extensions:


When health is low and see an enemy, retreat
When see power-ups during wandering, collect
them [hierarchical FSM]
Spring 2005
38
Example FSM with Retreat
Attack-ES
E,-D,S,-L
S
Attack-E
E,-D,-S,-L
-S
L
• States:
Retreat-S
-E,-D,S,L
L
-L
E
-E
E E
Wander-L
-E,-D,-S,L
L
-L
-L
E
L
Retreat-ES
E,-D,S,L
-S
-L
S
-E
Wander
-E E
-E,-D,-S,-L
Retreat-E
E,-D,-S,L
– E: enemy in
sight
– S: sound
audible
– D: dead
– L: Low health
A lot more states
got added
D D
D
Spring 2005
D
Spawn
D
(-E,-S,-L)
Chase
-E,-D,S,-L
S
39
Hierarchical FSMs
What if there is no simple action for a state?
Expand a state into its own FSM, which explains what
to do if in that state
Some events move you around the same level in the
hierarchy, some move you up a level
When entering a state, have to choose a state for its
child in the hierarchy



Set a default, and always go to that
Or, random choice
Depends on the nature of the behavior
Spring 2005
40
Hierarchical FSM Example
Wander
Attack
~E
E
~S
Pick-up
Powerup
Chase
S
Start
Turn Right
D
Spawn
~E
Go-through
Door
Note: This is not a complete FSM

Spring 2005

All links between top level states still
exist
Need more states for wander 41
Non-Deterministic Hierarchical
FSM (Markov Model)
Attack
Approach
Aim &
Slide Right
& Shoot
.3
Aim &
Slide Left
& Shoot
Spring 2005
.3
.4
.3
.3
Start
.4
Aim &
Jump &
Shoot
Adds variety to actions
Have multiple transitions
for the same event
Label each with a
probability that it will be
taken
Randomly choose a
transition at run-time
Markov Model: New
state only depends on
the previous state
42
FSM
Control System Implementation
FSM Implementation
Spring 2005
44
Efficient Implementation
Compile into an array of state-name,
event
state-namei+1 := array[state-namei, event]
Switch on state-name to call execution
logic
Markov: Have array of possible transitions
for every (state-name,event) pair, and
choose one at random
Hierarchical


event
state
Create array for every FSM
Have stack of states
 Classify events according to stack
 Update state which is sensitive to current
event
Spring 2005
45
FSM Advantages
Very fast – one array access
Expressive enough for simple behaviors or characters
that are intended to be “dumb”
Can be compiled into compact data structure


Dynamic memory: current state
Static memory: state diagram – array implementation
Can create tools so non-programmer can build
behavior
Non-deterministic FSM can make behavior
unpredictable
Spring 2005
46
FSM Disadvantages
Number of states can grow very fast

Exponentially with number of events: s=2e
Number of arcs can grow even faster: a=s2
Propositional representation


Difficult to put in “pick up the better powerup”, “attack the
closest enemy”
Expensive to count: Wait until the third time I see enemy,
then attack
 Need extra events: First time seen, second time seen, and
extra states to take care of counting
Spring 2005
47
Example
Spring 2005
48
Code 1
Spring 2005
Ad hoc implementation
49
Code 1p
Spring 2005
50
Code 2
Spring 2005
Structure, Readable, maintainable
51
Hierarchical …
Spring 2005
52
FSM Extensions
Stack-based FSM
Pushdown automaton
(PDA)
History stack: Remember
previous state; create
characters with a
memory …
Spring 2005
54
Goal-based vs. State-based
There is also a slight derivative to the statebased engine, but it used in more
complicated games like flight simulators and
games like MechWarrior. They use goal based engines - each entity within the game
is assigned a certain goal, be it 'protect base',
'attack bridge', 'fly in circles'. As the game
world changes, so do the goals of the various
entities.
Spring 2005
55
Processing Models
Polling



FSM update
frequency
Easy to implement
and debug
Inefficiency (Little
Red example)
Event-driven




Spring 2005
Publish-subscribe
messaging system
Game engine sends
event messages to
individual FSMs
An FSM subscribes only
to the events that have
the potential to change
the current state
Higher efficiency, nontrivial implementation
56
Interfacing with Game Engine
Query, act on the
game world
Move the function
calls (to the game
engine) as DLL;
reduce the amount
of recompilation
required
Spring 2005
57
Efficiency and Optimization
In AI, FSM is the most efficient
technology available
Yet, there is always room for
improvement

Level of Detail: depending on the condition
(e.g., distance with player), use different
FSM, or different update frequency
Spring 2005
58
References
Web references:





www.gamasutra.com/features/19970601/build_brains_into_
games.htm
csr.uvic.ca/~mmania/machines/intro.htm
www.erlang/se/documentation/doc4.7.3/doc/design_principles/fsm.html
www.microconsultants.com/tips/fsm/fsmartcl.htm
http://www.angelfire.com/dragon/letstry/tutorials/dfa/
Game Programming Gems Sections 3.0 & 3.1

It’s very very detailed, but also some cute programming
Spring 2005
59