1. Artificial Intelligence for Games Finite State Machines
Patrick Olivier

2. “Intelligent” Behaviours
Restrict “behaviour” to “intelligent behaviours”
For example:
trajectory of a car is not “intelligent” 
decision to fight and what to fight 
decision to flee and what to flee from 
Complex behaviours implement more intelligent opponents
define a kind a “personality” for NPCs or opponents
formalising behaviours makes gameplay explicit
Behaviour of a vehicle in a car simulator is not classed as “intelligent”.
Decisions are classed as “intelligent behaviours”.
Complex behaviours can be seen as more intelligent opponents: defining a kind of personality.
Can create formal models that explicitly model aspects of gameplay (suspense, resolution, sense of achievement, etc.)

3. Finite state machines: theory
Simple theoretical construct:
Set of states (S)
Input vocabulary (I)
Transition function T(s,i)
A way of denoting how an object can change its state over time 1 2 3 4 5 b a c d e f g
Behaviour model with a finite set of states (including a start state, and a set of accept/end states), transitions between the states based on the input, and (typically) actions to be performed on entry/exit/input or transition.
Typically modelled using a state transition diagram, or a state transition table.

4. Finite state machines: practice
Character AI modeled as a sequence of mental states
World events can force a state change
Easy to grasp, even for non-programmers
Input to the FSM continues as long as the game continues
Gather
Treasure
Flee
Fight
Monster In Sight
No Monster
Monster Dead
Cornered
Behaviour model.

5. Finite state machines: example
States
Atttack
Inspect
Spawn
Patrol
Events
E: see an enemy
S: hear a sound
D: die
Action performed
On each transition
On each update in some states (e.g. attack)
Spawn D
Inspect ~E
Attack
E,~D E S
Patrol ~S

6. Pac-man: Blinky (the red ghost)
(In depth detail for how the “Chase Player” state actually works) [more of an empirical/procedural scripted solution]
30 year old game, simple AI rules coded in a few lines of assembly that produced relatively complex results – each ghost seemed to have a subtle ‘personality’.
Blinky (red): Appears to chase the pac-man. Simply finds the relative position (without considering the direction) and at each junction will turn to reduce whichever is the greater of the horizontal distance or vertical distance between himself and Pac-Man. Pinky (pink): Appears to help ambush you in association with Blinky. Input is four tiles in front of Pac-Man (taking into account both the position and the direction of the Pac-Man). Attempts to home in on this target using the same logic as described for red.
Inky (blue): Appears to only attack when Blinky is around. Actually homing in on a target that is at the end of a line segment from Blinky to two spaces ahead of the pac-man, doubled in length.
Clyde (orange): Appears to be not very interested in attacking. Actually has a ‘far’ and a ‘near’ mode. When far from pac-man (further than 8 tiles), he uses Blinky’s logic to close in. However, when near to pac-man (within 8 tiles) he instead heads off for his corner of the maze (lower left).
(Edible ghosts: Random seed generator called at each junction. If a wall, attempts the remaining directions in turn.)

8. FSMs: advantages & disadvantages
Easy to understand
Easy to implement
Flexible: suitable for a range of problems
Disadvantages:
Scalability: can be cumbersome
State oscillation
State oscillation: state change with actions => exceed threshold for transition (quantized values) => state change with actions => exceed threshold for transition => (repeat)
= oscillating states.

9. FSMs: implementation issues
Polling
Simple and easy to debug
Inefficient since FSM’s are always evaluated
Event Driven Model
FSM registers which events it is interested in
Requires Observer model in engine
Multithreaded
Each FSM assigned its own thread
Requires thread-safe communication
Conceptually elegant
Difficult to debug
Alternatively: Microthreading/co-routines. For example (not FSA but), as Linden Lab’s Second Life executes “LSL” scripts in their modified Mono virtual machine.

10. Class exercise 10 minutes exercise work in pairs or threes
pick a game scenario
agree on percepts for the scenario
each develop an FSM for a different NPCs
aim for interesting interactions:
between player and NPCs
between NPCs
hand in a “final” sheet (with names on it)
Percepts = perceived form of some stimuli.

11. Extensions: Hierarchical FSMs
expand a state into its own sub-FSM
some events move you around the same level in the hierarchy, some move you up a level
when entering a state, choose a sub-state:
set a default, and always go to that
random choice
depends on the nature of the behaviour
Better scaling, re-use of logic in different contexts.

12. Extensions: Hierarchical FSMs
Attack
Wander ~E
Chase E
Pick-up
Powerup ~S S
Start
Turn Right D
Spawn
“Super state” has “generalized transitions” that don’t need to be applied to each of the sub-states.
Re-use of sub-states could be allowed. ~E
Go-through
Door

13. Extensions: stack-based FSMs
Gives the AI a (limited) form of ‘memory’
Switch states, then return to a previous state
Permits temporary interruption
More realistic behaviour
Not all states stack-able
Replacement behaviours
Place FSM state in a stack
Example: Thief 3 – NPC could be performing some action (chatting, patrolling), interrupted to perform search.

14. Extensions: inertial FSMs
Need to address the oscillation problem
For example – basketball:
LOS=FALSE  stand-state
LOS=TRUE  basket-drive-state
Introduce inertia to dampen state changes
State inertia (e.g. minimum running times)
Perceptual inertia (e.g. multiple firings)
LOS = Line of sight
State oscillation: state change with actions => exceed threshold for transition (quantized values) => state change with actions => exceed threshold for transition => (repeat)
= oscillating states.

15. Other extensions of FSMs…
Inter-character concurrent FSM
Coordinating multiple characters
Intra-character concurrent FSM
Coordinating multiple behaviours (in one NPC)
Levels of detail (LODs)
Analogous to LOD in graphics
e.g. crowd simulation
Proximal NPCs use fully elaborated FSM
Distal characters use fixed paths or worse
Inter-character concurrent FSM – coordinates behaviour between characters.
Intra-character concurrent FSM – coordinates multiple behaviours within on character.

16. Nondeterministic FSMs
Probabilistic FSMs
multiple transitions for same event
each transition has probability that it will be taken
probabilistically choose a transition at run-time
Attack
Start
Approach
Jump &
Shoot
Aim
& Shoot
Slide .1 .3 .4 .2
In deterministic finite automaton the next possible state is uniquely determined. However, a nondeterministic finite state machine is a finite state machine where for each pair of state and input symbol there may be several possible next states.

17. Questions? Behaviour model with a finite set of states start state
set of accept/end states
transitions between the states (based on the input)
actions to be performed on entry/exit/input or transition
- Lambda (null) transitions – can remove and collapse states.
Representation:
state transition diagram
state transition table.
[Additional: In depth detail for how the “Chase Player” Pac-Man state actually works – as an empirical/procedural scripted solution]