Artificial Intelligence for Games Finite State Machines Patrick Olivier p.l.olivier@ncl.ac.uk

“Intelligent” Behaviours Restrict “behaviour” to “intelligent behaviours” For example: trajectory of a car is not “intelligent”  decision to fight and what to fight  decision to flee and what to flee from  Complex behaviours implement more intelligent opponents define a kind a “personality” for NPCs or opponents formalising behaviours makes gameplay explicit Behaviour of a vehicle in a car simulator is not classed as “intelligent”. Decisions are classed as “intelligent behaviours”. Complex behaviours can be seen as more intelligent opponents: defining a kind of personality. Can create formal models that explicitly model aspects of gameplay (suspense, resolution, sense of achievement, etc.)

Finite state machines: theory Simple theoretical construct: Set of states (S) Input vocabulary (I) Transition function T(s,i) A way of denoting how an object can change its state over time 1 2 3 4 5 b a c d e f g Behaviour model with a finite set of states (including a start state, and a set of accept/end states), transitions between the states based on the input, and (typically) actions to be performed on entry/exit/input or transition. Typically modelled using a state transition diagram, or a state transition table.

Finite state machines: practice Character AI modeled as a sequence of mental states World events can force a state change Easy to grasp, even for non-programmers Input to the FSM continues as long as the game continues Gather Treasure Flee Fight Monster In Sight No Monster Monster Dead Cornered Behaviour model.

Finite state machines: example States Atttack Inspect Spawn Patrol Events E: see an enemy S: hear a sound D: die Action performed On each transition On each update in some states (e.g. attack) Spawn D Inspect ~E Attack E,~D E S Patrol ~S

Pac-man: Blinky (the red ghost) (In depth detail for how the “Chase Player” state actually works) [more of an empirical/procedural scripted solution] 30 year old game, simple AI rules coded in a few lines of assembly that produced relatively complex results – each ghost seemed to have a subtle ‘personality’. Blinky (red): Appears to chase the pac-man. Simply finds the relative position (without considering the direction) and at each junction will turn to reduce whichever is the greater of the horizontal distance or vertical distance between himself and Pac-Man. Pinky (pink): Appears to help ambush you in association with Blinky. Input is four tiles in front of Pac-Man (taking into account both the position and the direction of the Pac-Man). Attempts to home in on this target using the same logic as described for red. Inky (blue): Appears to only attack when Blinky is around. Actually homing in on a target that is at the end of a line segment from Blinky to two spaces ahead of the pac-man, doubled in length. Clyde (orange): Appears to be not very interested in attacking. Actually has a ‘far’ and a ‘near’ mode. When far from pac-man (further than 8 tiles), he uses Blinky’s logic to close in. However, when near to pac-man (within 8 tiles) he instead heads off for his corner of the maze (lower left). (Edible ghosts: Random seed generator called at each junction. If a wall, attempts the remaining directions in turn.)

FSMs: advantages & disadvantages Easy to understand Easy to implement Flexible: suitable for a range of problems Disadvantages: Scalability: can be cumbersome State oscillation State oscillation: state change with actions => exceed threshold for transition (quantized values) => state change with actions => exceed threshold for transition => (repeat) = oscillating states.

FSMs: implementation issues Polling Simple and easy to debug Inefficient since FSM’s are always evaluated Event Driven Model FSM registers which events it is interested in Requires Observer model in engine Multithreaded Each FSM assigned its own thread Requires thread-safe communication Conceptually elegant Difficult to debug Alternatively: Microthreading/co-routines. For example (not FSA but), as Linden Lab’s Second Life executes “LSL” scripts in their modified Mono virtual machine.

Class exercise 10 minutes exercise work in pairs or threes pick a game scenario agree on percepts for the scenario each develop an FSM for a different NPCs aim for interesting interactions: between player and NPCs between NPCs hand in a “final” sheet (with names on it) Percepts = perceived form of some stimuli.

Extensions: Hierarchical FSMs expand a state into its own sub-FSM some events move you around the same level in the hierarchy, some move you up a level when entering a state, choose a sub-state: set a default, and always go to that random choice depends on the nature of the behaviour Better scaling, re-use of logic in different contexts.

Extensions: Hierarchical FSMs Attack Wander ~E Chase E Pick-up Powerup ~S S Start Turn Right D Spawn “Super state” has “generalized transitions” that don’t need to be applied to each of the sub-states. Re-use of sub-states could be allowed. ~E Go-through Door

Extensions: stack-based FSMs Gives the AI a (limited) form of ‘memory’ Switch states, then return to a previous state Permits temporary interruption More realistic behaviour Not all states stack-able Replacement behaviours Place FSM state in a stack Example: Thief 3 – NPC could be performing some action (chatting, patrolling), interrupted to perform search.

Extensions: inertial FSMs Need to address the oscillation problem For example – basketball: LOS=FALSE  stand-state LOS=TRUE  basket-drive-state Introduce inertia to dampen state changes State inertia (e.g. minimum running times) Perceptual inertia (e.g. multiple firings) LOS = Line of sight State oscillation: state change with actions => exceed threshold for transition (quantized values) => state change with actions => exceed threshold for transition => (repeat) = oscillating states.

Other extensions of FSMs… Inter-character concurrent FSM Coordinating multiple characters Intra-character concurrent FSM Coordinating multiple behaviours (in one NPC) Levels of detail (LODs) Analogous to LOD in graphics e.g. crowd simulation Proximal NPCs use fully elaborated FSM Distal characters use fixed paths or worse Inter-character concurrent FSM – coordinates behaviour between characters. Intra-character concurrent FSM – coordinates multiple behaviours within on character.

Nondeterministic FSMs Probabilistic FSMs multiple transitions for same event each transition has probability that it will be taken probabilistically choose a transition at run-time Attack Start Approach Jump & Shoot Aim & Shoot Slide .1 .3 .4 .2 In deterministic finite automaton the next possible state is uniquely determined. However, a nondeterministic finite state machine is a finite state machine where for each pair of state and input symbol there may be several possible next states.

Questions? Behaviour model with a finite set of states start state set of accept/end states transitions between the states (based on the input) actions to be performed on entry/exit/input or transition - Lambda (null) transitions – can remove and collapse states. Representation: state transition diagram state transition table. [Additional: In depth detail for how the “Chase Player” Pac-Man state actually works – as an empirical/procedural scripted solution]