1. Finite State Machine for Games Spring 2005 Ref: Chenney, CS679 lectures AI Game Programming Wisdom 2

2. Spring 20052 Outline AI and Game Introduction/examples Design Intuition State-based Implementation Extending Stack-based Fuzzy-state machine

3. Spring 20053 What is AI? AI is the control of every non-human entity in a game The other cars in a car game The opponents and monsters in a shooter Your units, your enemy ’ s units and your enemy in a RTS game But, typically does not refer to passive things that just react to the player and never initiate action That ’ s physics or game logic For example, the blocks in Tetris are not AI, nor is a flag blowing in the wind

4. Spring 20054 AI in the Game Loop AI is updated as part of the game loop, after user input, and before rendering There are issues here: Which AI goes first? Does the AI run on every frame? (LOD problem) Is the AI synchronized?

5. Spring 20055 AI and Animation AI determines what to do and the animation does it AI drives animation, deciding what action the animation system should be animating Scenario 1: The AI issues orders like “ move from A to B ”, and it ’ s up to the animation system to do the rest Scenario 2: The AI controls everything down to the animation clip to play Which scenario is best depends on the nature of the AI system and the nature of the animation system Is the animation system based on move trees (motion capture), or physics, or something else Does the AI look after collision avoidance? Does it do detailed planning?

6. Spring 20056 AI Module AI Update Step The sensing phase determines the state of the world May be very simple - state changes all come by message Or complex - figure out what is visible, where your team is, etc The thinking phase decides what to do given the world The core of AI The acting phase tells the animation what to do Generally not interesting Game Engine Sensing Thinking Acting

7. Spring 20057 AI by Polling The AI gets called at a fixed rate Senses: It looks to see what has changed in the world. For instance: Queries what it can see Checks to see if its animation has finished running And then acts on it Why is this generally inefficient? Different characters might require different polling rate

8. Spring 20058 Event Driven AI Event driven AI does everything in response to events in the world Events sent by message (basically, a function gets called when a message arrives, just like a user interface) Example messages: A certain amount of time has passed, so update yourself You have heard a sound Someone has entered your field of view Note that messages can completely replace sensing, but typically do not. Why not? Real system are a mix - something changes, so you do some sensing

9. Spring 20059 AI Techniques in Games Basic problem: Given the state of the world, what should I do? A wide range of solutions in games: Finite state machines, Decision trees, Rule based systems, Neural networks, Fuzzy logic A wider range of solutions in the academic world: Complex planning systems, logic programming, genetic algorithms, Bayes-nets Typically, too slow for games

10. Spring 200510 Goals of Game AI Several goals: Goal driven - the AI decides what it should do, and then figures out how to do it Reactive - the AI responds immediately to changes in the world Knowledge intensive - the AI knows a lot about the world and how it behaves, and embodies knowledge in its own behavior Characteristic - Embodies a believable, consistent character Fast and easy development Low CPU and memory usage These conflict in almost every way

11. Spring 200511 Two Measures of Complexity Complexity of Execution How fast does it run as more knowledge is added? How much memory is required as more knowledge is added? Determines the run-time cost of the AI Complexity of Specification How hard is it to write the code? As more “ knowledge ” is added, how much more code needs to be added? Determines the development cost, and risk

12. Spring 200512 Expressiveness What behaviors can easily be defined, or defined at all? Propositional logic: Statements about specific objects in the world – no variables Jim is in room7, Jim has the rocket launcher, the rocket launcher does splash damage Go to room8 if you are in room7 through door14 Predicate Logic: Allows general statement – using variables All rooms have doors All splash damage weapons can be used around corners All rocket launchers do splash damage Go to a room connected to the current room

13. Spring 200513 Finite State Machines (FSMs) A set of states that the agent can be in Connected by transitions that are triggered by a change in the world Normally represented as a directed graph, with the edges labeled with the transition event Ubiquitous in computer game AI You might have seen them, a long time ago, in formal language theory (or compilers) What type of languages can be represented by finite state machines? How might this impact a character ’ s AI? How does it impact the size of the machine?

14. Spring 200514 Formal Definitions (N. Philips) "An abstract machine consisting of a set of states (including the initial state), a set of input events, a set of output events, and a state transition function. The function takes the current state and an input event and returns the new set of output events and the next state. Some states may be designated as "terminal states". The state machine can also be viewed as a function which maps an ordered sequence of input events into a corresponding sequence of (sets of) output events. Finite State Automaton: the machine with no output

15. Spring 200515 FSM with Output: vending machines [description] State table

16. Spring 200516 Vending Machine: state diagram

17. Spring 200517 FSM and Game Game character behavior can be modeled (in most cases) as a sequence of different “ mental state ”, where change is driven by the actions of player/other characters, … Natural choice for defining AI in games

18. Spring 200518 FSM with No Output

19. Spring 200519 Ex: predator vs. prey Prey (laalaa) Idle (stand,wave, … ) Flee (run) Sees predator No more threat captured Dead

20. Spring 200520 Predator (Raptor) Idle (stand) Hungry (wander) Pursuit (run) T idle > 5 Prey in sight T pursuit > 10 Dining Prey captured T dining >5

21. Spring 200521 Idling LaaLaa Stand Wave Wander (set random target) 50% 30% 20% Target arrived T wave >2 R T stand >4 This page illustrates: hierarchical state, Non-deterministic state transition This page illustrates: hierarchical state, Non-deterministic state transition

22. Spring 200522

23. Spring 200523 FSM + Cinematography The current state (and state transition) may have an important effect on how camera should be manipulated Idling: focus on character Movement: zoom out to show spatial info State-transition: camera animation … More complicated camera behavior may be coded in another FSM

24. Spring 200524 Camera Selection Strategy It is important to choose a camera that has the front of the character Z X

25. Spring 200525 Camera Selection (cont) -v A good camera: -v lies in the frustum

26. Spring 200526 Other Concerns Adjust zoom (fovy) so that the character occupies a good (fixed!?) portion of the screen x w r near d

27. Spring 200527 FOVY based on Postures Once in running mode, zoom out In stand/wave modes, zoom in Smooth transition between two fovy settings (by indirectly controlling the percent value)

28. FSM Design

29. Spring 200529 Quake2 Examples Quake2 uses 9 different states: standing, walking, running, dodging, attacking, melee, seeing the enemy, idle and searching. Incomplet e design Intuitive thinking: model the events and state changes

30. Spring 200530 Quake: Rocket

31. Spring 200531 Shambler monster

32. Spring 200532 Intuitive Design Say, a simple teletube baby has three states: idle, run, and wave Scenario: When an idle laalaa sees a butterfly, it waves to it. When the butterfly flies away, it returns to idle When an idle laalaa sees a mouse, it flees away. When the mouse is no longer in sight, it returns to idle

33. Spring 200533 Laalaa idle flee wave mouse ~mouse ~butterfly butterfly How to make sure the design complete? I.e., all states and transitions are considered Is there any systematic way of developing an FSM?

34. Spring 200534 Quake Bot Example (first-cut) Types of behavior to capture: Wander randomly if don ’ t see or hear an enemy When see enemy, attack When not see enemy and hear an enemy, chase enemy When die, re-spawn (new a bot from start) Events: see enemy, hear enemy, die States: wander, attack, chase, spawn

35. Spring 200535 Remark With 3 events, potentially there should be 2 3 states: (E,S,D)=(0,0,0),(1,0,0),(0,1,0), …,(1,1,1) Some doesn ’ t make sense E.g., ESD = 111 Name and create a state for the ones that we want to consider Wander (ESD=000) Chase (ESD=010) Attack (ESD=1x0), x for dont-care Die (ESD=xx1)

36. Spring 200536 FSM (first-cut) Events: E: see an enemy S: hear a sound D: die States: E: enemy in sight S: sound audible D: dead D ~E E D ~S S D ESD Problem: Can’t go directly from attack to chase. Why not? Spawn xx1 Wander 000 Attack 1x0 Chase 010 start

37. Spring 200537 FSM (first-cut) Events: E: see an enemy S: hear a sound D: die States: E: enemy in sight S: sound audible D: dead D ~E E D ~S S D ESD Spawn xx1 Wander 000 Attack 100 Chase 010 start E Attack+S 110 ~E ~S S D Extra state needs to be defined

38. Spring 200538 Quake Bot Example (refined) Types of behavior to capture: Wander randomly if don ’ t see or hear an enemy When see enemy, attack When not see enemy and hear an enemy, chase enemy When die, respawn Extensions: When health is low and see an enemy, retreat When see power-ups during wandering, collect them [hierarchical FSM]

39. Spring 200539 Example FSM with Retreat Spawn D (-E,-S,-L) Wander -E,-D,-S,-L E -S Attack-E E,-D,-S,-L E Chase -E,-D,S,-L S D S D D Retreat-E E,-D,-S,L L -E Retreat-S -E,-D,S,L Wander-L -E,-D,-S,L Retreat-ES E,-D,S,L Attack-ES E,-D,S,-L E E -E -L S -S L -E E L -L L D States: –E: enemy in sight –S: sound audible –D: dead –L: Low health A lot more states got added

40. Spring 200540 Hierarchical FSMs What if there is no simple action for a state? Expand a state into its own FSM, which explains what to do if in that state Some events move you around the same level in the hierarchy, some move you up a level When entering a state, have to choose a state for its child in the hierarchy Set a default, and always go to that Or, random choice Depends on the nature of the behavior

41. Spring 200541 Hierarchical FSM Example Note: This is not a complete FSM All links between top level states still exist Need more states for wander Start Turn Right Go-through Door Pick-up Powerup Wander Attack Chase Spawn ~E E ~S S D ~E

42. Spring 200542 Non-Deterministic Hierarchical FSM (Markov Model) Adds variety to actions Have multiple transitions for the same event Label each with a probability that it will be taken Randomly choose a transition at run-time Markov Model: New state only depends on the previous state Attack Start Approach Aim & Jump & Shoot Aim & Slide Left & Shoot Aim & Slide Right & Shoot.3.4.3.4

43. FSM Control System Implementation

44. Spring 200544 FSM Implementation

45. Spring 200545 Efficient Implementation Compile into an array of state-name, event state-name i+1 := array[state-name i, event] Switch on state-name to call execution logic Markov: Have array of possible transitions for every (state-name,event) pair, and choose one at random Hierarchical Create array for every FSM Have stack of states  Classify events according to stack  Update state which is sensitive to current event event state

46. Spring 200546 FSM Advantages Very fast – one array access Expressive enough for simple behaviors or characters that are intended to be “ dumb ” Can be compiled into compact data structure Dynamic memory: current state Static memory: state diagram – array implementation Can create tools so non-programmer can build behavior Non-deterministic FSM can make behavior unpredictable

47. Spring 200547 FSM Disadvantages Number of states can grow very fast Exponentially with number of events: s=2 e Number of arcs can grow even faster: a=s 2 Propositional representation Difficult to put in “ pick up the better powerup ”, “ attack the closest enemy ” Expensive to count: Wait until the third time I see enemy, then attack  Need extra events: First time seen, second time seen, and extra states to take care of counting

48. Spring 200548 Example

49. Spring 200549 Code 1 Ad hoc implementation

50. Spring 200550 Code 1p

51. Spring 200551 Code 2 Structure, Readable, maintainable

52. Spring 200552 Hierarchical …

53. FSM Extensions

54. Spring 200554 Stack-based FSM History stack: Remember previous state; create characters with a memory … Pushdown automaton (PDA)

55. Spring 200555 Goal-based vs. State-based There is also a slight derivative to the state- based engine, but it used in more complicated games like flight simulators and games like MechWarrior. They use goal - based engines - each entity within the game is assigned a certain goal, be it 'protect base', 'attack bridge', 'fly in circles'. As the game world changes, so do the goals of the various entities.

56. Spring 200556 Processing Models Polling FSM update frequency Easy to implement and debug Inefficiency (Little Red example) Event-driven Publish-subscribe messaging system Game engine sends event messages to individual FSMs An FSM subscribes only to the events that have the potential to change the current state Higher efficiency, non- trivial implementation

57. Spring 200557 Interfacing with Game Engine Query, act on the game world Move the function calls (to the game engine) as DLL; reduce the amount of recompilation required

58. Spring 200558 Efficiency and Optimization In AI, FSM is the most efficient technology available Yet, there is always room for improvement Level of Detail: depending on the condition (e.g., distance with player), use different FSM, or different update frequency

59. Spring 200559 References Web references: www.gamasutra.com/features/19970601/build_brains_into_ games.htm www.gamasutra.com/features/19970601/build_brains_into_ games.htm csr.uvic.ca/~mmania/machines/intro.htm www.erlang/se/documentation/doc- 4.7.3/doc/design_principles/fsm.html www.erlang/se/documentation/doc- 4.7.3/doc/design_principles/fsm.html www.microconsultants.com/tips/fsm/fsmartcl.htm http://www.angelfire.com/dragon/letstry/tutorials/dfa/ Game Programming Gems Sections 3.0 & 3.1 It ’ s very very detailed, but also some cute programming