1. Reminder Next class (Tuesday), you should have formed groups and select a topic for presentation among: (1)Real-Time Strategy games (Luis Villegas, Dulmovits, Alexander J): I: 8.2, 8.3, II: 7.5, 7.6, III: 4.7, 6.7; IV: 4.1, History of RTS games? - informed (2)Squad tactics (John Formica, Matt Mitchell, David Pennenga ): I: 5.3, 5.4, 5.5. 5.6; II: 3.3, - informed (3)Scripting languages () I: Section 10 (7 chapters) (4) First Person Shooters (Michael Caffrey, Shamus Field, Jon Hardy): I: 8.1, 8.4; II: 7.1, II: 5.2, 5.3, history of FPS? - informed (5)Racing games () I: 9.1, 9.2, 9.3, 9.4, II: 8.1, 8.2, (6) Role Playing Games (Josh Westbrook, Ethan Harman) I: 8.5, history of RPGs? IV: “Stop Getting Side-Tracked by Side-Quests”, I: “Level-Of-Detail AI for a Large Role-Playing Game”, I: “A Dynamic Reputation System Based on Event Knowledge” - informed (7) Story line, drama (): IV: “Implementing Story-Driven Games with the Aid of Dynamical Policy Models”, IV: “Dialogue Managers”, (8)Sport Games (Daniel Phillips, Dan Ruthrauff) I: 9.7, II: 8.4, 8.5, - informed (9)Individual NPC behavior (Daniel Phang, Sui Ying Teoh): I: 2.4, 8.6, 9.6, II: 3.1, 3.2; IV: 5.6, http://www.aaai.org/Papers/AIIDE/2008/AIIDE08-025.pdf, - informed http://www.aaai.org/Papers/AIIDE/2008/AIIDE08-025.pdf (10) Player Modeling () IV 7.3, IV: “Player Modeling for Interactive Storytelling: A Practical Approach”, III: “Preference-Based Player Modeling”, II: “Player Modeling for Adaptive Games”, http://www.gamegrep.com/top_list/14266-top_15_best_video_game_storylines_ever/, “Dynamic Game Balancing: An Evaluation of User Satisfaction Gustavo Andrade, Geber Ramalho, and Alex Sandro Gomes, Universidade Federal de Pernambuco; Vincent Corruble, Université Paris 6” http://www.gamegrep.com/top_list/14266-top_15_best_video_game_storylines_ever/ (11) Map, World, Wall generation ()

2. Most Popular Games Genres Adventure games:Adventure games –Solving puzzles –Finding clues E.g., through conversations with NPCs –Classic: The Longest Journey (http://www.youtube.com/watch?v=8XrkGztFr-0)http://www.youtube.com/watch?v=8XrkGztFr-0 Role Playing Games (RPG) –Player assumes fictional roles through Avatars – Avatar acquires skills by performing tasks/actions –“Levels” indicate the progress of the avatar Classic: Diablo 2: http://www.youtube.com/watch?v=vl9C8RTu3Ochttp://www.youtube.com/watch?v=vl9C8RTu3Oc

3. Most Popular Games Genres (II) Strategy games: –resource gathering, managing economy, technology research –Real-Time Strategy (RTS) Simultaneous actions (classic: Command & Conquer)Command & Conquer –Turn-based Games (TBS) Take turns (classic: Civilization II) Sport Games –Simulation of actual sport games (classic: FIFA series)FIFA Games of chance –Outcome highly influenced by a stochastic environment First-person shooter –Controlling avatar in first-person camera view (classic: Duke Nukem)

4. Most Popular Games Genres (III) And many sub-genres –Combine and/or enhance main genres Massive Multiplayer Online RPG (MMORPG) –RPGs with multiple other players in pervasive worlds Stealth FPS –FPS emphasizing stealth over shooting RTS/RPG –Combine both aspects (Warcraft III) RTS/FPS –Combine both aspects (classic: Battlezone)Battlezone

5. Finite State Machines in Games http://www.youtube.com/watch?v=3yxoEr5qeEk http://www.youtube.com/watch?v=3yxoEr5qeEk Original slides by: Jarret Raim

6. FSM’s in Theory Simple theoretical construct –Set of states (S) –Input signals (I) –Transitional function  (s,i) A way of denoting how an NPC can change its state over time.

7. FSM’s in Practice Each state represents some desired behavior. The transition resides across all states. –Each state “knows” how to transition to other states. Accepting states are considered to be the end of execution for an FSM. –Example of an accepting state in a game? Input to the FSM continues as long it has not reached an accepting state.

8. FSM’s in Games Character AI can be modeled as a sequence of mental states. World events can force a change in state. The mental model is easy to grasp, even for non-programmers –This is important! Gather Treasure Flee Fight Monster In Sight No Monster Monster DeadCornered

9. FSM Example States –E: enemy in sight –S: hear a sound –D: dead Events –E: see an enemy –S: hear a sound –D: die Action performed –On each transition –On each update in some states (e.g. attack) Spawn D Inspect ~E D Attack E,~D ~E E E D S Patrol E ~S~S S D

10. But it gets even easier (event- on-map) If player walks here then Spawn Mephisto Starting place of player Put 3 orcs here

11. Even at a Larger Scale If player cross here Then declare war

12. Ok Let Us Construct One Finite State Machine Lets program High Priestess Mar'li –Here is she in actionHere Step 1: list states and events Step 2: Construct the Finite State Machine

13. FSM Implementation - Code Simplest method After an action, the state might change. Requires a recompile for changes No pluggable AI Not accessible to non- programmers No set structure Can be a bottleneck. void RunLogic( int *state ) { switch( *state ) { case 0: //Wander Wander(); if( SeeEnemy() ) *state = 1; if( Dead() ) *state = 2; break; case 1: //Attack Attack(); *state = 0; if( Dead() ) *state = 2; break; case 3: //Dead SlowlyRot() break; }

14. FSM Implementation - Macros Forces structure Shallow learning curve More readable –Removes clutter by using macros. Easily debugged –Allows focus on important code. bool MyStateMachine::States( StateMachineEvent event, int state ); { BeginStateMachine State(0) OnUpdate Wander(); if( SeeEnemy() ) SetState(1); if( Dead() ) SetState(2); State(1) OnUpdate Attack(); SetState(0); if( Dead() ) SetState(2); State(2) OnUpdate RotSlowly(); EndStateMachine }

15. FSM Implementation – Scripts Developer creates scripting language to control AI. Script is translated to C++ or bytecode. Requires a vocabulary for interacting with the game engine. A ‘glue layer’ must connect scripting vocabulary to game engine internals. Allows pluggable AI modules, even after the game has been released.

16. FSM Processing Interpreted –Simple and easy to debug. –Inefficient since FSM’s are always evaluated. Interpreted languages were the “rage” in AI Event Driven Model –FSM registers which events it is interested in. –Requires complex Observer model in engine. –Hard to balance granularity of event model. Multithreaded –Each FSM assigned its own thread. –Requires thread-safe communication. –Conceptually elegant. –Difficult to debug..

17. Game Engine Interfacing Simple hard coded approach –Allows arbitrary parameterization –Requires full recompile Function pointers –Pointers are stored in a singleton or global –Implementation in Dynamic- link library Allows for pluggable AI. E.g., The Tielt project Data driven –An interface must provide glue from engine to script engine. EngineAIEngineAI DLL Engine S. Interface AI Compiler Byte Code

18. Optimization – Time Management Helps manage time spent in processing FSM’s. Scheduled Processing –Assigns a priority that decides how often that particular FSM is evaluated. –Results in uneven (unpredictable) CPU usage by the AI subsystem. Can be mitigated using a load balancing algorithm. Time Bounded –Places a hard time bound on CPU usage. Project # 1 will let you experience this –More complex: interruptible FSM’s

19. Optimization – Level of Detail It’s ok to cut corners if the user won’t notice. Each level of detail will require programmer time. The selection of which level to execute can be difficult. Many decisions cannot be approximated.

20. FSM Extensions Extending States –Adding onEnter() and onExit() states can help handle state changes gracefully. Example of when to use it in games? Stack Based FSM’s –Allows an AI to switch states, then return to a previous state. –Gives the AI ‘memory’ –More realistic behavior Hierarchical FSM’s

21. FSM Example Original version doesn’t remember what the previous state was. One solution is to add another state to remember if you heard a sound before attacking. Spawn D Inspect ~E D Attack E,~D ~E E E D S Patrol E ~S~S S D E Attack-P E,S,~D ~E ~S S D

22. FSM Example Spawn D (-E,-S,-L) Wander -E,-D,-S,-L E -S Attack-E E,-D,-S,-L E Chase -E,-D,S,-L S D S D D Retreat-E E,-D,-S,L L -E Retreat-S -E,-D,S,L Wander-L -E,-D,-S,L Retreat-ES E,-D,S,L Attack-ES E,-D,S,-L E E -E -L -S L -E E L -L L D Worst case: Each extra state variable can add 2 n extra states n = number of existing states Using a stack allows to “remember” without the extra states. and remembering N actions into the pass is not possible! (finite automata vs. pushdown automata)

23. Stack FSM – Thief 3 Stack allows AI to move back and forth between states. Leads to more realistic behavior without increasing FSM complexity.

24. Hierarchical FSMs Expand a state into its own sub-FSM Some events move you around the same level in the hierarchy, some move you up a level When entering a state, have to choose a state for it’s child in the hierarchy –Set a default, and always go to that –Random choice –Depends on the nature of the behavior

25. Hierarchical FSM Example Note: This is not a complete FSM –All links between top level states still exist –Need more states for wander Start Turn Right Go-through Door Pick-up Powerup Wander Attack Chase Spawn ~E E ~S S D ~E

26. Markov Models : An instance of Non- Deterministic FSMs Selects transition based on probability distributions Have multiple transitions for the same (state,input) pair Adds variety to actions Label each with a probability that it will be taken Probabilities can be learned (Reinforcement Learning) Attack Start Approach Aim & Jump & Shoot Aim & Slide Left & Shoot Aim & Slide Right & Shoot.3.4.3.4

27. More FSM Extensions Fuzzy State Machines –Degrees of truth allow multiple FSM’s to contribute to character actions. Multiple FSM’s –High level FSM coordinates several smaller FSM’s. Polymorphic FSM’s –Allows common behavior to be shared. –Soldier -> German -> Machine Gunner

28. Polymorphic FSM Example Soldier RiflemanOfficer BritishSoviet AmericanGerman Machine Gunner BritishSoviet AmericanGerman BritishSoviet AmericanGerman

29. Debugging FSM’s Offline Debugging –Logging –Verbosity Levels Online Debugging –Graphical representation is modified based on AI state –Command line to modify AI behavior on the fly.

30. Case Study: Robocode First determine what states are needed –Attack, Evade, Search, etc. Code up the FSM transition function. –Include an error state that is noticeable. Setup debugging. –Verbosity levels are a must.

31. Case Study: Robocode Attack Defend Search Implement and test each state separately. A test bot AI might help test single behaviors. (see Target bot)

32. Defense and Firing Power Enable your bot to dodge incoming fire. Every 20 ‘ticks’ reverse direction. Adds a circle strafe. Selects a bullet power based on our distance away from the target void doMovement() { if (getTime()%20 == 0) { direction *= -1; setAhead(direction*300); } setTurnRightRadians( target.bearing + (PI/2)); } void doFirePower() { firePower = 400/target.distance; }

33. Searching Reducing the scanner arc allows you to fire faster. If a target hasn’t been seen recently, spin. Scan where the target is. ‘Wobble’ to make sure to find the target. void doScanner() { double radarOffset; if(getTime() - target.ctime > 4) radarOffset = 360; else radarOffset = getRadarHeadingRadians() - absbearing(getX(),getY(),targ et.x,target.y); if( radarOffset < 0 ) radarOffset -= PI/8; else radarOffset += PI/8; setTurnRadarLeftRadians(Normalis eBearing(radarOffset)); }

34. References AI Game Programming Wisdom University of Wisconsin presentation Robocode Website