2. 8.3. A GENT AND D ECISION M AKING AI Agent driven AI and associated decision making techniques

3. In lecture exploration of answers to frequently asked student questions

4. Using an agent driven approach to control game character AI

5. “An autonomous agent is a system situated within and a part of an environment that senses that environment and acts on it, over time, in pursuit of its own agenda and so as to effect what it senses in the future.” Agents may act as an Opponent Ally Neutral character Loops through the following cycle: Sense ► Think ► Act Optional learning or remembering step

6. Within a game, agents can have access to perfect information about the game world (e.g. complete terrain layout, location and state of the player, etc.). Often a sensing model is used to avoid agent ‘cheating’ and ensure agents cannot ‘see’ through walls, know about unexplored areas, etc. Example sensing model: For each game object: Is it within the viewing distance of the agent? Is it within the viewing angle of the agent? Is it unobscured by the environment?

7. Do agents respond to sound? If so, how is sound propagation modelled? An event based model is typically used: When sound is emitted, it alerts interested agents Use distance and zones to determine how far sound can travel Travel distance may also depend upon type of incident surface and movement speed of the player, etc.

8. Agents should not normally see, hear and communicate instantaneously (or rather immediately commence actions following the sensing stage) Normally sufficient to introduce artificial reaction times, e.g.: Vision: ¼ to ½ second Hearing: ¼ to ½ second Communication: > 2 seconds

9. Approaches towards agent AI ‘decision making’ (not mutually exclusive) include: Use pre-coded expert knowledge Algorithmically search for a solution Many different techniques exist (we will explore some later) Aside: Encoding expert knowledge is appealing as it is relatively easy to obtain and use, but may not be scalable or adaptable. Whilst often scalable and adaptable, algorithmic approaches may not match a human expert in the quality of decision making or be computationally expensive.

10. ‘Dumbing’ down agents Sometimes it may be necessary to ‘dumb down’ agents, for example: Make shooting less accurate Introduce longer reaction times Change locations to make self more vulnerable, etc. Letting agents cheat This is sometimes necessary for: Highest difficultly levels CPU computation reasons Development time reasons

11. Sensing and thinking steps are invisible to player, i.e. acting is how player witnesses intelligence. Example of actions include: Move location Pick up object Play animation Play sound effect Fire weapon Agents might also use event- driven communication when within the vicinity of each other to: Alter other agents to some situation (i.e. agent hurt) Share agent knowledge (i.e. player last seen at location x)

12. Introduction to decision making techniques within game AI

13. The input to a decision making process is the knowledge possesses by a game object and the output is a requested action. The input knowledge can consist of internal knowledge (i.e. internal state) and external knowledge (i.e. game world). Likewise, actions can be directed towards changing internal state or the external (world) state. Aside: Most games need only use simple decision making techniques such as decision trees and state machines. Rule-based approaches may be needed for more complex needs. Internal knowledge External knowledge Decision Making Process Requested action(s) Internal change (s) External change (s)

14. Simple decision making using decision trees

15. Decision trees offer a simple, but fast form of decision making. A decision tree consists of a starting decision point, which is connected to more refined decision points. Each leaf contains an action that is executed once reached. The tree can be grown to encapsulate complex behaviour but then often become hard to manage.

16. The use of finite state machines to encapsulate a decision process

17. A finite state machine occupies one of a finite number of states at any point in time. Actions may be undertaken based on the current state. Inputs to the system can cause a transition from one state to another. T1T1 T2T2 T3T3 T4T4 A S1 A S2 A S3 In general: Each FSM has a number of possible states {S1... S N } Transition functions {T1... T M } define the conditions under which a state transition will occur. Every time a state transition occurs and a new state is entered, one or more state actions may be fired {A S1... A SN }

18. A finite state machine works by decomposing an object’s behaviour into defined chunks (states). So long as a character remains in a state it will use the same behaviour. State machines are very widely used, including: Controlling ghosts in Pac-man Controlling bots in Quake Sports simulations such as FIFA 2002 NPCs in RTSs such as Warcraft

19. Each ghost can be in a wander, chase or evade state (each ghost can have a different chase/wander behaviour). Once a powerpill is eaten, all ghosts transition to the evade state, which is exited once the timer expires. Pacman in range Pacman out of range Powerpill eaten Powerpill expired Aside: The wander state could be entirely removed in this FSM, i.e. Chase and Evade form the minimum behavioural set.

20. State machines along with scripting represent the most common forms of decision making in games, as: They are relatively quick and simple to code and debug They have little computational overhead (depending on the complexity of transition tests). They are flexible and can often be easily extended or modified. State based behaviour is good for modelling many game-world objects.

21. Simple FSMs cannot easily model all forms of behaviour. One example is ‘alarm behaviour’, an action that can be triggered from any state. Consider a robot whose ‘alarm’ behaviour is to recharge when power levels become low. Using a hierarchical FSM, the states can transition between ‘cleaning up’ and ‘getting power’ (at the top level). When in the ‘cleaning up’ state a lower-level FSMs controls behaviour.

22. Using goals to drive behaviour

23. GOB is used widely in games such as The Sims. Characters have a range of ‘emotional’ or ‘physical’ goals (or motives). Depending on the actions executed by the character (possibly player controlled) the goals (i.e. needs, desires, fears, etc.) will either increase or decrease. GOB algorithms try to fulfil the character’s goals by selecting between available actions that influence the goal parameters.

24. Characters can have a number of currently active goals. Goals might include: eat, seek health, defeat opponent. Each goal has an associated numeric insistence value representing the current importance of that goal. Some goals may be fully achievable (e.g. seek health) others may be only reducible by always remain (e.g. satiate hunger). A set of (possibly situational) actions is presented to the character. The character will select the action that best satisfies their current goal insistence values.

25. Consider the shown goals and actions. Which action should be selected? The notion of overall discontentment offers a useful means of selecting the best action. A good discontentment metric is to sum the squares of insistence values and select the action that results in lowest discontentment. More advanced approaches can consider the time to start/complete each activity, or more complex insistence contributions. Goals (Insistence low = 0, high = 5): Eat (4), Sleep(1), Bathroom (3) Actions: Eat-Food (Eat − 3, Bathroom +1) Eat-Snack (Eat − 2) Sleep-Bed (Sleep − 4) Sleep-Sofa (Sleep − 2) Drink-Cola (Eat − 1; Bathroom + 3) Visit-Bathroom (Bathroom − 4) Discontentment : Eat-Food: (4-3) 2 + 1 2 + (3+1) 2 = 18 Eat-Snack: (4-2) 2 + 1 2 + 3 2 = 14 Sleep Bed: 4 2 + (1-1) 2 + 3 2 = 25 Sleep Sofa: 4 2 + (1-1) 2 + 3 2 = 25 Drink-Cola: (4-1) 2 + 1 2 + (3+2) 2 = 35 Visit-Bathroom: 4 2 + 1 2 + (3-3) 2 = 17

26. To do: If applicable to your game, explore finite-state machines, agents and goal driven behaviour. Word towards your alpha hand-in goals. Today we explored: The notion of a game AI agent Decision making processes including finite state machines and goal driven behaviour