1. Using a Goal/Action Architecture to Integrate Modularity and Long-Term Memory into AI Behaviors
Marc Atkin John Abercrombie

2. Why Do We Want Better AI? Bad AI is always noticed
AI often inconsistent between games – why can’t we build on previous work?
The AI is a big part of what makes a game world immersive and fun

3. What’s Stopping Us? What makes AI complex?
AI has many possible reactions to many situations
Many behaviors are operating on the AI at any given time
Hard to account for all the possible interactions

4. The Basic Idea: Use Modularity
It’s not just for coding anymore!
Produces AI that is easier to understand and debug
Produces AI that it reusable within and between projects
The key point: Design the AI architecture so that it facilitates modularity

5. Part 1: Tyrion

6. Tyrion An architecture for specifying and executing AI behavior
Implements a language in which to write behavior
Encourages hierarchical, modular design

7. Resources, Goals, and Actions
Goal: A description of a desired world state
Action (or behavior): A method for achieving a goal
Resource: An entity required to perform an action

8. Resource Hierarchy Team Squads Vehicles Driver/Gunners Characters
Legs/Arms/Head

9. Resource hierarchy Team Strategic level Squads Tactical level Vehicles
Driver/Gunners
Individual Units
Characters
Legs/Arms/Head
Motor Control

10. Action/Goal Hierarchy
Attack Goal
Squad
Attack From All Sides
Attack Goal
Character
Attack Goal
NormalAttack
FlankingAttack
Legs /
Arms
MoveTo Goal
FireAt Goal
MoveTo
Fire Weapon

11. Action Structure Goal satisfied by action Selection heuristic
Evaluates appropriateness of action in a given situation
Message callback functions for child actions
Message callback functions for sensors
Body (Unreal script latent code)
Typically executes over a number of game ticks

12. Action Idioms Simplify writing action bodies
Encapsulate commonly performed tasks
Examples:
WaitForAllGoals
WaitForAnyGoal

13. Execution Model 10 times a second: Iterate over every resource:
Order unmatched goals by priority
Find actions that achieve them
Tick every running action
Tick every active periodic sensor; send messages
If an action completes, mark its goal as achieved/failed; send messages

14. Part 2: Building Modular Actions

15. SWAT4 What is SWAT4? First person tactical shooter
Player is team leader
Single player – player commands four officer AI’s
Object is to kill or subdue all enemies, and rescue the hostages

16. What Makes Up the SWAT4 AI?
Hierarchical resources for organizing behaviors
Squad
Tactical
Character
Individual
Motor Control
Movement
Weapon
Layer behaviors to form more complex behaviors

17. Modularity in SWAT4 Start from the ground up – Motor Control
The Legs - Movement Resource
Basic Movement – move to a position, move to a particular object, follow another AI
Lower body rotation
The Arms – Weapon Resource
Aiming – aim at a position, to a rotation, at something
Item usage – (i.e. guns, grenades, toolkit, explosives)

18. Modularity in SWAT4 The Character The Squad Controls arms and legs
Plays special full body animations
The Squad
Controls individual characters, tells them what to do so they can work together

19. Reuse
Tyrion framework encourages tasks to be broken apart into sub-tasks
Higher level behaviors use lower level behaviors to accomplish complicated tasks
Lower level behaviors can be used over and over again by different higher level behaviors
i.e. Movement, aiming, weapon usage

20. Layering
Each behavior on a resource is responsible for controlling everything below it
i.e. Officers move in a formation while aiming at points of interest in the world (Fall in Command)
Using multiple characters in a squad allows group to work together to accomplish more

21. Organizing Behaviors Use of action idioms
waitForGoal, waitForAllGoals, waitForAnyGoals, waitForResourcesAvailable
Parent action posts child goals to accomplish tasks
on the same resource or one lower in the hierarchy

22. Organizing Behaviors
In SWAT4 the Squad posts child goals on characters (the officers)
Character behaviors usually post child goals on movement and/or weapon resources
Each action can wait for child goals to complete using action idioms

23. Example: Move Squad to a Point
Squad behavior (MoveSquadTo):
Posts MoveTo Goal on each officer
WaitForAllGoals(MoveToGoals)
Character behavior (MoveTo):
Posts Aim Around Goal on weapon resource
Posts MoveToLocation Goal on movement resource
WaitForGoal(MoveToLocation)

24. Stack Up (on a door) User points at door, gives command
Squad behavior determines points for Officers to stack up at
Each officer locomotes to point given - while looking around or aiming at the door if they are able 1 1 2 2

25. How Stack Up Works
Squad Behavior (Squad Stack Up)
Post Stack Up goal on each officer
Pass Stack Up Point as destination parameter
WaitForAllGoals(StackUpGoals)
Character Behavior (Stack Up)
Post Aim At Door goal
Post MoveTo(StackUpPoint) goal
WaitForGoal(MoveToGoal)
Result: Movement & Weapon behaviors act independently of each other

26. Move & Clear Inherits from the Stack Up behavior
Re-use functionality of getting AI’s stacked up on the door
Adds new behaviors:
After finishing stacking up (but not trying the door):
Officer near the door opens it
Another Officer told to start moving as soon as the door is opened
Team then moves through the door into next room to points specified by a designer

27. Breach, Bang & Clear Inherits from the Move & Clear behavior
Open Door behavior replaced: first officer blows door open using C2 explosives or a breaching shotgun
Second officer throws flashbang grenade into room before entering
Squad organizes characters
Some basic character behaviors are reused across many commands
i.e. deploy grenade, deploy shotgun, or deploy C2

28. Squad StackUp Squad Move & Clear Squad Breach Bang & Clear
Squad Behaviors
Stack Up
Squad Move
& Clear
Move &
Clear
Squad Breach
Bang & Clear
Aim At
Character Behaviors
Open
Door
Move To
Throw
Grenade
Aim Around
Movement & Weapon Behaviors
Move To
Deploy C2
Use
Grenade
Deploy
Shotgun
Move To
Move To
Inherits From
Child Behavior
Move To

29. Summary and Results Framework is key to organizing use of system
Time to prototype and develop AI behaviors reduced
Interesting SWAT4 Stats:
Number of Squad Behaviors: 33
Number of Character Behaviors: 65
Number of Weapon Behaviors: 10
Number of Movement Behaviors: 11
Number of Months Programming: 16 (May 2003 – September 2004)

30. Part 3: Long-term Memory

31. The Problem of Forgetful AI
“Oh hello – have we met before?”
State machines don’t remember past events
What’s a natural way to represent task-related memory?

32. Two Types of Memory Task memory: Cognitive memory:
What needs to be done
The context of the current action
Represented by the set of goals
Cognitive memory:
Sensor data that has accumulated during the AI’s life time
Represented inside sensors and actions

33. Tribes: Vengeance First-person shooter
Emphasizes speed and freedom of movement (jetpacks!)
3rd in the series

34. A Day in the Life of a Grunt
A Grunt AI with three goals:
PatrolGoal (priority 40)
AttackGoal (priority 50, dormant)
DodgeGoal (priority 90, dormant)
AttackGoal will activate when an enemy is sighted
DodgeGoal will activate when a visible projectile will hit the AI

35. Patrol
Goal
Attack
Goal
Dodge
Goal
Patrol is executing 40
legs
arms

36. X Patrol is executing 40 Enemy spotted! Sensor wakes up AttackGoal 40
Dodge
Goal
Patrol is executing 40
legs
Enemy spotted!
Sensor wakes up AttackGoal
arms X 40 50
Attack executes; posts subgoals
for legs & arms
legs 50
arms

37. X X X Patrol is executing 40 Enemy spotted! Sensor wakes up AttackGoal
Dodge
Goal
Patrol is executing 40
legs
Enemy spotted!
Sensor wakes up AttackGoal
arms X 40 50
Attack executes; posts subgoals
for legs & arms
legs 50
arms
Projectile spotted!
Sensor wakes up DodgeGoal X X 40 50 90
Dodge executes; needs legs;
arms continue executing Attack!
legs 50
arms

38. X X X X Patrol is executing 40 Enemy spotted!
Goal
Attack
Goal
Dodge
Goal
Patrol is executing 40
legs
Enemy spotted!
Sensor wakes up AttackGoal
arms X 40 50
Attack executes; posts subgoals
for legs & arms
legs 50
arms
Projectile spotted!
Sensor wakes up DodgeGoal X X 40 50 90
Dodge executes; needs legs;
arms continue executing Attack!
legs 50
arms X
Dodge finishes: legs resume
Attack subgoal 40 50
legs 50
arms

39. Seeing the Big Picture
Task memory: An AI’s goal set constitutes the “big picture” of tasks to be accomplished
Failed or interrupted goals hang around
Cognitive memory: actions potentially contain lots of stored data
High level actions have long life span; their data sticks around for that time, too

40. Seeing the Big Picture
Explicit goal representation: AI can reason about remaining tasks
Behaviors can be chosen that achieve multiple goals
Separate rules decide when to remove unachieved goals

41. Part 4: Collaboration

42. Why Collaborate? Increases productivity
Both working on FPS’s starting at about the same time using the same engine (Unreal)
Development teams based in Boston (SWAT4) and Canberra, Australia (Tribes)
Teams shared code for rendering, networking, and more
Tyrion was an already tested system

43. How to Work Together? Communication Shared Codebase Perforce
Programming (of course!)

44. Communication
Initial technical design document was written in Australia, then discussed via between offices for weeks
response time was usually a day (due to time difference)
More complicated work could be accomplished over the phone
An in-person visit helped SWAT4 make greater use of Tyrion’s features

45. Communication: Lessons Learned
We underestimated the amount of time required for effective communication
During design phase, both of us spent about 1/3rd of our time communicating – less later on
We underestimated the amount of time required for “tech support”
Collaboration allowed both teams to identify weaknesses in Tyrion that we would discuss and address

46. Shared Codebase
Shared code between offices to speed development of both Tribes & SWAT4
One shared codebase, which both projects could integrate code from or to

47. Perforce
Very useful program, allowed for easy collaboration between offices
We didn't have to worry about sharing the code, we got to worry about the code itself!

48. Programming Marc did most of the implementation
John's prior experience in Unreal helped optimize and streamline the code
We both felt comfortable editing the code in the shared codebase for our own needs
Code today is clean and well commented, although it does contain a few #if IG_SWAT and #if IG_TRIBES3 preprocessor definitions!

49. Programming: Lessons Learned
Would have been nice to have shared a common AI base class
Both games exposed weaknesses that were addressed
Early in SWAT, the idle goal needed to be satisfied by two subclasses of the base Idle Action class
Change was made to system to allow subclasses of actions to satisfy goals
Claiming leaf (motor control) resources issue

50. Programming: Lessons Learned
Inheritance vs Building Blocks
Building blocks: new behaviors are implemented by combining simpler ones
Inheritance: Change an existing behavior by sub-classing  Tyrion didn’t intend for this
Not everyone will use your engine the way you intended
Having multiple ways of accomplishing tasks allows clients to create and innovate
Engine’s strength determined by client’s effective use

51. Collaboration Results
The shared development of using Tyrion worked well
We feel both games had challenging and fun AI
Its use has made prototyping AI behaviors easy and quick!
Marc got to wake up at 7 a.m. in the morning for a fire alarm while staying at John's house

52. The Future We move on to new projects, and will continue to use Tyrion
Still finding improvements and new ways of using Tyrion
Engine Improvements
Better goal/sensor integration
Make AI more “designer-friendly”
Shared AI subclass

53. Questions?