Managing Complexity in the Halo2 AI Damián Isla Bungie Studios

Managing Scalability in the Halo2 AI Damián Isla Bungie Studios

Scalability? No! Scalability!

Why Scalable AI? More is BETTER The brute force approach to common sense: the more unique situations you can recognize and react to, the more common-sensical you are. (e.g. Cyc, OpenMind projects)

3 Dimensions of Scalability heretic grunt elite major flood swarms hunters hell-jumpers brute captain Miranda sniper marine DRAMA! pacing story challenge Variety Variation drive the warthog Shoot the needler melee attack fight perch hide boarding Strafe target Volume

The Ultimate Goal Designer I want X to do Y whenever Z! Engineer Okay. (To himself) Hmm … and I know exactly where to put that … I’m happy now. (Trundles back to design-bubble) Two points: A great architecture has one PERFECT place for each new feature X,Y,Z – as in almost all of the designer’s requests – describe something observable by the player. It is not smart, it is in fact a predictable rule that will always happen. The only question is whether it will play nice with the other 500 XYZs the designer has asked for.

AI Design Requirements Transparency Facilitate the ongoing narrative in the player’s head Coherence Focused action, right priorities Directability For the designer Workability For the engineer See [Butcher & Griesemer] GDC 2002 Transparency – often about clear animation, clear lines of dialogue, etc. NEXT SLIDE: TITLE SCALABLE DECISION MAKING

Managing Scalable Decision-Making in the Halo2 AI Damián Isla Bungie Studios

Behaviors Behavior: a program that takes temporary control of the actor in order to get something done Behaviors are action over time: Move to a point Orient Shoot Crouch Play dialogue Trigger “actions” Behaviors have notion of relevancy and duration e.g., Fight Vehicle entry Throw grenade Find cover Cover friend Check body Wake up a sleeping grunt Etc. Halo2: ~130 in total

The Combat Cycle Massive amounts of complexity hiding in here!

Decision-Making Behavior Tree (or DAG) Good Bad Intuitive Modular Scalable Bad Never really that simple Debugging is hard Memory is hard

Decision routines Parent custom Child-competitive

Given A, B or C, always choose D (all of the above) Decision routines Child-competitive strategies Analog Relevancy Binary Relevancy Prioritized-list (root, engage) Sequential Sequential-looping (search) Randomized One-off randomized (postcombat) DP #1: customizability Priorities list is most common COHERENCE IS ABOUT PRIORITIES Design Principle #1 : Given A, B or C, always choose D (all of the above)

Unless the player is in vehicle, in which case... Impulses Problem: What happens (with a prioritized list) when the priority is not constant? Unless the player is in vehicle, in which case...

Design Principle #2 : Formalize complexity (don’t hide it) Impulses Solution: Separate alternative trigger conditions out into separate impulse Two execution options In-place Redirect Design Principle #2 : Formalize complexity (don’t hide it)

Design Principle #3: Embrace the hackery Impulses The other purpose of impulses: HACKS Or, “localized code with ad-hoc Functionality” e.g. crouch_on_danger impulse dive impulse Design Principle #3: Embrace the hackery Two things demanded of the architecture: Allow us to do it. Help us contain/catalog/keep track of it

Behavior Metadata Problem: In determining relevancy, we check the same conditions over and over and over and over Actor’s vehicle status (infantry, driver, passenger) Actor’s alert status (idle, in combat, visible target) Solution: Use metadata to describe execution conditions Force all behaviors to declare their execution conditions Halo2: “conditions” bitvector compared to “actual state” bitvector Acts as a behavior/impulse mask (modifying the basic structure of the tree itself)

Design Principle #4 : Variation from a stable base Custom Behaviors Problem: character-type specific behaviors Don’t want all characters evaluating behaviors that only one type can do E.g. grunt’s “retreat from scary enemy” impulse Solution: attach custom behaviors to tree ALTERNATIVE: New TREE

Stimulus Behaviors Problem: rare event-driven behaviors tested for every tick e.g., Grunts flee when their leader dies Solution: Stimulus Behaviors Behaviors or impulses dynamically and asynchronously placed into tree at specified location “Actor Died” 1 sec. Why not just force the actor to start running the behavior??? Maybe a higher-priority XYZ is running The tree ensures that the priorities of the designers are respected. TREE PLACEMENT as much a part of the decision as the RELEVANCY function

Joint Behaviors Remember Design Principle #1! (always choose D, all of the above)

Behavior Summary Behavior DAG Binary decision routines Impulses Metadata Custom Behaviors Stimulus Behaviors Joint behaviors The behavior tree is fundamentally a DYNAMIC structure!

Memory is a problem fundamental to the behavior tree structure. No system is complete with an explicit memory solution.

Memory and Memory Problem: Persistent behavior state is impossible (too much data to keep around!) Solution: Only keep around state for behaviors that are running Problem: What about state we need to keep around regardless of whether we’re running the behavior or not? (e.g. last grenade-throw time) Part of maintaining coherence is remembering what I’ve been doing!!!

The Anatomy of Memory …memory is a complicated thing. Store memory as… Behavior state (short-term) Persistent per-behavior Persistent per-target Props are also Perception caches Our knowledge model “Props”

(formalize complexity, The Anatomy of Memory e.g. Search Aquire target 1 Lose sight of target 1 Search target 1 Last known location inspected New search point selected Aquire target 2 Search deactivated, short-term state discarded Target 2 killed Switch to target 1 Where do we search? Existing search point used EXPERIENCE is VERY coherent No tree architecture is complete without a memory solution Remember Design Principle #2 (formalize complexity, don’t hide it)

Designer Direction Variety Variation Character definition Model, animation graph Behavior parameters Static control Level-scripting Dynamic Control Variation

Static Control: Variety Character types Elites Grunts Marines Variants Gold elite Red elite Honor guard elite Jetpack elite Each variant gets a .character definition file A LOT of parameters to author and maintain! 64(variants) x 80 (behaviors/variant) x 3 (parameters/behavior) = 15,360

Static Control: Parameters Character files divided into parameter blocks Vitality properties Perception properties Search properties Weapon usage Etc. Blocks can be instantiated or not Hierarchy: children characters instantiate only significant variations from parent Root of tree is the “generic” character, which provides “reasonable” values for all parameters Remember Design Principle #4! (Variation from a stable base)

Static Control: Styles Styles are cool: Behavior mask Bias control parameters Superposable (Another reason why Impulses are great) VARIETY Another reason why separating different trigger conditions into different impulses is a great idea: expose those triggers to the designers Back to vehicle entry behavior

Dynamic Control: Organization Halo1: “Encounters” Groups of actors Groups of areas Problem: moving actors from one encounter to the next was “architecturally discouraged” Halo2: “Squads” Groups of actors “Zones” Groups of areas “Orders” A mapping between the two

Dynamic Control: Orders The Metaphor: “Take that hill!” “Hole up in that bunker!” Or “Occupy this space and behave this way!” Position direction Behavior direction

Blah blah blah Actor is a member of a squad Squad gets an order Actors get free range inside of region

Orders and Behavior Behavior direction Vehicle behaviors enabled/disabled Active camouflage enabled/disabled Follow player enabled/diabled Rules of engagement Style Behavior mask Bias control parameter A way that orders can directly constrain the decision space of the actors Tricky to author … but the nice thing is that you only need a few of them, like (assaulting, bunkering, harassing…) Only about 5 or 6 used in Halo2 NO SCRIPTING “Alright, men, engage active camouflage and move forward cautiously. And don’t open fire until the Master Chief does!”

Design Principle #5 : Work with your representations In summary Design Principle #5 : Work with your representations

Recap Design Principles: Always choose D (all of the above) Formalize complexity (don’t hide it) Embrace the hackery Variation from stable base Work with your representations

Technical Takeaways Dynamic behavior tree Binary relevancy Behavior masking Principled approach to memory Take smarts out of the behavior and put it in the knowledge model

Conclusions All systems need to “play ball” with the core decision mechanism Not looking for “smart” architecture, looking for expressive architecture

The Ultimate Goal Designer I want X to do Y whenever Z!

Questions? Damián Isla damiani@microsoft.com