3 States and a Plan: The AI of F.E.A.R. Jeff Orkin Monolith Productions/ MIT Media Lab

Survey Says!

FSM: 3 States

FSM: 2 States!

FSM: Transition Logic void StateAttack::Update() { //... if( Ammo == 0 ) { pState = Reload(bCrouch); return; } //... }

Shogo, 1998

No One Lives Forever, 2000

F.E.A.R., 2005

Halo 2, 2004 Allowable behaviors for infantry, drivers, and passengers

FSM vs Planning FSMPlanning - How- What

FSM vs Planning FSMPlanning - How- What - Procedural- Declarative

Motivation

Here’s the Plan: STRIPS Planning Overview Planning in F.E.A.R. Differences from STRIPS Squad Behaviors & Communication Beyond F.E.A.R.

What is Planning? Planning is a formalized process of searching for sequence of actions to satisfy a goal. Process is called “Plan Formulation.”

STRIPS Planning …in a nutshell

STRIPS Planning STRIPS = STanford Research Institute Problem Solver

STRIPS Planning STRIPS Goal: Desired state of the world to reach. STRIPS Actions: –Preconditions –Effects

States

States: FSM Attack Search

States: Planning Represented as a logical sentence: AtLocation(Home) ^ Wearing(Tie) Represented as a vector: (AtLocation, Wearing) = (Home, Tie)

States: Planning Example: Lemonade Stand ( weather, #lemons, $$ )= (,,) or (,,)

STRIPS Planning Example

State: (phone#, recipe, hungry?) Goal: ( --, --, NO ) (, --,YES) (, --,NO ) ( --,,YES) ( --,,NO )

STRIPS Planning Example State: (phone#, recipe, hungry?) Goal: ( --, --, NO ) (,,NO ) (,,YES) (,,NO )

STRIPS Planning Example

State: (phone#, recipe, hungry?) Action Preconditions: (, --, -- ) Effects: Delete List: Hungry( YES ) Add List: Hungry( NO )

STRIPS Planning Example State: (phone#, recipe, hungry?) Action Preconditions: (, --, -- ) Effects: Hungry( NO ) Hungry( YES ) ^ Hungry( NO )

STRIPS Planning Example Action: Buy( object ) Preconditions: ( -- ) Effects: Delete List: -- Add List: Own( object ) Own( ) ^ Own( ) ^ Own( )

STRIPS Planning Example

FPS Statistics Average lifespan of FPS enemy AI:

FPS Statistics Average lifespan of FPS enemy AI: seconds

FPS Statistics Average lifespan of FPS enemy AI: seconds Number of enemy AI who die in FPS’s per day:

FPS Statistics Average lifespan of FPS enemy AI: seconds Number of enemy AI who die in FPS’s per day: 19,789,203

FPS Statistics Sally Struthers says “Save the AI!”

Here’s the Plan: STRIPS Planning Overview Planning in F.E.A.R. Differences from STRIPS Squad Behaviors & Communication Beyond F.E.A.R.

Design Philosophy Designer’s job is: Create environments that allow AI to showcase their behaviors. Designer’s job is NOT: Script behavior of individual AI.

Planning Video #1

Planning Video #2

Planning Video #3

Planning Video #4

SoldierAssassin Rat

Benefits of Planning 1.Decoupling Goals & Actions 2.Layering Behaviors 3.Dynamic Problem Solving

Decoupling Goals & Actions

+ =

Benefits of Planning 1.Decoupling Goals & Actions 2.Layering Behaviors 3.Dynamic Problem Solving

Layering Behaviors

Kill Enemy Attack Ranged

Layering Behaviors Kill Enemy Attack Ranged Attack Melee

Layering Behaviors Kill Enemy Attack Ranged Attack Melee Dodge Shuffle Dodge Roll Dodge

Layering Behaviors Kill Enemy Attack Ranged Attack Melee Dodge Shuffle Dodge Roll Dodge Cover Goto Dodge Covered

Layering Behaviors Kill Enemy Attack Ranged Attack Melee Dodge Shuffle Dodge Roll Dodge Cover Goto Dodge Covered Blind Fire

Layering Behaviors Kill Enemy Attack Ranged Attack Melee Dodge Shuffle Dodge Roll Dodge Cover Goto Dodge Covered Blind Fire Ambush

Layering Behaviors Kill Enemy Attack Ranged Attack Melee Dodge Shuffle Dodge Roll Dodge Cover Goto Dodge Covered Blind Fire Ambush

Layering Behaviors

Benefits of Planning 1.Decoupling Goals & Actions 2.Layering Behaviors 3.Dynamic Problem Solving

Dynamic Problem Solving

Differences From STRIPS 1.Cost Per Action 2.No Add/Delete Lists 3.Procedural Preconditions 4.Procedural Effects

Cost Per Action

2 8

2 8 * A

A

A*NavigationPlanning Nodes:NavMesh PolysWorld States Edges:NavMesh Poly Actions Edges Goal:NavMesh PolyWorld State

Cost Per Action A

Differences From STRIPS 1.Cost Per Action 2.No Add/Delete Lists 3.Procedural Preconditions 4.Procedural Effects

No Add/Delete Lists State: (phone#, recipe, hungry?) Action Preconditions: (, --, -- ) Effects: Delete List: Hungry( YES ) Add List: Hungry( NO )

No Add/Delete Lists State: [phone#, recipe, hungry?] Action Preconditions: [, --, -- ] Effects: [ --, --, NO ]

No Add/Delete Lists Attack Preconditions: [ --, --, ] Reload Effects: [ --, --, ]

No Add/Delete Lists [ --, --, -- ] 4-byte values TargetDead[bool] WeaponLoaded[bool] OnVehicleType[enum] AtNode[HANDLE] - or - AtNode[variable*]

No Add/Delete Lists Goal Cover: [ --, CoverNode63, -- ] Action Goto Effect: [ --, variable*, -- ]

No Add/Delete Lists [ --, --, ]

No Add/Delete Lists [ --, --, ]

Differences From STRIPS 1.Cost Per Action 2.No Add/Delete Lists 3.Procedural Preconditions 4.Procedural Effects

Procedural Preconditions class Action { // [ --, --, -- ] WORLD_STATE m_Preconditions; WORLD_STATE m_Effects; bool CheckPreconditions(); };

Procedural Preconditions

Differences From STRIPS 1.Cost Per Action 2.No Add/Delete Lists 3.Procedural Preconditions 4.Procedural Effects

Procedural Effects class Action { // [ --, --, -- ] WORLD_STATE m_Preconditions; WORLD_STATE m_Effects; bool CheckPreconditions(); void ActivateAction(); };

Procedural Effects void ActionFlee::ActivateAction() { //... }

Squad Behaviors

Global Coordinator

Squad 1 Squad 2 Squad 3

Squad Behaviors 1.Simple 2.Complex

Squad Behaviors 1.Simple –Suppression fire –Goto positions –Follow leader 2.Complex –Flanking –Coordinated strikes –Retreats –Reinforcements

Simple Squad Behaviors Get-to-Cover Advance-Cover Orderly-Advance Search

Simple Squad Behaviors 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail.

Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

Squad Behaviors 1.Simple –Suppression fire –Goto positions –Follow leader 2.Complex –Flanking –Coordinated strikes –Retreats –Reinforcements

Complex Squad Behaviors There are none!!

Complex Squad Behaviors Player

Complex Squad Behaviors Player

Complex Squad Behaviors Player

Squad Behavior Implementation Key is separation between squad layer and individual behaviors. Possibly formalize with Hierarchical Task Network (HTN) planning.

Squad Communication “I need reinforcements!”

Squad Communication Dialogue, instead of announcements: –Announcement: “Ouch!” –Dialogue: AI1: “What’s your status?” AI2: “I’m hit!” (or “I’m alright!”)

Squad Communication Dialogue, instead of announcements: –Announcement: “Where did he go?” –Dialogue: AI1: “Can you see anything?” AI2: “No sir!” (or “He’s behind the boxes!”)

Squad Communication Dialogue also explains lack of action: –Dialogue: AI1: “He’s aiming at you! Get out of there!” AI2: “I’ve got nowhere to go!”

Squad Communication “…Not only do they give each other orders, but they actually DO what they’re told!”

Squad Communication “…Not only do they give each other orders, but they actually DO what they’re told!” Reality: Squad behavior has bird’s eye view of the situation. Find appropriate dialogue sequence after the fact.

Here’s the Plan: STRIPS Planning Overview Planning in F.E.A.R. Differences from STRIPS Squad Behaviors & Communication Beyond F.E.A.R.

“… just like the marines in Half-Life 1”

MIT Media Lab: Cognitive Machines Group

Dialogue Integration “What’s your status?”

Unified Planning for Actions and Speech “Look out! Grenade!” Precondition: Effect:

Unified Planning for Actions and Speech OpenDoor -or- “Open the door!” Precondition: door is closed Effect: door is open

Plan Recognition UnlockDoor CreateKey LightFireGotoForge GetGold

Plan Recognition Plan as Context Free Grammar (CFG): UnlockDoorCreateKey LightFireGotoForge GetGold CreateKey GotoForge : : :

Plan Recognition PLAN RECOGNITION VIDEO

Questions? Jeff Orkin Monolith is hiring! Engineers Artists Designers