1. 3 States and a Plan: The AI of F.E.A.R. Jeff Orkin Monolith Productions/ MIT Media Lab

4. Survey Says!

7. FSM: 3 States

10. FSM: 2 States!

11. FSM: Transition Logic void StateAttack::Update() { //... if( Ammo == 0 ) { pState = Reload(bCrouch); return; } //... }

12. Shogo, 1998

13. No One Lives Forever, 2000

14. F.E.A.R., 2005

15. Halo 2, 2004 Allowable behaviors for infantry, drivers, and passengers

16. FSM vs Planning FSMPlanning - How- What

17. FSM vs Planning FSMPlanning - How- What - Procedural- Declarative

18. Motivation

19. Here’s the Plan: STRIPS Planning Overview Planning in F.E.A.R. Differences from STRIPS Squad Behaviors & Communication Beyond F.E.A.R.

20. What is Planning? Planning is a formalized process of searching for sequence of actions to satisfy a goal. Process is called “Plan Formulation.”

21. STRIPS Planning …in a nutshell

22. STRIPS Planning STRIPS = STanford Research Institute Problem Solver

23. STRIPS Planning STRIPS Goal: Desired state of the world to reach. STRIPS Actions: –Preconditions –Effects

24. States

25. States: FSM Attack Search

26. States: Planning Represented as a logical sentence: AtLocation(Home) ^ Wearing(Tie) Represented as a vector: (AtLocation, Wearing) = (Home, Tie)

27. States: Planning Example: Lemonade Stand ( weather, #lemons, $$ )= (,,) or (,,)

28. STRIPS Planning Example

33. State: (phone#, recipe, hungry?) Goal: ( --, --, NO ) (, --,YES) (, --,NO ) ( --,,YES) ( --,,NO )

34. STRIPS Planning Example State: (phone#, recipe, hungry?) Goal: ( --, --, NO ) (,,NO ) (,,YES) (,,NO )

35. STRIPS Planning Example

36. State: (phone#, recipe, hungry?) Action Preconditions: (, --, -- ) Effects: Delete List: Hungry( YES ) Add List: Hungry( NO )

37. STRIPS Planning Example State: (phone#, recipe, hungry?) Action Preconditions: (, --, -- ) Effects: Hungry( NO ) Hungry( YES ) ^ Hungry( NO )

38. STRIPS Planning Example Action: Buy( object ) Preconditions: ( -- ) Effects: Delete List: -- Add List: Own( object ) Own( ) ^ Own( ) ^ Own( )

39. STRIPS Planning Example

44. FPS Statistics Average lifespan of FPS enemy AI:

45. FPS Statistics Average lifespan of FPS enemy AI: 12.23 seconds

46. FPS Statistics Average lifespan of FPS enemy AI: 12.23 seconds Number of enemy AI who die in FPS’s per day:

47. FPS Statistics Average lifespan of FPS enemy AI: 12.23 seconds Number of enemy AI who die in FPS’s per day: 19,789,203

48. FPS Statistics Sally Struthers says “Save the AI!”

49. Here’s the Plan: STRIPS Planning Overview Planning in F.E.A.R. Differences from STRIPS Squad Behaviors & Communication Beyond F.E.A.R.

50. Design Philosophy Designer’s job is: Create environments that allow AI to showcase their behaviors. Designer’s job is NOT: Script behavior of individual AI.

51. Planning Video #1

57. Planning Video #2

58. Planning Video #3

59. Planning Video #4

60. SoldierAssassin Rat

61. Benefits of Planning 1.Decoupling Goals & Actions 2.Layering Behaviors 3.Dynamic Problem Solving

62. Decoupling Goals & Actions

65. + =

71. Benefits of Planning 1.Decoupling Goals & Actions 2.Layering Behaviors 3.Dynamic Problem Solving

72. Layering Behaviors

88. Kill Enemy Attack Ranged

89. Layering Behaviors Kill Enemy Attack Ranged Attack Melee

90. Layering Behaviors Kill Enemy Attack Ranged Attack Melee Dodge Shuffle Dodge Roll Dodge

91. Layering Behaviors Kill Enemy Attack Ranged Attack Melee Dodge Shuffle Dodge Roll Dodge Cover Goto Dodge Covered

92. Layering Behaviors Kill Enemy Attack Ranged Attack Melee Dodge Shuffle Dodge Roll Dodge Cover Goto Dodge Covered Blind Fire

93. Layering Behaviors Kill Enemy Attack Ranged Attack Melee Dodge Shuffle Dodge Roll Dodge Cover Goto Dodge Covered Blind Fire Ambush

94. Layering Behaviors Kill Enemy Attack Ranged Attack Melee Dodge Shuffle Dodge Roll Dodge Cover Goto Dodge Covered Blind Fire Ambush

95. Layering Behaviors

96. Benefits of Planning 1.Decoupling Goals & Actions 2.Layering Behaviors 3.Dynamic Problem Solving

97. Dynamic Problem Solving

99. Differences From STRIPS 1.Cost Per Action 2.No Add/Delete Lists 3.Procedural Preconditions 4.Procedural Effects

100. Cost Per Action

101. 2 8

102. 2 8 * A

103. A

104. A*NavigationPlanning Nodes:NavMesh PolysWorld States Edges:NavMesh Poly Actions Edges Goal:NavMesh PolyWorld State

105. Cost Per Action A

106. Differences From STRIPS 1.Cost Per Action 2.No Add/Delete Lists 3.Procedural Preconditions 4.Procedural Effects

107. No Add/Delete Lists State: (phone#, recipe, hungry?) Action Preconditions: (, --, -- ) Effects: Delete List: Hungry( YES ) Add List: Hungry( NO )

108. No Add/Delete Lists State: [phone#, recipe, hungry?] Action Preconditions: [, --, -- ] Effects: [ --, --, NO ]

109. No Add/Delete Lists Attack Preconditions: [ --, --, ] Reload Effects: [ --, --, ]

110. No Add/Delete Lists [ --, --, -- ] 4-byte values TargetDead[bool] WeaponLoaded[bool] OnVehicleType[enum] AtNode[HANDLE] - or - AtNode[variable*]

111. No Add/Delete Lists Goal Cover: [ --, CoverNode63, -- ] Action Goto Effect: [ --, variable*, -- ]

112. No Add/Delete Lists [ --, --, ]

113. No Add/Delete Lists [ --, --, ]

114. Differences From STRIPS 1.Cost Per Action 2.No Add/Delete Lists 3.Procedural Preconditions 4.Procedural Effects

115. Procedural Preconditions class Action { // [ --, --, -- ] WORLD_STATE m_Preconditions; WORLD_STATE m_Effects; bool CheckPreconditions(); };

116. Procedural Preconditions

117. Differences From STRIPS 1.Cost Per Action 2.No Add/Delete Lists 3.Procedural Preconditions 4.Procedural Effects

118. Procedural Effects class Action { // [ --, --, -- ] WORLD_STATE m_Preconditions; WORLD_STATE m_Effects; bool CheckPreconditions(); void ActivateAction(); };

119. Procedural Effects void ActionFlee::ActivateAction() { //... }

120. Squad Behaviors

121. Global Coordinator

122. Squad 1 Squad 2 Squad 3

123. Squad Behaviors 1.Simple 2.Complex

124. Squad Behaviors 1.Simple –Suppression fire –Goto positions –Follow leader 2.Complex –Flanking –Coordinated strikes –Retreats –Reinforcements

125. Simple Squad Behaviors Get-to-Cover Advance-Cover Orderly-Advance Search

126. Simple Squad Behaviors 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail.

127. Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

128. Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

129. Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

130. Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

131. Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

132. Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

133. Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

134. Get-to-Cover 1.Find participants. 2.Send orders. 3.Monitor progress. 4.Succeed or fail. Player

135. Squad Behaviors 1.Simple –Suppression fire –Goto positions –Follow leader 2.Complex –Flanking –Coordinated strikes –Retreats –Reinforcements

136. Complex Squad Behaviors There are none!!

137. Complex Squad Behaviors Player

138. Complex Squad Behaviors Player

139. Complex Squad Behaviors Player

140. Squad Behavior Implementation Key is separation between squad layer and individual behaviors. Possibly formalize with Hierarchical Task Network (HTN) planning.

141. Squad Communication “I need reinforcements!”

142. Squad Communication Dialogue, instead of announcements: –Announcement: “Ouch!” –Dialogue: AI1: “What’s your status?” AI2: “I’m hit!” (or “I’m alright!”)

143. Squad Communication Dialogue, instead of announcements: –Announcement: “Where did he go?” –Dialogue: AI1: “Can you see anything?” AI2: “No sir!” (or “He’s behind the boxes!”)

144. Squad Communication Dialogue also explains lack of action: –Dialogue: AI1: “He’s aiming at you! Get out of there!” AI2: “I’ve got nowhere to go!”

145. Squad Communication “…Not only do they give each other orders, but they actually DO what they’re told!”

146. Squad Communication “…Not only do they give each other orders, but they actually DO what they’re told!” Reality: Squad behavior has bird’s eye view of the situation. Find appropriate dialogue sequence after the fact.

147. Here’s the Plan: STRIPS Planning Overview Planning in F.E.A.R. Differences from STRIPS Squad Behaviors & Communication Beyond F.E.A.R.

148. “… just like the marines in Half-Life 1”

149. 1998 2005

150. MIT Media Lab: Cognitive Machines Group http://www.media.mit.edu/cogmac

152. Dialogue Integration “What’s your status?”

153. Unified Planning for Actions and Speech “Look out! Grenade!” Precondition: Effect:

154. Unified Planning for Actions and Speech OpenDoor -or- “Open the door!” Precondition: door is closed Effect: door is open

155. Plan Recognition UnlockDoor CreateKey LightFireGotoForge GetGold

156. Plan Recognition Plan as Context Free Grammar (CFG): UnlockDoorCreateKey LightFireGotoForge GetGold CreateKey GotoForge : : :

157. Plan Recognition PLAN RECOGNITION VIDEO

158. Questions? Jeff Orkin jorkin@blarg.net Monolith is hiring! Engineers Artists Designers http://www.lith.com/jobs