Artificial Intelligence in Game Design Goal-Oriented Action Planning

Planning Creating series of actions to meet some goal Planning actions to meet multiple needs Chosen so final state has highest utility Intermediate states after part of sequence should not be unacceptable Allows fast actions to be chosen in logical circumstances Planning actions with multiple steps before payoff Purchase ingredients Cook ingredients in oven Eat meal No effect on hunger until after last step

Planning Example 85 81 107 169 Best action if single action allowed Effect on Fun Effect on Energy Paint -2 +1 Go to Concert -9 +3 Nap in Chair Sleep in Bed +5 -10 Need Current level After Paint After Go to Concert After Nap in Chair After Sleep in Bed Fun 8 62 02 92 132 Energy 6 72 42 Total effect 100 85 81 107 169 Best action if single action allowed Will be very tired afterward!

Planning Example Better approach for two large needs: Example: 107 49 Take fast action to relieve one Then take another action to relieve other Example: Take nap before concert Need Current level After Nap in Chair Then After Go to Concert Fun 8 92 02 Energy 6 42 72 Total effect 100 107 49 Better than taking single action

Planning and Search Trees Must try all possible combinations of actions Compute total discontentment for each path Choose path with lowest total root Paint Go to Concert Nap in Chair Sleep in Bed Paint Paint Go to Concert Paint Nap in Chair Paint Sleep in Bed Nap in Chair Paint Nap in Chair Go to Concert Nap in Chair Nap in Chair Sleep in Bed Go to Concert Paint Go to Concert Go to Concert Nap in Chair Go to Concert Sleep in Bed Sleep in Bed Paint Sleep in Bed Go to Concert Sleep in Bed Nap in Chair Sleep in Bed

Planning and Search Trees Expanding search tree is costly Given a possible actions Given n possible levels an possibilities to test Will only be able to test small number of action combinations (lookahead limit) Even two actions enough to seem intelligent Looks like character thinking ahead

Planning and Search Trees Algorithm: depth limited search Depth-first search to some fixed limit n an leafs in tree At each leaf, compute discontentment Just keep track of best path found so far Storage cost: n Best sequence of actions found so far Current path being tested

Pruning and Search Trees Can cut off search down obviously bad branches Branches with intermediate state unacceptably bad Can possibly save search time (but no guarantee) Character dead No path can be better than best found so far, so no further search Character dead No path can be better than best found so far, so no further search Discontentment = 172

Planning and Preconditions Actions may have preconditions before they can be taken Multiple actions required to meet needs Action Preconditions Postconditions Preconditions which character meets Action Postconditions meets Action Preconditions Postconditions meet need 9

Planning and Preconditions Example: Microwaving food Uncooked food in fridge Get food Have uncooked food meets Have uncooked food Have microwave Cook food in microwave Have cooked food meets Have cooked food Eat food Hunger - 5

STRIPS Planning Simple planning for worlds with preconditions on actions “Stanford research intelligent planning system” Prohibitively expensive for complex problems (thousands of possible actions, prerequisites, etc.) Widely used in games where actions limited (Quake, Age of Empires, etc.) Plan of attack for single NPC Series of actions to meet large-scale goal (“increase number of citizens farming”) Action Preconditions Postconditions

STRIPS Planning Regressing searching: Working backwards from goals to preconditions goallist = {unmet needs} while (unmet goals in goallist) { choose goal G from goallist select action A with G in postconditions remove G from goallist add unmet preconditions of A to goallist }

STRIPS Planning Example Initial goal Choose “attack” action which meets initial goal Precondition added to list Choose “load weapon” to meet precondition Its precondition added That precondition already true, so plan complete

Selecting Actions Different characters  Different set of possible actions Algorithm selects from that character’s actions which meet prerequisites Key in current room Get Key Door unlocked Open Door Door unlocked Be in room Smash Door Be in room

Planning as Search Can take form of search tree Depth-limited search Must backtrack if current plan has unmeetable preconditions Adds appearance of intelligence Can be expensive if happens too often No available action has this as postcondition Key in current room Get Key Door unlocked Open Door Door unlocked Backtrack and try this action Pick Lock Door unlocked

Planning as A* Search Can use A* to choose next action to add to tree Cost of path found so far Actions assigned costs Cost of rest of plan from current state = sum of actions from current state to goal Heuristic estimate of rest of path Number of unmet preconditions Possibly weighted by some “expected cost” measure Action Action Precondition Goal Precondition Precondition

Planning as A* Search Example: “Open Door” action has cost 1 “Get Key” action has cost 2 “Pick Lock” action has cost 5 “Key in current room” precondition estimated cost = 1 Try first since estimated path cost = 4 Key in current room Get Key Door unlocked Open Door Door unlocked Try next since estimated path cost = 6 Pick Lock Door unlocked

Planning as Tree Building Actions with multiple preconditions represented as trees Actions with preconditions  Nodes with preconditions as branches Actions with no preconditions  Leafs Have 1000 wood Order troops to collect 1000 wood Build Cannon Factory Have 1000 iron Build Cannons Order troops to collect 100 wood Build Iron Mine Mine iron Produce Cannons in Factory Order troops to mine 1000 iron

Planning and Scheduling Key question: How to schedule the actions that result from plan Some branches may have to be accomplished before others Some branches can be done in parallel Can be done in parallel Order troops to collect 1000 wood Build Cannon Factory Build Cannons preceeds Order troops to collect 100 wood Build Iron Mine Mine iron preceeds Produce Cannons in Factory Order troops to mine 1000 iron

Hierarchical Planning Overall plan created at different levels of abstraction Low-level subplans created to do higher level tasks High-level tasks have preconditions and postconditions High-level tasks have methods for expanding into subplans Steps of subplan accomplishes postconditions given preconditions Task Task Task Preconditions Postconditions Main plan Action Task Action Preconditions Postconditions Subplan Action Action

Hierarchical Planning General plan to capture a city Gather weapons Move to city Attack city Have weapons At city Gather wood Build bows and arrows Move to bridge Cross bridge Move to city At city Subplan generated for this particular city, level of technology, etc.

Dynamic Replanning Conditions used to create plan may no longer apply in course of executing plan Bridge NPC destroyed by player while gathering weapons Can create a different subplan to accomplish specific task without replanning entire plan “Move to city” subplan accomplished by building boats Move to city Have weapons At city Move to river Gather wood Build boats Cross river in boats Move to city At city

Planning for Hierarchical Teams Goals of team should direct goals of individuals Individuals take actions to support overall actions of team

Planning for Hierarchical Teams Individual units meet goal based on individual abilities No micromanaging from overall system Pick Lock Open Door Look in Room Search all rooms Look in Room with X-ray Vision Overall team goal Smash Door Look in Room

Planning for Hierarchical Teams Individual units can balance team goals with individual goals to give appearance of independent thought Each individual/subteam has relevance measure for different goals Meets goal with highest current relevance Goal Charge TakeCover Retreat Patrol Relevance 0.6 0.8 0.3 0.0 Current action

Planning for Hierarchical Teams Individual goal modified by goals of team Example: Leader orders squad to charge, and not retreat Goal Charge TakeCover Retreat Patrol Relevance 0.6 0.8 0.3 0.0 Modifier 0.5 -0.5 Total 1.1 -0.2 Current action

Planning for Hierarchical Teams If individual goals strong, team goals may be overridden! Example: Leader orders squad to patrol This unit currently under ambush Goal Charge TakeCover Retreat Patrol Relevance 0.4 0.8 0.6 0.0 Modifier 0.5 Total Current action