State-Space Planning Sources: Ch. 3 Appendix A Slides from Dana Nau’s lecture Dr. Héctor Muñoz-Avila

Reminder: Some Graph Search Algorithms (I) 20 G = (V,E) 10 G Z Edges: set of pairs of vertices (v,v’) 9 16 B F 6 C 8 24 A 4 6 Vertices E D 20 Breadth-first search: Use a queue (FIFO policy) to control order of visits State-firsts search: Use an stack (FILA policy) to control order of visits Best-first search: minimize an objective function f(): e.g., distance. Djikstra Greedy search: select best local f() and “never look back” Hill-climbing search: like greedy search but every node is a solution Iterative deepening: exhaustively explore up to depth i

State-Space Planning C A B C A B C B A B A C B A C B A B C C B A B A C Search is performed in the state space State space V: set of states E: set of transitions (s, (s,a)) But: computed graph is a subset of state space Remember the number of states for DWR states? Search modes: Forward Backward C A B C B A B A C B A C B A B C C B A B A C A C A A B C B C C B A A B C

Nondeterministic Search The oracle definition: given a choice between possible transitions, there is a device, called the oracle, that always chooses the transition that leads to a state satisfying the goals ... … ... Oracle chooses this transition because it leads to a favorable state “current state” ... Yes, they are both equivalent. One can exhaustively try all possible derivations to find the one selected by the oracle. Efficiency issues (i.e., oracle would find a favorable state faster) are not considered (topic of CSE 340) Alternative definition (book): parallel computers

Forward Search goals+(g)  s and goals–(g)  s =  satisfied take c3 … move r1 …

Soundness and Completeness A planning algorithm A can be sound, complete, and admissible Soundness: Deterministic: Given a problem and A returns P  failure then P is a solution for the problem Nondeterministic: Every plan returned by A is a solution Completeness: Deterministic: Given a solvable problem then A returns P  failure Nondeterministic: Given a solvable problem then A returns a solution Admissible: If there is a measure of optimality, and a solvable problem is given, then A returns an optimal solution Result: Forward-search is sound and complete Proof: …

Reminder: Plans and Solutions Plan: any sequence of actions  = a1, a2, …, an such that each ai is a ground instance of an operator in O The plan is a solution for P=(O,s0,g) if it is executable and achieves g i.e., if there are states s0, s1, …, sn such that  (s0,a1) = s1  (s1,a2) = s2 …  (sn–1,an) = sn sn satisfies g Example of a deterministic variant of Forward-search that is not sound? Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Deterministic Implementations Some deterministic implementations of forward search: breadth-first search depth-first search best-first search (e.g., A*) greedy search Breadth-first and best-first search are sound and complete But they usually aren’t practical because they require too much memory Memory requirement is exponential in the length of the solution In practice, more likely to use depth-first search or greedy search Worst-case memory requirement is linear in the length of the solution In general, sound but not complete But classical planning has only finitely many states Thus, can make depth-first search complete by doing loop-checking s1 a1 s4 a4 s0 s2 sg a2 s5 a5 … a3 s3 So why they are typically not used? How can we make it complete? Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Another domain: The Blocks World Infinitely wide table, finite number of children’s blocks Ignore where a block is located on the table A block can sit on the table or on another block Want to move blocks from one configuration to another e.g., initial state goal Can be expressed as a special case of DWR But the usual formulation is simpler a d b c c a b e

Classical Representation: Symbols Constant symbols: The blocks: a, b, c, d, e Predicates: ontable(x) - block x is on the table on(x,y) - block x is on block y clear(x) - block x has nothing on it holding(x) - the robot hand is holding block x handempty - the robot hand isn’t holding anything d c a b e Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Classical Operators c a b unstack(x,y) Precond: on(x,y), clear(x), handempty Effects: ~on(x,y), ~clear(x), ~handempty, holding(x), clear(y) stack(x,y) Precond: holding(x), clear(y) Effects: ~holding(x), ~clear(y), on(x,y), clear(x), handempty pickup(x) Precond: ontable(x), clear(x), handempty Effects: ~ontable(x), ~clear(x), ~handempty, holding(x) putdown(x) Precond: holding(x) Effects: ~holding(x), ontable(x), clear(x), handempty c a b c a b c b a c a b Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Branching Factor of Forward Search … a50 a3 initial state goal Forward search can have a very large branching factor E.g., many applicable actions that don’t progress toward goal Why this is bad: Deterministic implementations can waste time trying lots of irrelevant actions Need a good heuristic function and/or pruning procedure Chad and Hai are going to explain how to do this

How do we write this formally? Backward Search For forward search, we started at the initial state and computed state transitions new state = (s,a) For backward search, we start at the goal and compute inverse state transitions new set of subgoals = –1(g,a) To define -1(g,a), must first define relevance: An action a is relevant for a goal g if a makes at least one of g’s literals true g  effects(a) ≠  a does not make any of g’s literals false g+  effects–(a) =  and g–  effects+(a) =  How do we write this formally? Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Inverse State Transitions If a is relevant for g, then –1(g,a) = (g – effects+(a))  precond(a) Otherwise –1(g,a) is undefined Example: suppose that g = {on(b1,b2), on(b2,b3)} a = stack(b1,b2) What is –1(g,a)? What is –1(g,a) = {s : (s,a) satisfies g}?

How about using –1(g,a) instead of –1(g,a)? Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Efficiency of Backward Search … a50 a3 initial state goal Compared with forward search? Backward search can also have a very large branching factor E.g., many relevant actions that don’t regress toward the initial state As before, deterministic implementations can waste lots of time trying all of them Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Lifting p(a,a) foo(a,a) p(a,b) foo(x,y) precond: p(x,y) effects: q(x) foo(a,b) p(a,c) q(a) foo(a,c) … Can reduce the branching factor of backward search if we partially instantiate the operators this is called lifting foo(a,y) q(a) p(a,y) Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Lifted Backward Search More complicated than Backward-search Have to keep track of what substitutions were performed But it has a much smaller branching factor Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

The Search Space is Still Too Large Lifted-backward-search generates a smaller search space than Backward-search, but it still can be quite large Suppose a, b, and c are independent, d must precede all of them, and d cannot be executed We’ll try all possible orderings of a, b, and c before realizing there is no solution More about this in Chapter 5 (Plan-Space Planning) d a b c d b a d b a goal b d a c d b c a d c b Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

STRIPS π  the empty plan do a modified backward search from g instead of -1(s,a), each new set of subgoals is just precond(a) whenever you find an action that’s executable in the current state, then go forward on the current search path as far as possible, executing actions and appending them to π repeat until all goals are satisfied π = a6, a4 s = ((s0,a6),a4) g6 satisfied in s0 g1 a1 a4 a6 g4 g2 g a2 g3 a3 g5 a5 a3 g3 Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ current search path

The Sussman Anomaly Initial state goal c b a b c Initial state goal On this problem, STRIPS can’t produce an irredundant solution Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

The Register Assignment Problem State-variable formulation: Initial state: {value(r1)=3, value(r2)=5, value(r3)=0} Goal: {value(r1)=5, value(r2)=3} Operator: assign(r,v,r’,v’) precond: value(r)=v, value(r’)=v’ effects: value(r)=v’ STRIPS cannot solve this problem at all Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

How to Fix? Several ways: Do something other than state-space search e.g., Chapters 5–8 Use forward or backward state-space search, with domain-specific knowledge to prune the search space Can solve both problems quite easily this way Example: block stacking using forward search Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Domain-Specific Knowledge A blocks-world planning problem P = (O,s0,g) is solvable if s0 and g satisfy some simple consistency conditions g should not mention any blocks not mentioned in s0 a block cannot be on two other blocks at once etc. Can check these in time O(n log n) If P is solvable, can easily construct a solution of length O(2m), where m is the number of blocks Move all blocks to the table, then build up stacks from the bottom Can do this in time O(n) With additional domain-specific knowledge can do even better …

Additional Domain-Specific Knowledge A block x needs to be moved if any of the following is true: s contains ontable(x) and g contains on(x,y) s contains on(x,y) and g contains ontable(x) s contains on(x,y) and g contains on(x,z) for some y≠z s contains on(x,y) and y needs to be moved Axioms initial state goal e d b a c Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Domain-Specific Algorithm loop if there is a clear block x such that x needs to be moved and x can be moved to a place where it won’t need to be moved then move x to that place else if there is a clear block x such that x needs to be moved then move x to the table else if the goal is satisfied then return the plan else return failure repeat initial state goal e d b a c Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Easily Solves the Sussman Anomaly loop if there is a clear block x such that x needs to be moved and x can be moved to a place where it won’t need to be moved then move x to that place else if there is a clear block x such that x needs to be moved then move x to the table else if the goal is satisfied then return the plan else return failure repeat initial state goal a c b a b c

Properties The block-stacking algorithm: Sound, complete, guaranteed to terminate Runs in time O(n3) Can be modified to run in time O(n) Often finds optimal (shortest) solutions But sometimes only near-optimal (Exercise 4.22 in the book) PLAN LENGTH for the blocks world is NP-complete Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

An Actual Application: FEAR. F.E.A.R.: First Encounter Assault Recon Here is a video clip Behavior of opposing NPCs is modeled through STRIPS operators Attack Take cover … Why? Otherwise you would need too build an FSM for all possible behaviours in advance

FSM vs AI Planning FSM: Planning Operators A resulting plan: Patrol Preconditions: No Monster Effects: patrolled Fight Monster in sight Patrol Fight Monster In Sight No Monster FSM: A resulting plan: Patrol patrolled Fight No Monster Monster in sight Neither is more powerful than the other one

But Planning Gives More Flexibility “Separates implementation from data” --- Orkin reasoning knowledge Many potential plans: Patrol Fight … Planning Operators Patrol Preconditions: No Monster Effects: patrolled Fight Monster in sight … If conditions in the state change making the current plan unfeasible: replan!