1. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 1 Chapter 11 Hierarchical Task Network Planning Lecture slides for Automated Planning: Theory and Practice Dana S. Nau CMSC 722, AI Planning University of Maryland, Fall 2004

2. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 2 Motivation l We may already have an idea how to go about solving problems in a planning domain l Example: travel to a destination that’s far away: u Domain-independent planner: »many combinations vehicles and routes u Experienced human: small number of “recipes” e.g., flying: 1. buy ticket from local airport to remote airport 2. travel to local airport 3. fly to remote airport 4. travel to final destination l How to enable planning systems to make use of such recipes?

3. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 3 Two Approaches l Control rules (previous chapter): u Write rules to prune every action that doesn’t fit the recipe l Hierarchical Task Network (HTN) planning: u Describe the actions and subtasks that do fit the recipe

4. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 4 HTN Planning method travel(x,y) get-ticket (a(x), a(y)) travel (x, a(x))fly (a(x), a(y)) travel (a(y),y) air-travel(x,y) get-taxiride-taxi (x,y)pay-driver taxi-travel(x,y) travel(UMD, LAAS) get-ticket(BWI, Toulouse) go to Orbitz find-flights(BWI,Toulouse) buy-ticket(BWI,Toulouse) travel(UMD, BWI) get-taxi ride-taxi(UMD, BWI) pay-driver fly(BWI, Toulouse) travel(Toulouse, LAAS) get-taxi ride-taxi(Toulouse, LAAS) pay-driver task l A type of problem reduction u Decompose tasks into subtasks u Handle constraints (e.g., taxi not good for long distances) u Resolve interactions (e.g., take taxi early enough to catch plane) u If necessary, backtrack and try other decompositions

5. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 5 HTN Planning l HTN planners may be domain-specific u e.g., see Chapters 20 (robotics) and 23 (bridge) l Or they may be domain-configurable u Domain-independent planning engine u Domain description »methods, operators u Problem description »domain description, initial state, initial task network

6. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 6 Simple Task Network (STN) Planning l A special case of HTN planning l States and operators u The same as in classical planning l Task: an expression of the form t(u 1,…,u n ) u t is a task symbol, and each u i is a term u Two kinds of task symbols (and tasks): »primitive: tasks that we know how to execute directly task symbol is an operator name »nonprimitive: tasks that must be decomposed into subtasks use methods (next slide)

7. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 7 Methods l Totally-ordered method: a 4-tuple m = (name(m), task(m), precond(m), subtasks(m)) u name(m): an expression of the form n(x 1,…,x n ) »x 1,…,x n are parameters - variable symbols u task(m): a nonprimitive task u precond(m): preconditions (literals) u subtasks(m): a sequence of tasks  t 1, …, t k  air-travel (x,y) task: travel (x,y) precond: long-distance (x,y) subtasks:  buy-ticket (a(x),b(x)), travel (x,a(x)), fly (a(x),b(x)), travel (b(x),y)  travel(x,y) buy-ticket (a(x), a(y))travel (x, a(x))fly (a(x), a(y))travel (a(y), y) long-distance(x,y) air-travel(x,y)

8. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 8 l Partially ordered method: a 4-tuple m = (name(m), task(m), precond(m), subtasks(m)) u name(m): an expression of the form n(x 1,…,x n ) »x 1,…,x n are parameters - variable symbols u task(m): a nonprimitive task u precond(m): preconditions (literals) u subtasks(m): a partially ordered set of tasks {t 1, …, t k } air-travel (x,y) task: travel (x,y) precond: long-distance (x,y) network:u 1 = buy-ticket (a(x),b(x)), u 2 = travel (x,a(x)), u 3 = fly (a(x),b(x)) u 4 = travel (b(x),y), {(u 1,u 3 ), (u 2,u 3 ), (u 3,u 3 )} travel(x,y) buy-ticket (a(x), a(y))travel (x, a(x))fly (a(x), a(y))travel (a(y), y) long-distance(x,y) air-travel(x,y) Methods

9. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 9 Domains, Problems, Solutions l STN planning domain: methods, operators l STN planning problem: methods, operators, initial state, task list l Total-order STN planning domain and planning problem: u Same as above except that all methods are totally ordered l Solution: any executable plan that can be generated by recursively applying u methods to nonprimitive tasks u operators to primitive tasks nonprimitive task precond method instance s0s0 precondeffectsprecondeffects s1s1 s2s2 primitive task operator instance

10. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 10 Example l Suppose we want to move three stacks of containers in a way that preserves the order of the containers

11. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 11 Example (continued) l A way to move each stack: u first move the containers from p to an intermediate pile r u then move them from r to q

12. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 12 Partial-Order Formulation

13. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 13 Total-Order Formulation

14. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 14 Solving Total-Order STN Planning Problems state s; task list T=( t 1,t 2,…) action a state  (s,a) ; task list T=(t 2, …) task list T=( u 1,…,u k,t 2,…) task list T=( t 1,t 2,…) method instance m

15. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 15 Expressivity Relative to Classical Planning l Any classical planning planning problem can be translated into an ordered-task-planning problem in polynomial time l Several ways to do this. One is roughly as follows: u For each goal or precondition e, create a task t e u For each operator o and effect e, create a method m o,e »Task: t e »Subtasks: t c1, t c2, …, t cn, o, where c 1, c 2, …, c n are the preconditions of o »Partial-ordering constraints: each t ci precedes o l There are HTN planning problems that cannot be translated into classical planning problems at all l Example on the next page

16. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 16 l Two methods: u No arguments u No preconditions Two operators, a and b u Again, no arguments and no preconditions Initial state is empty, initial task is t Set of solutions is { a n b n | n > 0} l No classical planning problem has this set of solutions u The state-transition system is a finite-state automaton  No finite-state automaton can recognize { a n b n | n > 0} method1 bt a t method2 ba t Example

17. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 17 Comparison to Forward and Backward Search l In state-space planning, must choose whether to search forward or backward l In HTN planning, there are two choices to make about direction: u forward or backward u up or down TFD goes down and forward s0s0 s1s1 s2s2 … … op 1 op 2 op i S i–1 s0s0 s1s1 s2s2 … task t m … … task t n op 1 op 2 op i S i–1 task t 0

18. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 18 Comparison to Forward and Backward Search Like a backward search, TFD is goal-directed u Goals correspond to tasks l Like a forward search, it generates actions in the same order in which they’ll be executed l Whenever we want to plan the next task u we’ve already planned everything that comes before it u Thus, we know the current state of the world s0s0 s1s1 s2s2 … task t m … … task t n op 1 op 2 op i S i–1 task t 0

19. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 19 Us:East declarer, West dummy Opponents:defenders, South & North Contract:East – 3NT On lead:West at trick 3 East:  KJ74 West:  A2 Out:  QT98653 Increasing Expressivity Further l Knowing the current state makes it easy to do things that would be difficult otherwise u States can be arbitrary data structures u Preconditions and effects can include »logical inferences (e.g., Horn clauses) »complex numeric computations »interactions with other software packages l e.g., SHOP: http://www.cs.umd.edu/projects/shop

20. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 20 Example l Simple travel-planning domain u Go from one location to another u State-variable formulation

21. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 21 Precond: distance(home,park) ≤ 2 Precond: cash(me) ≥ 1.50 + 0.50*distance(home,park) Initial task: travel(me,home,park) Precondition succeeds travel-by-foottravel-by-taxi Precondition fails Decomposition into subtasks home park Planning Problem: I am at home, I have $20, I want to go to a park 8 miles away s 1 = {location(me)=home, location(taxi)=home, cash(me)=20, distance(home,park)=8} Initial state s 0 = {location(me)=home, cash(me)=20, distance(home,park)=8} call-taxi(me,home)ride(me,home,park)pay-driver(me,home,park) Precond: … Effects: … Precond: … Effects: … Precond: … Effects: … s 2 = {location(me)=park, location(taxi)=park, cash(me)=20, distance(home,park)=8 s 3 = {location(me)=park, location(taxi)=park, cash(me)=14.50, distance(home,park)=8} Final state s 1 s 2 s 3 s 0

22. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 22 l Cannot interleave subtasks of different tasks l Sometimes this can make things awkward l Need methods that reason globally instead of locally walk(a,b) pickup(p) walk(b,a) get(p)get(q) get-both(p,q) goto(b) pickup(p)pickup(q) get-both(p,q) Limitation of Ordered-Task Planning pickup-both(p,q) walk(a,b) pickup(q) walk(b,a) walk(a,b) goto(a) walk(b,a)

23. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 23 Generalize the Methods l Generalize methods to allow the subtasks to be partially ordered l Consequence: plans may interleave subtasks of different tasks l This makes the planning algorithm more complicated walk(a,b)pickup(p) get(p) stay-at(b)pickup(q) get(q) get-both(p,q) walk(b,a)stay-at(a)

24. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 24 π={a 1 …, a k, a }; w’={t 2,t 3 …} w={ t 1,t 2,…} method instance m w’={ u 1,…,u k,t 2,…} π={a 1,…, a k }; w={ t 1,t 2, t 3 …} operator instance a Generalize TFD to interleave subtasks

25. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 25 w={ t 1,t 2,…} method instance m w’={ u 1,…,u k,t 2,…} l  (w, u, m,  ) has a complicated definition in the book. Here’s what it means: u We selected t 1 because it’s possible for t 1 to come first u We’re planning for t 1 under the assumption that it will come first u Insert ordering constraints to ensure it will come first u The same constraints also must apply to all subtasks of t 1 Generalize TFD to interleave subtasks

26. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 26 Discussion PFD is sound and complete Can generalize in the same ways as TFD SHOP2: implementation of PFD -like algorithm + generalizations u Won one of the top four awards in the AIPS-2002 Planning Competition u Freeware, open source u Implementation available at http://www.cs.umd.edu/projects/shop

27. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 27 HTN Planning l HTN planning is even more general u Can have constraints associated with tasks and methods »Things that must be true before, during, or afterwards u Some algorithms use causal links and threats like those in PSP l There’s a little about this in the book u I won’t discuss it

28. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 28 TLPlan’s Expressivity Compared with SHOP and SHOP2 l Equivalent expressive power u Both are Turing-complete, because both allow function symbols u Both allow attached procedures, numeric computations l Both know the current state at each step of the planning process, and use this to prune operators l Both can call external subroutines u SHOP uses “eval” to call LISP functions u In TLPlan, a function symbol can correspond to a computed function l Main difference u in SHOP and SHOP2, the methods talk about what can be done »SHOP and SHOP2 don’t do anything unless a method says to do it u TLPlan’s control rules talk about what cannot be done »TLPlan will try everything that the control rules don’t prohibit l Which approach is more convenient depends on the problem domain

29. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 29 Experimental Comparison l Several years ago, we did a comparison of SHOP, TLPlan, and Blackbox u Blackbox is a domain-independent planner that uses a combination of Graphplan and satisfiability u One of the two fastest planners in the 1998 planning competition l Test domain: the logistics domain u A classical planning problem »Much simpler than real logistics planning u Scenario: use trucks and airplanes to deliver packages u Like a simplified version of the DWR domain in which containers don’t get stacked on each other l Test conditions u SHOP and TLPlan on a 167-MHz Sun Ultra with 64 MB of RAM u We couldn’t run Blackbox on our machine u Published results: Blackbox on a faster machine with 8 GB of RAM

30. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 30 Logistics Domain Results

31. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 31 Logistics Domain Results (continued) AverageBlackboxTLPlanSHOP CPU time 327.1 2.9 1.1 Average no.BlackboxTLPlanSHOP of actions 82.5 54.5 51.9 Same graph as before, but on a logarithmic scale Number of actions in the plans

32. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 32 Summary l TLPlan and SHOP took similar amounts of time u In this experiment, SHOP was slightly faster, but in others TLPlan may be faster l Blackbox took about 1000 times as much time and needed about 100 times as much memory l Reasons why: u SHOP’s input included domain-specific methods & axioms u TLPlan’s input included domain-specific control rules »This enabled them to find near-optimal solutions in low- order polynomial time and space u Blackbox is a fully automated planner »No domain-specific knowledge »trial-and-error search, exponential time and space

33. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 33 Domain-Configurable Planners Compared to Classical Planners l Disadvantage: writing a knowledge base can be more complicated than just writing classical operators l Advantage: can encode “recipes” as collections of methods and operators u Express things that can’t be expressed in classical planning u Specify standard ways of solving problems »Otherwise, the planning system would have to derive these again and again from “first principles,” every time it solves a problem »Can speed up planning by many orders of magnitude (e.g., polynomial time versus exponential time)

34. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 34 Example from the AIPS-2002 Competition l The satellite domain u Planning and scheduling observation tasks among multiple satellites u Each satellite equipped in slightly different ways l Several different versions. I’ll show results for the following: u Simple-time: »concurrent use of different satellites »data can be acquired more quickly if they are used efficiently u Numeric: »fuel costs for satellites to slew between targets; finite amount of fuel available. »data takes up space in a finite capacity data store »Plans are expected to acquire all the necessary data at minimum fuel cost. u Hard Numeric: »no logical goals at all – thus even the null plan is a solution »Plans that acquire more data are better – thus the null plan has no value »None of the classical planners could handle this

35. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 35

36. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 36

37. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 37

38. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 38

39. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 39

40. Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ 40