1. 1 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Planning This material is covered in R&N 3 rd edition chapter 10 This material is covered in R&N 2 nd edition chapters 10.3, 11.1, 11.2, 11.4

2. 2 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 4.A drawing or diagram made to scale showing the structure or arrangement of something. 5.In perspective rendering, one of several imaginary planes perpendicular to the line of vision between the viewer and the object being depicted. 6.A program or policy stipulating a service or benefit: a pension plan. Synonyms: blueprint, design, project, scheme, strategy plan n. 1.A scheme, program, or method worked out beforehand for the accomplishment of an objective: a plan of attack. 2.A proposed or tentative project or course of action: had no plans for the evening. 3.A systematic arrangement of elements or important parts; a configuration or outline: a seating plan; the plan of a story.

3. 3 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 4.A drawing or diagram made to scale showing the structure or arrangement of something. 5.In perspective rendering, one of several imaginary planes perpendicular to the line of vision between the viewer and the object being depicted. 6.A program or policy stipulating a service or benefit: a pension plan. Synonyms: blueprint, design, project, scheme, strategy plan n. 1.A scheme, program, or method worked out beforehand for the accomplishment of an objective: a plan of attack. 2.A proposed or tentative project or course of action: had no plans for the evening. 3.A systematic arrangement of elements or important parts; a configuration or outline: a seating plan; the plan of a story.

4. 4 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 [a representation] of future behavior … usually a set of actions, with temporal and other constraints on them, for execution by some agent or agents. - Austin Tate [MIT Encyclopedia of the Cognitive Sciences, 1999] A portion of a manufacturing process plan

5. 5 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Modes of Planning Mixed Initiative Planning Automated Plan Generation

6. 6 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Why Planning Intelligent agents must operate in the world. They are not simply passive reasoners (Knowledge Representation, reasoning under uncertainty) or problem solvers (Search), they must also act on the world. We want intelligent agents to act in “intelligent ways”. Taking purposeful actions, predicting the expected effect of such actions, composing actions together to achieve complex goals.

7. 7 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Why Planning E.g. if we have a robot we want the robot to decide what to do; how to act to achieve our goals

8. 8 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 A Planning Problem How to change the world to suit our needs Critical issue: we need to reason about what the world will be like after doing a few actions, not just what it is like now GOAL: Steven has coffee CURRENTLY: robot in mailroom, has no coffee, coffee not made, Steven in office, etc. TO DO: goto lounge, make coffee,…

9. 9 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example Planning Applications

10. 10 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Autonomous planning, scheduling, control NASA: JPL and Ames Remote Agent Experiment (RAX) Deep Space 1 Mars Exploration Rover (MER) Autonomous Agents for Space Exploration

11. 11 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Other Autonomous Systems

12. 12 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Other Autonomous Systems But many of these currently lack any substantial type of “System 2” deliberative thinking, and in particular the type of deliberative thinking that we study here.

13. 13 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Sheet-metal bending machines - Amada Corp Software to plan the sequence of bends [Gupta and Bourne, J. Manufacturing Sci. and Engr., 1999] Manufacturing Automation

14. 14 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 E.g., Bridge Baron - Great Game Products 1997 world champion of computer bridge [Smith, Nau, Throop, AI Magazine, 1998] 2004: 2nd place (North —  Q) …… PlayCard(P 3 ; S, R 3 )PlayCard(P 2 ; S, R 2 )PlayCard(P 4 ; S, R 4 ) FinesseFour(P 4 ; S) PlayCard(P 1 ; S, R 1 ) StandardFinesseTwo(P 2 ; S) LeadLow(P 1 ; S) PlayCard(P 4 ; S, R 4 ’ ) StandardFinesseThree(P 3 ; S) EasyFinesse(P 2 ; S)BustedFinesse(P 2 ; S) FinesseTwo(P 2 ; S) StandardFinesse(P 2 ; S) Finesse(P 1 ; S) Us:East declarer, West dummy Opponents:defenders, South & North Contract:East – 3NT On lead:West at trick 3 East:  KJ74 West:  A2 Out:  QT98653 (North —  3) East —  J West —  2 North —  3South —  5South —  Q Games

15. 15 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Scheduling with Action Choices & Resource Requirements Problems in supply chain management HSTS (Hubble Space Telescope scheduler) Workflow management Air Traffic Control Route aircraft between runways and terminals. Crafts must be kept safely separated. Safe distance depends on craft and mode of transport. Minimize taxi and wait time. Character Animation Generate step-by-step character behaviour from high-level spec Plan-based Interfaces E.g. NLP to database interfaces Plan recognition, Activity Recognition Other Applications (cont.)

16. 16 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Other Applications (cont.) Web Service Composition Compose web services, and monitor their execution Many of the web standards have a lot of connections to plan representation languages BPEL; BPEL-4WS allow workflow specifications OWL-S allows process specifications Grid Services/Scientific Workflow Management Genome Rearrangement The relationship between different organisms can be measured by the number of “evolution events” (rearrangements) that separate their genomes Find shortest (or most likely) sequence of rearrangements between a pair of genomes

17. PLANNING 17 Sheila McIlraith, CSC384, University of Toronto, Winter 2016

18. 18 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Planning Reasoning about what the world will be like after doing a few actions is similar to what we have already examined. Only now we want to reason about dynamic environments. in(robby,room1), lightOn(room1) are true: will they be true after robby performs the action turnOffLights? in(robby,room1) is true: what does robby need to do to make in(robby,room3) true? Reasoning about the effects of actions, and computing what actions can achieve certain effects is at the heart of decision making.

19. 19 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Planning under Uncertainty One of the major complexities in planning that we will deal with later is planning under uncertainty. Our knowledge of the world will almost certainly be incomplete. We may wish to model this probabilistically. Sensing is subject to noise (especially in robots). Actions and effectors are also subject to error (uncertainty in their effects).

20. 20 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Classical Planning For now we restrict our attention to the deterministic case. We will examine: Complete initial state specifications deterministic effects of actions. finding sequences of actions that can achieve a desired set of effects. This will in some ways be a lot like search, but we will see that representation also plays an important role.

21. 21 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 What we’ll cover** Representing and reasoning about actions (including planning) in the situation calculus, a (mostly) first-order logical language for reasoning about action and change, and as a consequence, planning as deductive plan synthesis. Planning in STRIPS Planning as Search SAT-based Planning (to be discussed but not part of the syllabus) ** We will likely not get through all of this material in class. The point where you can stop for exam studying is well marked.

22. 22 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Situation Calculus First we look at how to model dynamic worlds within first- order logic. The situation calculus is an important formalism developed for this purpose. Situation Calculus is a (mostly) first-order language. Include in the domain of individuals a special set of objects called situations. Of these s 0 is a special distinguished constant which denotes the “initial” situation.

23. 23 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Situation Calculus Building Blocks Situations Fluents Actions

24. 24 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Situations Situations are the history of actions from s 0. You can think of them as indexing “states” of the world, but two different situations can have the same state. (E.g., “scratch, eat” may lead to the same state of the world as “eat, scratch” When dealing with dynamic environments, the world has different properties at different points in time. E.g., in(robby,room1, s 0 ),  in(robby,room3,s 0 )  in(robby,room3,s 1 ), in(robby,room1,s 1 ). Different things are true in situation s 1 than in the initial situation s 0. Contrast this with the previous kinds of knowledge we examined.

25. 25 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Fluents Previously, we were encoding a property of a term as a relation in first-order logic. The distinction here is that properties that change from situation to situation (called fluents) take an extra situation argument. E.g., clear(b)  clear(b,s) “clear(b)” is no longer statically true, it is true contingent on what situation we are talking about

26. 26 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Blocks World Example. clear(c,s 0 ) on(c,a,s 0 ) clear(b,s 0 ) handempty(s 0 ) C AB robot hand

27. 27 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Actions Actions are also part of language A set of “primitive” action objects in the (semantic) domain of individuals. In the syntax they are represented as functions mapping objects to primitive action objects. Examples: pickup(X) function mapping blocks to actions pickup(c) = “the primitive action object corresponding to ‘picking up block c’ stack(X,Y) stack(a,b) = “the primitive action object corresponding to ‘stacking a on top of b’

28. 28 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Actions applied to situation  new situation Remember that actions are terms in the language. In order to talk about the situation that results from executing an action in a particular situation, there is a “generic” action application function do(A,S). do maps a primitive action A and a situation S to a new situation. The new situation is the situation that results from applying A to S. Example: do(pickup(c), s 0 ) = the new situation that is the result of applying action “pickup(c)” to the initial situation s 0.

29. 29 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 What do Actions do? Actions affect the situation by changing what is true. on(c,a,s 0 ) i.e., c is on a in situation s 0 clear(a,do(pickup(c),s 0 )) i.e., a is clear in the situation resulting from picking up c in s 0 Two type of axioms are used to characterize the properties of actions: Action Precondition Axioms Action Effect Axioms

30. 30 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Specifying the effects of actions Action Preconditions: Certain things must hold for actions to have a predictable effect. the action pickup(c) is only applicable to situations S when clear(c,S)  handempty(S) is true. Action effects: Actions make certain things true and certain things false. holding(c, do(pickup(c), S)) i.e., “holding c” is true in the situation resulting from picking up c in situation S  X.  handempty(do(pickup(X),S)) i.e., “ handempty” is false in the situation resulting from picking up object X in situation S

31. 31 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Specifying the effects of actions Action effects are conditional on their precondition being true. The effect of action pickup(x):  S,X. ontable(X,S)  clear(X,S)  handempty(S)  holding(X, do(pickup(X),S))   handempty( do(pickup(X),S))   ontable(X, do(pickup(X,S))   clear(X, do(pickup(X,S)). Green indicates a situation term

32. 32 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Plan Generation There are many ways to generate plans. Here we show how to do it by representing actions in the situation calculus (as you have just seen) and generating a plan via deductive plan synthesis. This is not the approach taken by state-of-the-art planners, as we will see later, but it is where the field started and is still used for specifying, studying and advancing research for more complex tasks in reasoning about action and change …so for now, back to resolution!

33. 33 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Plan Generation Let’s use Resolution to find a plan. The KB contains a description of The initial state of the world The precondition axioms for actions The effect axioms for actions The query is the goal condition that we wish to achieve (we negate it to do resolution refutation)

34. 34 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Reasoning with the Situation Calculus 1. clear(c,s 0 ) 2. on(c,a,s 0 ) 3. clear(b,s 0 ) 4. ontable(a,s 0 ) 5. ontable(b,s 0 ) 6. handempty(s 0 ) Query:  Z.holding(b,Z) 7. (  holding(b,Z), ans(Z)) Does there exists a situation in which we are holding b? And if so what is the name of that situation. C AB

35. 35 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Resolution Convert “pickup” action axiom into clause form:  S,Y. ontable(Y,S)  clear(Y,S)  handempty(S)  holding(Y, do(pickup(Y),S))   handempty(do(pickup(Y),S))   ontable(Y,do(pickup(Y,S))   clear(Y,do(pickup(Y,S)). 8. (  ontable(Y,S),  clear(Y,S),  handempty(S), holding(Y,do(pickup(Y),S)) 9. (  ontable(Y,S),  clear(Y,S),  handempty(S),  handempty(do(pickup(X),S))) 10. (  ontable(Y,S),  clear(Y,S),  handempty(S),  ontable(Y,do(pickup(Y,S))) 11. (  ontable(Y,S),  clear(Y,S),  handempty(S),  clear(Y,do(pickup(Y,S)))

36. 36 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Resolution 12. R[8d, 7]{Y=b,Z=do(pickup(b),S)} (  ontable(b,S),  clear(b,S),  handempty(S), ans(do(pickup(b),S))) 13. R[12a,5] {S=s 0 } (  clear(b,s 0 ),  handempty(s 0 ), ans(do(pickup(b),s 0 ))) 14. R[13a,3] {} (  handempty(s 0 ), ans(do(pickup(b),s 0 ))) 15. R[14a,6] {} ans(do(pickup(b),s 0 ))

37. 37 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 The answer? Recall the query was  Z.holding(b,Z) and we wanted to extract the answer, which was the binding for the situation term Z that would make “holding b” true. Our resolution refutation gave us the answer ans(do(pickup(b),s 0 )) This says that a situation in which you are holding b is described by the situation term “do(pickup(b),s 0 )” The plan to achieve “holding(b)” from situation s 0 can be extracted from this situation term. In this case it is simply one action: pickup(b) which must be executed in s 0

38. 38 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Two types of reasoning. Two common types of queries : 1.Temporal Projection Predicting the effects of a given sequence of actions E.g., on(b,c, do(stack(b,c), do(pickup(b), s 0 )) ) 2.Plan Synthesis Computing a sequence of actions that achieve a goal conditions E.g.,  S. on(b,c,S)  on(c,a,S)

39. 39 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 The Frame Problem Unfortunately, logical reasoning won’t immediately yield the answer to these kinds of questions. e.g., query: on(c,a,do(pickup(b),s 0 ))? is c still on a after we pickup b? Intuitively it should be Can logical reasoning reach this conclusion given the representation of actions that we have proposed thus far?

40. 40 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 The Frame Problem 1. clear(c,s 0 ) 2. on(c,a,s 0 ) 3. clear(b,s 0 ) 4. ontable(a,s 0 ) 5. ontable(b,s 0 ) 6. handempty(s 0 ) 8. (  ontable(Y,S),  clear(Y,S),  handempty(S), holding(Y,do(pickup(Y),S)) 9. (  ontable(Y,S),  clear(Y,S),  handempty(S),  handempty(do(pickup(X),S))) 10. (  ontable(Y,S),  clear(Y,S),  handempty(S),  ontable(Y,do(pickup(Y,S))) 11. (  ontable(Y,S),  clear(Y,S),  handempty(S),  clear(Y,do(pickup(Y,S))) 12.  on(c,a,do(pickup(b),s 0 )) {QUERY) Nothing can resolve with 12!

41. 41 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Logical Consequence Remember that resolution only computes logical consequences. We stated the effects of pickup(b), but did not state that it doesn’t affect on(c,a). Hence there are models in which on(c,a) no longer holds after pickup(b) (as well as models where it does hold). The problem is that representing the non-effects of actions is very tedious and in general is not possible. Think of all of the things that pickup(b) does not affect!

42. 42 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 The Frame Problem Finding an effective way of specifying the non- effects of actions, without having to explicitly write them all down is the frame problem. Good solutions have been proposed, and the situation calculus has been a powerful way of dealing with dynamic worlds. e.g., successor state axioms

43. 43 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Computation Problems Although the situation calculus is a very powerful representation. It is not always efficient enough to use to compute sequences of actions (planning). Situation calculus not ideal for automated plan generation (But very good for studying the formal properties of planning and for modelling more complex issues in planning) Next we will study some less expressive representations that support more efficient planning.

44. 44 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 From Situation Calculus to STRIPS We will no longer talk about “situations” but rather about “states”

45. 45 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Simplifying the Planning Problem We simplify the planning problem as follows: Assume complete information about the initial state through the closed-world assumption (CWA). Assume a finite domain of objects Assume that action effects are restricted to making (conjunctions of) atomic formulae true or false. No conditional effects, no disjunctive effects, etc. Assume action preconditions are restricted to conjunctions of ground atoms. Perform Classical Planning. No incomplete or uncertain knowledge.

46. 46 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Closed World Assumption (CWA) The knowledge base used to represent a state of the world is a list of positive ground atomic facts. (Like a database.) Use the database “Closed World Assumption” in our knowledge representation and reasoning CWA is the assumption that a)if a ground atomic fact is not in our list of “known” facts, its negation must be true. b)the constants mentioned in KB are all the domain objects. This gives us complete information about the state of the system.

47. 47 Querying a Closed World KB CWA treats a knowledge base like a database: if employed(John, CIBC) is not in the database, we conclude that  employed(John, CIBC) is true. With the CWA, we can evaluate the truth or falsity of arbitrarily complex first-order formulas. The knowledge base acts as a model This process is very similar to query evaluation in databases. Just as databases are useful, so are CW KB’s. “CW KB” or “CW-KB” = Closed-world knowledge base “CWA” = Closed World Assumption

48. 48 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 CWA Example KB = {handempty clear(c), clear(b), on(c,a), ontable(a), ontable(b)} Arbitrary queries: 1.clear(c)  clear(b)? 2.  on(b,c)? 3.on(a,c)  on(b,c)? 4.  X.on(X,c)? (D = {a,b,c}) 5.  X.ontable(X)  X = a  X = b? C AB

49. 49 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Querying A Closed-World KB Query(F, KB) /*return whether or not KB |= F */ if F is atomic return(F  KB)

50. 50 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Querying A CW KB if F = F 1  F 2 return(Query(F 1 ) && Query(F 2 )) if F = F 1  F 2 return(Query(F 1 ) || Query(F 2 )) if F =  F 1 return(! Query(F 1 )) if F = F 1  F 2 return(!Query(F 1 ) || Query(F 2 ))

51. 51 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Querying A CW KB if F =  X.F 1 for each constant c  KB if (Query(F 1 {X=c})) return(true) return(false). if F =  X.F 1 for each constant c  KB if (!Query(F 1 {X=c})) return(false) return(true).

52. 52 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Querying A CW KB Guarded quantification (for efficiency). if F =  X.F 1 for each constant c  KB if (!Query(F 1 {X=c})) return(false) return(true). E.g., consider checking  X. apple(x) → sweet(x) we already know that the formula is true for all “non-apples”

53. 53 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Querying A CW KB Guarded quantification (for efficiency).  X:[p(X)] F 1  X: p(X) → F 1 for each constant c s.t. p(c) if (!Query(F 1 {X=c})) return(false) return(true).  X:[p(X)]F 1  X: p(X)  F 1 for each constant c s.t. p(c) if (Query(F 1 {X=c})) return(true) return(false).

54. 54 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 STRIPS representation. STRIPS (Stanford Research Institute Problem Solver.) is a way of representing actions. Actions are modeled as ways of modifying the world. since the world is represented as a CW-KB, a STRIPS action represents a way of updating the CW-KB. Now actions yield new KB’s, describing the new world— the world as it is once the action has been executed.

55. Sequence of Worlds (Databases)

56. 56 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Sequences of Worlds In the situation calculus we could Have incomplete information (specified by a FOL formula) Refer to two different situations at the same time in one logical sentence, e.g.,: on(a,b,s 0 )   on(a,b,s 1 ) In STRIPS, we have Complete information (specified by a CW-KB ) Information about the state of the world at different times would have to be captured by two separate database. This is much like in search where each state was represented in a separate data structure.

57. 57 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 STRIPS Actions STRIPS represents actions using 3 lists. 1. A list of action preconditions. 2. A list of action add effects. 3. A list of action delete effects. These lists contain variables, so that we can represent a whole class of actions with one specification. Each ground instantiation of the variables yields a specific action.

58. 58 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 STRIPS Actions: Example pickup(X): Pre: {handempty, clear(X), ontable(X)} Adds: {holding(X)} Dels: {handempty, clear(X), ontable(X)} “pickup(X)” is called a STRIPS operator. a particular instance (e.g.“pickup(a)”) is called an action. These are sets of atomic formulae interpretted as a conjunctions of literals. (in contrast to our previous clause notation in KR)!

59. 59 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Operation of a STRIPS action. For a particular STRIPS action (ground instance) to be applicable to a state (a CW-KB) every fact in its precondition list must be true in KB. This amounts to testing membership since we have only atomic facts in the precondition list. If the action is applicable, the new state is generated by removing all facts in Dels from KB, then adding all facts in Adds to KB.

60. 60 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Operation of a Strips Action: Example KB = {handempty clear(c), clear(b), on(c,a), ontable(a), ontable(b)} C AB C A B pickup(b) KB = { holding(b), clear(c), on(c,a), ontable(a)} pre = {handmpty, clear(b), ontable(b)} add = {holding(b)} del = {handmpty, clear(b), ontable(b)}

61. 61 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 STRIPS Blocks World Operators. pickup(X) (from the table) putdown(X) (on the table) unstack(X,Y) (pickup from a stack of blocks) stack(X,Y) (putdown on a block) Blocks World Assumptions: The table has infinite space, so it is always clear. If we stack something on the table (Y=Table), we cannot delete clear(Table), but if Y is an ordinary block we must delete clear(Y).

62. 62 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 STRIPS Blocks World Operators. pickup(X) Pre: {clear(X), ontable(X), handempty} Add: {holding(X)} Del: {clear(X), ontable(X), handempty} putdown(X) Pre: {holding(X)} Add: {clear(X), ontable(X), handempty} Del: {holding(X)} unstack(X,Y) Pre: {clear(X), on(X,Y), handempty} Add: {holding(X), clear(Y)} Del: {clear(X), on(X,Y), handempty} stack(X,Y) Pre: {holding(X),clear(Y)} Add: {on(X,Y), handempty, clear(X)} Del: {holding(X),clear(Y)}

63. 63 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 STRIPS has no Conditional Effects Unlike ADL (Action Description Language) which we’ll see later. STRIPS has no conditional effects Since STRIPS has no conditional effects, we must sometimes utilize extra actions: one for each type of condition. We embed the condition in the precondition, and then alter the effects accordingly. putdown(X) Pre: {holding(X)} Add: {clear(X), ontable(X), handempty} Del: {holding(X)} stack(X,Y) Pre: {holding(X),clear(Y)} Add: {on(X,Y), handempty, clear(X)} Del: {holding(X),clear(Y)}

64. 64 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Other Example Domains 8 Puzzle as a planning problem The Constants A constant representing each position, P1,…,P9 P1P2P3 P4P5P6 P7P8P9 A constant for each tile (and blank) B,T1, …, T8.

65. 65 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 8-Puzzle The Relations/Predicates/Properties at(T,P) tile T is at position P. 125 78 643 adjacent(P1,P2) P1 is next to P2 (i.e., we can slide the blank from P1 to P2 in one move. adjacent(P5,P2), adjacent(P5,P8), … at(T1,P1), at(T2,P2), at(T5,P3), …

66. 66 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 8-Puzzle A single operator (creating lots of ground actions) slide(T,X,Y) Pre: {at(T,X), at(B,Y), adjacent(X,Y)} Add: {at(B,X), at(T,Y)} Del: {at(T,X), at(B,Y)} 125 78 643 at(T1,P1), at(T5,P3), at(T8,P5), at(B,P6), …, slide(T8,P5,P6) 125 7 8 643 at(T1,P1), at(T5,P3), at(B,P5), at(T8,P6), …,

67. 67 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Elevator Control

68. 68 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Elevator Control Schindler Lifts. Central panel to enter your elevator request. Your request is scheduled and an elevator assigned to you. You can’t travel with someone going to a secure floor, emergency travel has priority, etc. Modeled as a planning problem and fielded in one of Schindler’s high end elevators.

69. 69 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Planning as a Search Problem Given a CW-KB representing the initial state, a set of STRIPS or ADL (Action Description Language) operators, that map a state to a new state and a goal condition (specified either as a conjunction of facts, or as a formula) The planning problem is to determine a sequence of actions that when applied to the initial CW-KB yield an updated CW-KB which satisfies the goal. This is known as the classical planning task.

70. 70 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Planning As Search This is a search problem in which our state space representation is a CW-KB. The initial CW-KB is the initial state. The actions are operators mapping a state (a CW-KB) to a new state (an updated CW-KB). The goal is satisfied by any state (CW-KB) that satisfies the goal. Typically the goal is a conjunction of primitive facts, so we just need to check if all of the facts in the goal are contained in the CW-KB.

71. 71 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example. C AB move(b,c) C A B move(c,b) C AB move(c,table) C AB move(a,b) B AC

72. 72 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Problems Search tree is generally quite large randomly reconfiguring 9 blocks takes thousands of CPU seconds. But: the representation suggests some structure. Each action only affects a small set of facts, actions depend on each other via their preconditions. Planning algorithms are designed to take advantage of the fact that the representation makes the “locality” of action changes explicit.

73. 73 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Planning We will look at one technique: Relaxed Plan heuristics used with heuristic search. The heuristics are domain independent. As such they are part of a class of so-called domain-independent heuristic search for planning

74. 74 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Planning as Search The material that follows is *not* on the final exam. *Except*, you should know the difference between STRIPS and ADL, which is elaborated upon at the end of the slide deck.

75. 75 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Reachability Analysis. The idea is to consider what happens if we ignore the delete lists of actions. This yields a “relaxed problem” that can produce a useful heuristic estimate. Recall: we can obtain heuristics by solving a relaxed version of the 8-tile problem in which one of the restrictions move to adjacent field only move to blank field only is relaxed. It’s Similar here!

76. 76 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Reachability Analysis In the relaxed problem actions add new facts, but never delete facts. In particular, successor states always have more “facts” true so eventually by just adding more facts we can add all of the facts in any conjunctive goal or we hit a dead end where no more facts can be added. In this setting we can do reachability analysis, which is much simpler than searching for a solution. This involves considering the set of actions that can be applied rather than considering individual actions one at a time.

77. 77 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Reachability We start with the initial state S 0. We alternate between state and action layers. We find all actions whose preconditions are contained in S 0. These actions comprise the first action layer A 0. The next state layer contains: S 0 U all states added by the actions in A 0. In general: A i … set of actions whose preconditions are in S i. S i = S i-1 U the add lists of all of the actions in A i

78. 78 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 STRIPS Blocks World Operators – Relaxed! pickup(X) Pre: {handempty, ontable(X), clear(X)} Add: {holding(X)} Del: {handempty, ontable(X), clear(X)} putdown(X) Pre: {holding(X)} Add: {handempty, ontable(X), clear(X)} Del: {holding(X)} unstack(X,Y) Pre: {handempty, clear(X), on(X,Y)} Add: {holding(X), clear(Y)} Del: {handempty, clear(X), on(X,Y)} stack(X,Y) Pre: {holding(X),clear(Y)} Add: {handempty, clear(X), on(X,Y)} Del: {holding(X),clear(Y)}

79. 79 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example a b cd on(a,b), on(b,c), ontable(c), ontable(d), clear(a), clear(d), handempty unstack(a,b) pickup(d) a b cd S0S0 A0A0 on(a,b), on(b,c), ontable(c), ontable(d), clear(a), handempty, clear(d), holding(a), clear(b), holding(d) a d this is not a state as some of these facts cannot be true at the same time! S1S1

80. 80 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example on(a,b), on(b,c), ontable(c), ontable(d), clear(a), clear(d), handempty, holding(a), clear(b), holding(d) S1S1 unstack(a,b) pickup(d) putdown(a), putdown(d), stack(a,b), stack(a,a), stack(d,a), stack(d,b), stack(d,d), unstack(b,c) … A1A1 a b cd a dA0A0 from Impossible, but we don’t know because we ignore dels.

81. 81 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example a b cd on(a,b), on(b,c), ontable(c), ontable(d), clear(a), clear(d), handempty unstack(a,b) pickup(d) a b cd S0S0 A0A0 on(a,b), on(b,c), ontable(c), ontable(d), clear(a), handempty, clear(d), holding(a), clear(b), holding(d) a d this is not a state! S1S1

82. 82 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example on(a,b), on(b,c), ontable(c), ontable(d), clear(a), clear(d), handempty, holding(a), clear(b), holding(d) S1S1 putdown(a), putdown(d), stack(a,b), stack(a,a), stack(d,b), stack(d,a), pickup(d), … unstack(b,c) … A1A1

83. 83 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Reachabilty We continue until: the goal G is contained in the state layer, or until the state layer no longer changes (reached fix point). Intuitively: the actions at level A i are the actions that could be executed at the i-th step of some plan, and the facts in level S i are the facts that could be made true within a plan of length i. Some of the actions/facts have this property. But not all!

84. 84 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Reachability a bc on(a,b), ontable(c), ontable(b), clear(a), clear(c), handempty unstack(a,b) pickup(c) S0S0 A0A0 on(a,b), ontable(c), ontable(b), clear(a), clear(c), handempty, holding(a), clear(b), holding(c) S1S1 stack(c,b) … A1A1 on(c,b), … but stack(c,b) cannot be executed after one step to reach on(c,b) requires 4 actions a bc on(a,b), ontable(c), ontable(b), clear(a), clear(c), handempty

85. 85 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Heuristics from Reachability Analysis Grow the levels until the goal is contained in the final state level S K. If the state level stops changing and the goal is not present: The goal is unachievable under the assumption that (a) the goal is a set of positive facts, and (b) all preconditions are positive facts. Then do the following

86. 86 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Heuristics from Reachability Analysis CountActions(G,S K ): /* Compute the number of actions contained in a relaxed plan achieving the goal. */ Split G into facts in S K-1 and elements in S K only. G P contains the previously achieved (in S K-1 ) and G N contains the just achieved parts of G (only in S K ). Find a minimal set of actions A whose add effects cover G N. may contain no redundant actions, but may not be the minimum sized set (computing the minimum sized set of actions is the set cover problem and is NP-Hard) NewG := S K-1 U preconditions of A. return CountAction(NewG,S K-1 ) + size(A)

87. 87 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Heuristics from Reachability Analysis CountActions(G,S K ): /* Compute the number of actions contained in a relaxed plan achieving the goal. */ Split G into facts in S K-1 and elements in S K only. G P contains the previously achieved (in S K-1 ) and G N contains the just achieved parts of G (only in S K ). Find a minimal set of actions A whose add effects cover G N. may contain no redundant actions, but may not be the minimum sized set (computing the minimum sized set of actions is the set cover problem and is NP-Hard) NewG := G P U preconditions of A. return CountAction(NewG,S K-1 ) + size(A)

88. 88 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example legend: [pre]act[add] S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} Goal: f 6,f 5,f 1 Actions: [f 1 ]a 1 [f 4 ] [f 2 ]a 2 [f 5 ] [f 2,f 4,f 5 ]a 3 [f 6

89. 89 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example legend: [pre]act[add] S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } Goal: f 6,f 5,f 1 Actions: [f 1 ]a 1 [f 4 ] [f 2 ]a 2 [f 5 ] [f 2,f 4,f 5 ]a 3 [f 6

90. 90 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example legend: [pre]act[add] S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } A 1 = {[f 2,f 4,f 5 ]a 3 [f 6 ]} Goal: f 6,f 5,f 1 Actions: [f 1 ]a 1 [f 4 ] [f 2 ]a 2 [f 5 ] [f 2,f 4,f 5 ]a 3 [f 6

91. 91 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example legend: [pre]act[add] S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } A 1 = {[f 2,f 4,f 5 ]a 3 [f 6 ]} S 2 ={f 1,f 2,f 3,f 4,f 5,f 6 } Goal: f 6,f 5,f 1 Actions: [f 1 ]a 1 [f 4 ] [f 2 ]a 2 [f 5 ] [f 2,f 4,f 5 ]a 3 [f 6

92. 92 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example legend: [pre]act[add] S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } A 1 = {[f 2,f 4,f 5 ]a 3 [f 6 ]} S 2 ={f 1,f 2,f 3,f 4,f 5,f 6 } G = {f 6,f 5,f 1 }

93. 93 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example legend: [pre]act[add] S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } A 1 = {[f 2,f 4,f 5 ]a 3 [f 6 ]} S 2 ={f 1,f 2,f 3,f 4,f 5,f 6 } G = {f 6,f 5, f 1 } We split G into G P and G N : Goal: f 6,f 5,f 1 Actions: [f 1 ]a 1 [f 4 ] [f 2 ]a 2 [f 5 ] [f 2,f 4,f 5 ]a 3 [f 6

94. 94 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example legend: [pre]act[add] S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } A 1 = {[f 2,f 4,f 5 ]a 3 [f 6 ]} S 2 ={f 1,f 2,f 3,f 4,f 5,f 6 } G = {f 6,f 5,f 1 } G N = {f 6 } (newly achieved) G p = {f 5, f 1 } (achieved before)

95. 95 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example CountActs(G,S 2 ) G P ={f 5, f 1 } //already in S1 G N = {f 6 } //New in S2 A = {a 3 } //adds all in G N //the new goal: G P  Pre(A) G 1 = {f 5,f 1,f 2,f 4 } Return 1 + CountActs(G 1,S 1 ) legend: [pre]act[add] S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } A 1 = {[f 2,f 4,f 5 ]a 3 [f 6 ]} S 2 ={f 1,f 2,f 3,f 4,f 5,f 6 } G = {f 6,f 5, f 1 } We split G into G P and G N :

96. 96 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example Now, we are at level S1 S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } A 1 = {[f 2,f 4,f 5 ]a 3 [f 6 ]} S 2 ={f 1,f 2,f 3,f 4,f 5,f 6 } G 1 = {f 5,f 1,f 2,f 4 } CountActs(G 1,S 1 )

97. 97 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example Now, we are at level S1 S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } A 1 = {[f 2,f 4,f 5 ]a 3 [f 6 ]} S 2 ={f 1,f 2,f 3,f 4,f 5,f 6 } G 1 = {f 5,f 1,f 2,f 4 } We split G 1 into G P and G N : CountActs(G 1,S 1 )

98. 98 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example Now, we are at level S1 S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } A 1 = {[f 2,f 4,f 5 ]a 3 [f 6 ]} S 2 ={f 1,f 2,f 3,f 4,f 5,f 6 } G 1 = {f 5,f 1,f 2,f 4 } We split G 1 into G P and G N : G N = {f 5,f 4 } G P = {f 1,f 2 } CountActs(G 1,S 1 ) G P ={f 1,f 2 } //already in S0 G N = {f 4,f 5 } //New in S1 A = {a 1,a 2 } //adds all in G N //the new goal: G P  Pre(A) G 2 = {f 1,f 2 } Return 2 + CountActs(G 2,S 0 )

99. 99 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example Now, we are at level S1 S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } A 1 = {[f 2,f 4,f 5 ]a 3 [f 6 ]} S 2 ={f 1,f 2,f 3,f 4,f 5,f 6 } G 2 = {f 1,f 2 } We split G 2 into G P and G N : G N = {f 1,f 2 } G P = {} CountActs(G 2,S 0 ) G N ={f 1,f 2 } //already in S0 G P = {} //New in S1 A = {} //No actions needed. Return 0

100. 100 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example Now, we are at level S1 S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } A 1 = {[f 2,f 4,f 5 ]a 3 [f 6 ]} S 2 ={f 1,f 2,f 3,f 4,f 5,f 6 } G 2 = {f 1,f 2 } We split G 2 into G P and G N : G N = {f 1,f 2 } G P = {} CountActs(G 2,S 0 ) G N ={f 1,f 2 } //already in S0 G P = {} //New in S1 A = {} //No actions needed. Return 0 So, in total CountActs(G,S2)=1+2+0=3

101. 101 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Using the Heuristic First, build a layered structure from a state S that reaches a goal state. CountActions: counts how many actions are required in a relaxed plan. Use this as our heuristic estimate of the distance of S to the goal. This heuristic tends to work better with greedy best-first search rather than A* search That is when we ignore the cost of getting to the current state.

102. 102 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Admissibility A minimum sized plan in the delete relaxed problem would be a lower bound on the optimal size of a plan in the real problem. And could serve as an admissible heuristic for A*. However, CountActions does NOT compute the length of the optimal relaxed plan. The choice of which action set to use to achieve G P (“just achieved part of G”) is not necessarily optimal – it is minimal, but not necessary a minimum. Furthermore even if we picked a true minimum set A at each stage of CountActions, we might not obtain a minimum set of actions for the entire plan---the set A picked at each state influences what set can be used at the next stage!

103. 103 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Admissibility It is NP-Hard to compute the optimal length plan even in the relaxed plan space. So CountActions cannot be made into an admissible heuristic without making it much harder to compute. Empirically, refinements of CountActions performs very well on a number of sample planning domains.

104. 104 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Beyond STRIPS STRIPS operators are not very expressive and as a consequence not as compact as they might be. ADL (Action Description Language) extends the expressivity of STRIPS

105. 105 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 ADL Operators. ADL operators add a number of features to STRIPS. 1. Their preconditions can be arbitrary formulas, not just a conjunction of facts. 2. They can have conditional and universal effects. 3. Open world assumption: 1. States can have negative literals 2. The effect (P ∧ ¬ Q) means add P and ¬ Q but delete ¬ P and Q. But ADL operators must still specify atomic changes to the knowledge base (add or delete ground atomic facts).

106. 106 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 ADL Operators Examples. move(X,Y,Z) Pre: on(X,Y)  clear(Z) Effs: ADD[on(X,Z)] DEL[on(X,Y)] Z  table  DEL[clear(Z)] Y  table  ADD[clear(Y)]

107. 107 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 ADL Operators, example KB = { clear(c), clear(b), on(c,a), on(a,table), on(b,table)} C AB C AB move(c,a,b) KB = {on(c,b) clear(c), clear(a) on(a,table), on(b,table)} move(c,a,b) Pre: on(c,a)  clear(b) Effs: ADD[on(c,b)] DEL[on(c,a)] b  table  DEL[clear(b)] a  table  ADD[clear(a)]

108. 108 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 ADL Operators Examples. clearTable() Pre: Effs:  X. on(X,table)  DEL[on(X,table)]

109. 109 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 ADL Operators. 1. Arbitrary formulas as preconditions.  in a CW-KB we can evaluate whether or not the preconditions hold for an arbitrary precondition. 2. They can have conditional and universal effects.  Similarly we can evaluate the condition to see if the effect should be applied, and find all bindings for which it should be applied. Specify atomic changes to the knowledge base. CW-KB can be updated just as before.

110. 110 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example CountActs(G,S 2 ) G P ={f 5, f 1 } //already in S1 G N = {f 6 } //New in S2 A = {a 3 } //adds all in G N //the new goal: G P  Pre(A) G 1 = {f 5,f 1,f 2,f 4 } Return 1 + CountActs(G 1,S 1 ) legend: [pre]act[add] S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } A 1 = {[f 2,f 4,f 5 ]a 3 [f 6 ]} S 2 ={f 1,f 2,f 3,f 4,f 5,f 6 } G = {f 6,f 5, f 1 } We split G into G P and G N :

111. 111 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example Now, we are at level S1 S 0 = {f 1, f 2, f 3 } A 0 = {[f 1 ]a 1 [f 4 ], [f 2 ]a 2 [f 5 ]} S 1 = {f 1,f 2,f 3,f 4,f 5 } A 1 = {[f 2,f 4,f 5 ]a 3 [f 6 ]} S 2 ={f 1,f 2,f 3,f 4,f 5,f 6 } G 1 = {f 5,f 1,f 2,f 4 } We split G1 into G P and G N : CountActs(G 1,S 1 ) G P ={f 1,f 2 } //already in S0 G N = {f 4,f 5 } //New in S1 A = {a 1,a 2 } //adds all in G N //the new goal: G P  Pre(A) G 2 = {f 1,f 2 } Return 2 + CountActs(G 2,S 0 ) = 2 + 0 So, in total CountActs(G,S2)=1+2=3

112. 112 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Using the Heuristic 1. To use CountActions as a heuristic, we build a layered structure from a state S that reaches the goal. 2. Then we CountActions to see how many actions are required in a relaxed plan. 3. We use this count as our heuristic estimate of the distance of S to the goal. 4. This heuristic tends to work better as a best-first search, i.e., when the cost of getting to the current state is ignored.

113. 113 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Admissibility An optimal length plan in the relaxed problem (actions have no deletes) will be a lower bound on the optimal length of a plan in the real problem. However, CountActions does NOT compute the length of the optimal relaxed plan. The choice of which action set to use to achieve G P (“just achieved part of G”) is not necessarily optimal. In fact it is NP-Hard to compute the optimal length plan even in the relaxed plan space. So CountActions will not be admissible.

114. 114 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Empirically However, empirically refinements of CountActions performs very well on a number of sample planning domains.

115. 115 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 GraphPlan GraphPlan is an approach to planning that is built on ideas similar to “reachability”. But the approach is not heuristic: delete effects are not ignored. The performance is not as good as heuristic search, but GraphPlan can be generalized to other types of planning, e.g., finding optimal plans, planning with sensing, etc.

116. 116 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Graphplan Operates in two phases. Phase I. Guess a “concurrent” plan length k, then build a leveled graph with k alternating layers. Phase II. Search this leveled graph for a plan. If no plan is found, return to phase I and build a bigger leveled graph with k+1 alternating layers. The final plan, if found, consists of a sequence of sets of actions {a 1 1,a 2 1,…} → {a 1 2, a 2 2,…} → {a 1 3, a 2 3, …} → … The plan is “concurrent” in the sense that at stage I, all actions in the i-th set are executed in parallel.

117. 117 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Graphplan The leveled graph alternates between levels containing propositional nodes and levels containing action nodes. (Similar to the reachability graph). Three types of edges: precondition-edges, add-edges, and delete-edges.

118. 118 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 GraphPlan Level Graph onTable(A) onTable(B) clear(A) clear(B) handempty Initial state Only the propositions true in the initial state. Next state All propositions added by actions in previous level Possible actions Only the actions whose preconditions are in the previous level. Also have no-ops for capturing non- changes. holding(A) holding(B) onTable(A) onTable(B) clear(A) clear(B) handempty pickup(A) pickup(B) no-op(onTable(A)) no-op(onTable(B)) no-op(Clear(A)) no-op(Clear(B)) no-op(handempty) Precondition Delete Add A0A0 S0S0 S1S1

119. 119 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 GraphPlan Level Graph onTable(A) onTable(B) clear(A) clear(B) handempty Level S 0 contains all facts true in the initial state. Level A 0 contains all actions whose preconditions are true in S 0. Included in the set of actions are no-ops. One no-op for every ground atomic fact. The precondition of the no-op is its fact, its add effect is its fact. … Level S i contains all facts that are added by actions at level A i-1 Level A i contains all actions whose preconditions are true in S i holding(A) holding(B) onTable(A) onTable(B) clear(A) clear(B) handempty pickup(A) pickup(B) no-op(onTable(A)) no-op(onTable(B)) no-op(Clear(A)) no-op(Clear(B)) no-op(handempty) A0A0 S0S0 S1S1

120. 120 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 GraphPlan Mutexes. onTable(A) onTable(B) clear(A) clear(B) handempty In addition to the facts/actions. GraphPlan also computes and adds mutexes to the graph. Mutexes are edges between two labels, indicating that these two labels cannot be true at the same time. Mutexes are added as we construct each layer, and in fact alter the set of labels the eventually appear in a layer. holding(A) holding(B) onTable(A) onTable(B) clear(A) clear(B) handempty pickup(A) pickup(B) no-op(onTable(A)) no-op(onTable(B)) no-op(Clear(A)) no-op(Clear(B)) no-op(handempty)

121. 121 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Mutexes A mutex between two actions a 1 and a 2 in the same action layer A i, means that a 1 and a 2 cannot be executed simultaneously (in parallel) at the i th step of a concurrent plan. A mutex between two facts F 1 and F 2 in the same state layer S i, means that F 1 and F 2 cannot be simultaneously true after i stages of parallel action execution.

122. 122 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Mutexes It is not possible to compute all mutexes. This is as hard as solving the planning problem, and we want to perform mutex computation as a precursor to solving a planning instance. However, we can quickly compute a subset of the set of all mutexes. Although incomplete these mutexes are still very useful. This is what GraphPlan does.

123. 123 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Mutexes onTable(A) onTable(B) clear(A) clear(B) handempty pickup(A) pickup(B) no-op(onTable(A)) no-op(onTable(B)) no-op(Clear(A)) no-op(Clear(B)) no-op(handempty) holding(A) holding(B) onTable(A) onTable(B) clear(A) clear(B) handempty Two actions are mutex if either action deletes a precondition or add effect of another. Note no-ops participate in mutexes. Intuitively these actions have to be sequenced—they can’t be executed in parallel

124. 124 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Mutexes Two propositions p and q are mutex if all actions adding p are mutex of all actions adding q. Must look at all pairs of actions that add p and q. Intuitively, can’t achieve p and q together at this stage because we can’t concurrently execute achieving actions for them at the previous stage. onTable(A) onTable(B) clear(A) clear(B) handempty pickup(A) pickup(B) no-op(onTable(A)) no-op(onTable(B)) no-op(Clear(A)) no-op(Clear(B)) no-op(handempty) holding(A) holding(B) onTable(A) onTable(B) clear(A) clear(B) handempty

125. 125 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Mutexes Two actions are mutex if two of their preconditions are mutex. Intuitively, we can’t execute these two actions concurrently at this stage because their preconditions can’t simultaneously hold at the previous stage. putdown(A) putdown(B) no-op(onTable(A)) no-op(onTable(B)) no-op(Clear(A)) no-op(Clear(B)) no-op(handempty) holding(A) holding(B) onTable(A) onTable(B) clear(A) clear(B) handempty

126. 126 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 How Mutexes affect the level graph. 1. Two actions are mutex if either action deletes a precondition or add effect of another 2. Two propositions p and q are mutex if all actions adding p are mutex of all actions adding q 3. Two actions are mutex if two of their preconditions are mutex We compute mutexes as we add levels. S 0 is set of facts true in initial state. (Contains no mutexes). A 0 is set of actions whose preconditions are true in S 0. Mark as mutex any action pair where one deletes a precondition or add effect of the other. S 1 is set of facts added by actions at level A 0. Mark as mutex any pair of facts p and q if all actions adding p are mutex with all actions adding q. A 1 is set of actions whose preconditions are true and not mutex at S 1. Mark as mutex any action pair with preconditions that are mutex in S 1, or where one deleted a precondition or add effect of the other.

127. 127 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 How Mutexes affect the level graph. 1. Two actions are mutex if either action deletes a precondition or add effect of another 2. Two propositions p and q are mutex if all actions adding p are mutex of all actions adding q 3. Two actions are mutex if two of their preconditions are mutex … S i is set of facts added by actions in level A i-1 Mark as mutex all facts satisfying 2 (where we look at the action mutexes of A i-1 is set of facts true in initial state. (Contains no mutexes). A i is set of actions whose preconditions are true and non-mutex at S i. Mark as mutex any action pair satisfying 1 or 2.

128. 128 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 How Mutexes affect the level graph. Hence, mutexes will prune actions and facts from levels of the graph. They also record useful information about impossible combinations.

129. 129 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example on(a,b), on(b,c), ontable(c), ontable(b), clear(a), clear(c), handempty unstack(a,b) pickup(c) on(a,b), on(b,c), ontable(c), ontable(d), clear(a), clear(c), handempty, holding(a), clear(b), holding(c) precondition add effect del effect pickup deletes handempty, one of unstack(a,b)’s preconditions. NoOp-on(a,b) unstack(a,b) deletes the add effect of NoOp- on(a,b), so these actions are mutex as well

130. 130 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example on(a,b), on(b,c), ontable(c), ontable(b), clear(a), clear(c), handempty unstack(a,b) pickup(c) on(a,b), on(b,c), ontable(c), ontable(d), clear(a), clear(c), handempty, holding(a), clear(b), holding(c) precondition add effect del effect NoOp-on(a,b) unstack(a,b) is the only action that adds clear(b), and this is mutex with pickup(c), which is the only way of adding holding(c).

131. 131 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example on(a,b), on(b,c), ontable(c), ontable(b), clear(a), clear(c), handempty unstack(a,b) pickup(c) on(a,b), on(b,c), ontable(c), ontable(d), clear(a), clear(c), handempty, holding(a), clear(b), holding(c) precondition add effect del effect NoOp-on(a,b) These two are mutex for the same reason.

132. 132 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Example on(a,b), on(b,c), ontable(c), ontable(b), clear(a), clear(c), handempty unstack(a,b) pickup(c) on(a,b), on(b,c), ontable(c), ontable(d), clear(a), clear(c), handempty, holding(a), clear(b), holding(c) precondition add effect del effect NoOp-on(a,b) unstack(a,b) is also mutex with the NoOp-on(a,b). So these two facts are mutex (NoOp is the only way on(a,b) can be created).

133. 133 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Phase II. Searching the Graphplan on(A,B) on(B,C) onTable(c) onTable(B) clear(A) clear(B) handempty K Build the graph to level k, such that every member of the goal is present at level k, and no two are mutex. Why?

134. 134 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Searching the Graphplan on(A,B) on(B,C) onTable(c) onTable(B) clear(A) clear(B) handempty stack(A,B) stack(B,C) no-op(on(B,C))... Find a non-mutex collection of actions that add all of the facts in the goal. K

135. 135 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Searching the Graphplan on(A,B) on(B,C) onTable(c) onTable(B) clear(A) clear(B) handempty K stack(A,B) stack(B,C) no-op(on(B,C))... holding(A) clear(B) on(B,C) K-1 The preconditions of these actions at level K-1 become the new goal at level K-1. Recursively try to solve this new goal. If this fails, backtrack and try a different set of actions for solving the goal at level k.

136. 136 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Phase II-Search Solve(G,K) forall sets of actions A={a i } such that no pair (a i, a j )  A is mutex the actions in A suffice to add all facts in G Let P = union of preconditions of actions in A If Solve(P,K-1) Report PLAN FOUND At end of forall. Exhausted all possible action sets A Report NOPLAN This is a depth first search.

137. 137 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Graph Plan Algorithm Phase I. build leveled graph. Phase II. Search leveled graph. Phase I: While last state level does not contain all goal facts with no pair being mutex add new state/action level to graph if last state/Action level = previous state/action level (including all MUTEXES) graph has leveled off report NO PLAN. Phase II: Starting at last state level search backwards in graph for plan. Try all ways of moving goal back to initial state. If successful report PLAN FOUND. Else goto Phase I.

138. 138 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Dinner Date Example Initial State {dirty, cleanHands, quiet} Goal {dinner, present, clean} Actions Cook: Pre: {cleanHands} Add: {dinner} Wrap:Pre: {quiet} Add: {present} Tidy:Pre: {} Add: {clean} Del: {cleanHands, dirty} Vac:Pre: {} Add: {clean} Del: {quite, dirty}

139. 139 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Dinner example: rule1 action mutex Actions (including all No-OP actions) Cook: Pre: {H}Add: {D} Del: {} Wrap:Pre: {Q} Add: {P} Del: {} Tidy:Pre: {} Add: {C} Del: {H,R} Vac:Pre: {} Add: {C} Del: {Q, R} NO(C):Pre: {C} Add: {C} Del: {} NO(D):Pre: {D} Add: {D} Del: {} NO(H):Pre: {H} Add: {H} Del: {} NO(P):Pre: {P} Add: {P} Del: {} NO(Q):Pre: {Q} Add: {Q} Del: {} NO(R):Pre: {R} Add: {R} Del: {} Look at those with non-empty Del, and find others that have these Del in their Pre or Add: So, Rule 1 action mutex are as follows (these are fixed): (Tidy,Cook), (Tidy, NO(H)), (Tidy, NO(R)), (Vac, Wrap), (Vac,NO(Q)), (Vac, NO(R)) Rule 3 action mutex depend on state layer and you have to build the graph. Legend: NO:No-Op, C:clean, D: Dinner, H: cleanHands, P:Present, Q:Quiet, R: diRty

140. 140 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Dinner Example: Legend: ● Arrows: Blue: pre, Green: add, Red: Del, Black: Mutex ● D : Dinner, C :clean, H : cleanHands, Q :quiet, P :Present, R : diRty ● Init={R,H,Q} Goal={D,P,C} RHQRHQ DPcRHQDPcRHQ Cook Wrap Tidy Vac NO(R) NO(H) NO(Q) S0 A0 S1 Note:  At layer S1 all goals are present and no pair forms a mutex  So, go to phase II and search the graph:  i.e. Find a set of non-mutex actions that adds all goals {D,P,C}: x {Cook, Wrap, Tidy} mutex Tidy&Cook x {Cook, Wrap, Vac} mutex Vac&Wrap  No such set exists, nothing to backtrack, so goto phase I and add one more action and state layers

141. 141 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Dinner Example: ● Arrows: Blue: pre, Green: add, Red: Del, Black: Mutex ● D : Dinner, C :clean, H : cleanHands, Q :quiet, P :Present, R : diRty ● Init={R,H,Q} Goal={D,P,C} RHQRHQ DPcRHQDPcRHQ Cook Wrap Tidy Vac NO(R) NO(H) NO(Q) Cook Wrap Tidy Vac NO(D) NO(P) NO(C) NO(R) NO(H) NO(Q) S0 A0 S1 A1 S2 DPcRHQDPcRHQ At layer A1, first draw rule1 action mutex, then find rule3 action mutex (for this only look at mutex fact at level S1). At S2, apply rule 2 for fact mutex. At layer S2 all goals are present and no pair forms a mutex, so PhaseII phase II: Find a set of non- mutex actions that adds all goals {D,P,C}: x {Cook, Wrap, Tidy} x {Cook, Wrap, Vac} {Cook, Wrap, NO(C)} NewG={H,Q,C} @ S1  Cannot find any non-mutex action set in A0  Backtrack to S2, try another action set {Cook, NO(P), Vac}

142. 142 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Dinner Example: RHQRHQ DPcRHQDPcRHQ Cook Wrap Tidy Vac NO(R) NO(H) NO(Q) Cook Wrap Tidy Vac NO(D) NO(P) NO(C) NO(R) NO(H) NO(Q) S0 A0 S1 A1 S2 DPcRHQDPcRHQ so try {Cook, NO(P), Vac} NewG={H,P} @ S1 Find a nonmutex set of act in A0 to get NewG:  {NO(H),Wrap}  NewG’={H,Q} @ S0  Done!  So, the actions are  Wrap, {Cook, Vac}: I.e., sequentially either  Wrap,Cook,Vac  Wrap,Vac,Cook  Note that we could still backtrack to S2, try remaining action sets! ● Arrows: Blue: pre, Green: add, Red: Del, Black: Mutex ● D : Dinner, C :clean, H : cleanHands, Q :quiet, P :Present, R : diRty ● Init={R,H,Q} Goal={D,P,C}

143. 143 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 Beyond STRIPS STRIPS operators are not very expressive and as a consequence not as compact as they might be. ADL (Action Description Language) extends the expressivity of STRIPS

144. 144 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 ADL Operators. ADL operators add a number of features to STRIPS. 1. Their preconditions can be arbitrary formulas, not just a conjunction of facts. 2. They can have conditional and universal effects. 3. Open world assumption: 1. States can have negative literals 2. The effect (P ∧ ¬ Q) means add P and ¬ Q but delete ¬ P and Q. But they must still specify atomic changes to the knowledge base (add or delete ground atomic facts).

145. 145 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 ADL Operators Examples. move(X,Y,Z) Pre: on(X,Y)  clear(Z) Effs: ADD[on(X,Z)] DEL[on(X,Y)] Z  table  DEL[clear(Z)] Y  table  ADD[clear(Y)]

146. 146 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 ADL Operators, example KB = { clear(c), clear(b), on(c,a), on(a,table), on(b,table)} C AB C AB move(c,a,b) KB = {on(c,b) clear(c), clear(a) on(a,table), on(b,table)} move(c,a,b) Pre: on(c,a)  clear(b) Effs: ADD[on(c,b)] DEL[on(c,a)] b  table  DEL[clear(b)] a  table  ADD[clear(a)]

147. 147 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 ADL Operators Examples. clearTable() Pre: Effs:  X. on(X,table)  DEL[on(X,table)]

148. 148 Sheila McIlraith, CSC384, University of Toronto, Winter 2016 ADL Operators. 1. Arbitrary formulas as preconditions.  in a CW-KB we can evaluate whether or not the preconditions hold for an arbitrary precondition. 2. They can have conditional and universal effects.  Similarly we can evaluate the condition to see if the effect should be applied, and find all bindings for which it should be applied. Specify atomic changes to the knowledge base. CW-KB can be updated just as before.