1. PLANNING Partial order regression planning Temporal representation 1 Deductive planning in Logic Temporal representation 2

2. 2  A second class of planners: based on A.R. The many motivations...  Provide an additional (more complex) illustration of AR.  Show important trade-offs in Knowledge Representation.  Introduce Temporal Representation (Situation Calculus).

3. Temporal representation 1 From STRIPS to a simple situation calculus

4. 4 Towards planning as Automated Reasoning  Formulate in a logic theory T:  The initial situation  For each operator:  its preconditions  the relation between the previous situation and the next one.  Prove the logical consequence F: “ There exists a sequence of operations, such that if they are applied to the initial situation, the resulting situation satisfies some given properties”. “ There exists a sequence of operations, such that if they are applied to the initial situation, the resulting situation satisfies some given properties”.  Extract the plan from the proof.

5. 5 What the STRIPS represen- tation lacks to do this: do NOT describe WHEN (at which time point or in which situation) these properties hold. 1) State representations, like on(A,B) clear(A) on(B,Table)  Thus: if the goal is to have on(A,Table), this is inconsistent with the current property on(A,B)  Remember: logic is monotonic … no consequences get removed! 2) Add and Delete list are procedural.  These do not declaratively describe the relation between the previous situation and the next.

6. 6 Introducing temporal information: 1) What kind of temporal identifications to use? Time points Situation names Extra argument Meta-predicate 2) How to add this information? on(A,B,T) on(A,B,S) on(A,B,S) holds(on(A,B),T) holds(on(A,B),S) holds(on(A,B),S)  Two dimensions of options:

7. 7 A tiny example problem: A B B A Initial situation Goal situation on(B,A,S 0 ) on(A,Table,S 0 ) clear(B,S 0 )  s on(B,Table,s)

8. 8 Situation calculus, extra argument.  Examples of situations:  moveATableB(S 0 )  moveCBA(moveABTable(S 0 ))  Operations are described by functors:  Examples:  move A from B to C: moveABC(x)  move A from Table to B: moveATableB(x)  Situation descriptions using ‘terms’:  The initial situation is described by a constant: S 0  A situation obtained from a situation s by applying an operator move. from. to. as the term: move…(s)

9. 9 The situations in this world S0S0 A B moveBATable( moveBATable(S 0 ) AB moveATableB(moveBATable( moveATableB(moveBATable(S 0 )) moveBTableA(moveBATable( moveBTableA(moveBATable(S 0 )) A B A B

10. 10 The plan ? A B B A Initial situation Goal situation  s on(B,Table,s) Finding a proof = finding a substitution for s of the form: THUS: the value for s IS THE PLAN. s / moveBATable(S 0 )

11. 11 Completing situations  Note: also valid are: ~on(A,B,S 0 ) ~on(B,Table,S 0 ) ~clear(A,S 0 )  In STRIPS (and in LP) these are implicitly represented.  In FOL, we could instead add: PS: also completes all later situations  x,y,z,s on(x,y,s)  ~y=Table  ~x=z  ~on(z,y,s) y xz  x,y,z,s on(x,y,s)  ~y=z  ~on(x,z,s) y x z  x,y,s on(x,y,s)  ~y=Table  ~clear(y,s) y xNotclear

12. 12 Relating different situations (1)  s on(B,A,s)  clear(B,s)  on(B,Table,moveBATable(s))  clear(A,moveBATable(s))  clear(A,moveBATable(s))  For each operator we need to represent the relation between previous and next situation if on(B,A) clear(B) clear(B) add on(B,Table) clear(A) clear(A) delete on(B,A) move B from A to Table  Example:  Note: delete on(B,A) is implicitly represented because of add on(B,Table) and the completing formulae.

13. 13 Relating different situations: Frame axioms (1):  STRIPS operators only express which things change ! if on(B,A) clear(B) clear(B) add clear(A) on(B,Table) on(B,Table) delete on(B,A) move B from A to Table on(A,Table) initially on(A,Table) still holds  Example:  If on(A,Table,s) is true and move B from A to Table is applicable, then on(A,Table,moveBATable(s)) is true. FRAME AXIOMS  We need to express this explicitly !

14. 14 Relating different situations: Frame axioms (2):  Example: if on(B,A) clear(B) clear(B) add clear(A) on(B,Table) on(B,Table) delete on(B,A) move B from A to Table  Anything that was on something else before, except for B on A, still is:  x,y,s on(B,A,s)  clear(B,s)  on(x,y,s)  ~(x=B  y=A)  on(x,y,moveBATable(s))  on(x,y,moveBATable(s))  Anything that was clear before, still is:  x,s on(B,A,s)  clear(B,s)  clear(x,s)  clear(x,moveBATable(s))  clear(x,moveBATable(s))

15. 15 Discussion:  This is one of the most simple options.  It requires:  very many axioms and frame axioms  for EACH separate operation  Generalizing to operators patterns in not easy.  But: this is sufficient to illustrate planning as deduction.  We discuss more refined representations later.

16. Planning as deduction Automated reasoning applied to a simple situation calculus representation

17. 17 The theory and goal on(B,A,S 0 ) on(A,Table,S 0 ) clear(B,S 0 )  s on(B,A,s)  clear(B,s)  on(B,Table,moveBATable(s))  clear(A,moveBATable(s))  clear(A,moveBATable(s)) + the consistency axioms + the Frame axioms + similar formulae for other operators T: F:  s on(B,Table,s) ~F:false  on(B,Table,s) ~F: false  on(B,Table,s)

18. 18 Normalization (T  {~F})’: on(B,A,S 0 ) on(A,Table,S 0 ) clear(B,S 0 ) on(B,Table,moveBATable(s))  on(B,A,s)  clear(B,s) clear(A,moveBATable(s))  on(B,A,s)  clear(B,s) false  on(B,Table,s) + normalization of others

19. 19 A linear top-down proof What is the plan?? The answer substitution: {s 1 /moveBATable(S 0 )} false  on(B,Table,s 1 ) on(B,Table,moveBATable(s 2 ))  on(B,A,s 2 )  clear(B,s 2 ) on(B,A,s 2 )  clear(B,s 2 ) {s 1 /moveBATable(s 2 )} false  on(B,A,s 2 )  clear(B,s 2 ) on(B,A,S 0 ) false  clear(B,S 0 ) {s2/S0}{s2/S0}{s2/S0}{s2/S0} clear(B,S 0 ) false 

20. 20 The relevance of Frame axioms?  Find a plan such that B gets on the Table AND A is still on the Table. false  on(B,Table,s)  on(A,Table,s)  ~F:  Proof: first part identical to the previous one, except that every goal gets an extra conjunct:  on(A,Table,s 1 ), where s 1 gradually get instantiated to moveBATable(S 0 ).

21. 21 A linear top-down proof extended false  on(B,Table,s 1 ) on(B,Table,moveBATable(s 2 ))  on(B,A,s 2 )  clear(B,s 2 ) on(B,A,s 2 )  clear(B,s 2 ) false  on(B,A,s 2 )  clear(B,s 2 ) {s 1 /moveBATable(s 2 )} on(B,A,S 0 ) false  clear(B,S 0 ) {s2/S0}{s2/S0}{s2/S0}{s2/S0} clear(B,S 0 ) false   on(A,Table, s 1 )  on(A,Table, s 1 )  on(A,Table, moveBATable(s 2 ))  on(A,Table, moveBATable(s 2 ))  on(A,Table, moveBATable( S 0 ))  on(A,Table, moveBATable( S 0 )) on(A,Table, moveBATable( S 0 )) on(A,Table, moveBATable( S 0 )) No longer resolves with anything, except frame axioms !

22. 22 Normalization of Frame Axiom 1:  x,y,s on(B,A,s)  clear(B,s)  on(x,y,s)  ~(x=B  y=A)  on(x,y,moveBATable(s))  x,y,s on(x,y,moveBATable(s))  ~on(B,A,s)  ~clear(B,s)  ~on(x,y,s)  (x=B  y=A)  x,y,s (on(x,y,moveBATable(s))  x=B  ~on(B,A,s)  ~clear(B,s)  ~on(x,y,s))  (on(x,y,moveBATable(s))  y=A  ~on(B,A,s)  ~clear(B,s)  ~on(x,y,s)) on(x,y,moveBATable(s))  x=B  on(B,A,s)  clear(B,s)  on(x,y,s) on(x,y,moveBATable(s))  y=A  on(B,A,s)  clear(B,s)  on(x,y,s)

23. 23 false  on(A,Table, moveBATable(S 0 )) The continuation of the proof false  ???? on(B,A,S 0 ) A=B  clear(B,S 0 )  on(A,Table,S 0 ) clear(B,S 0 ) A=B  on(A,Table,S 0 ) on(A,Table,S 0 ) A=B  on(x,y,moveBATable(s))  x=B  on(B,A,s)  clear(B,s)  on(x,y,s)  on(B,A,s)  clear(B,s)  on(x,y,s) A=B  on(B,A,S 0 )  clear(B,S 0 )  on(A,Table,S 0 ) {x/A,y/Table, s/S 0 }

24. 24 Unique Names Axioms  For each two non-unifiable syntactic objects o1 and o2: ~o1=o2.  Here: ~A=B, ~A=Table, ~B=Table false  A=B false  A=Table false  B=Table  In normalized form: A=B  false  A=B false   Thus we get:

25. 25 Can deductive planning achieve the STRIPS control?  Definitely does regression (goal directed search) !  Partial order planning?  No, not with this simple representation.  Reason:  s property(s)  Resolution steps construct:  s = move…(move…(move…(... ))) –this is a total order !!  Planning with operator patterns?  No: are not even represented here!  Need more refined temporal representations to do these.

26. Temporal representation 2 Representing operator patterns Meta-representation: situation calculus Time-point representation: event calculus

27. 27 Representing operator patterns (1)  S 0 still represents the initial situation.  Representation of a situation resulting from an operator pattern: functor move/4 s move(x,y,z,s)  Example: move x from y to z  x,y,z,s on(x,y,s)  clear(x,s)  clear(z,s)  on(x,z,move(x,y,z,s))  clear(y,move(x,y,z,s))  clear(y,move(x,y,z,s))  Effect on relating situations: example:  =WE get MUCH LESS different axioms!

28. 28 Representing operator patterns (2)  Better represent other operator patterns with other functors (to avoid confusion - and unification with wrong ones). s movefromT(x,y,s) s movetoT(x,y,s) move x from Table to y move x from y to Table  x,y,s on(x,Table,s)  clear(x,s)  clear(y,s)  on(x,y,movefromT(x,y,s))  on(x,y,movefromT(x,y,s))  Relating situations: example:  Still: 3 such axioms + 6 frame axioms needed !

29. 29 The full theory: on(B,A,S 0 ) on(A,Table,S 0 ) clear(B,S 0 )  x,y,z,s on(x,y,s)  clear(x,s)  clear(z,s)  on(x,z,move(x,y,z,s))  clear(y,move(x,y,z,s)) on(x,z,move(x,y,z,s))  clear(y,move(x,y,z,s))  x,y,s on(x,Table,s)  clear(x,s)  clear(y,s)  on(x,y,movefromT(x,y,s))  x,y,s on(x,y,s)  clear(x,s)  on(x,Table,movetoT(x,y,s))  clear(y,movetoT(x,y,s))  x,y,z,s on(x,y,s)  ~y=Table  ~x=z  ~on(z,y,s)  x,y,z,s on(x,y,s)  ~y=z  ~on(x,z,s)  x,y,s on(x,y,s)  ~y=Table  ~clear(y,s) Initial situation: Consistency axioms: New initiated properties for each action:

30. 30 The full theory (2)  x,y,z,u,s on(x,y,s)  clear(x,s)  clear(z,s)  clear(u,s)  ~(u=z)  clear(z,s)  clear(u,s)  ~(u=z)  clear(u,move(x,y,z,s))  clear(u,move(x,y,z,s))  x,y,z,u,v,s on(x,y,s)  clear(x,s)  clear(z,s)  on(u,v,s)  ~(u=x  v=y)  clear(z,s)  on(u,v,s)  ~(u=x  v=y)  on(u,v,move(x,y,z,s))  on(u,v,move(x,y,z,s))  x,y,u,v,s on(x,y,s)  clear(x,s)  on(u,v,s)  ~(u=x  v=y)  on(u,v,s)  ~(u=x  v=y)  on(u,v,movetoTable(x,y,s))  on(u,v,movetoTable(x,y,s))  x,y,u,s on(x,y,s)  clear(x,s)  x,y,u,s on(x,y,s)  clear(x,s)  clear(u,s)  clear(u,movetoTable(x,y,s))  clear(u,movetoTable(x,y,s))  x,y,u,v,s on(x,Table,s)  clear(x,s)  clear(y,s)  on(u,v,s)  ~(u=x  v=Table)  on(u,v,movefromTable(x,y,s))  ~(u=x  v=Table)  on(u,v,movefromTable(x,y,s))  x,y,u,s on(x,Table,s)  clear(x,s)  clear(y,s)  clear(u,s)  ~(u=y)  clear(u,movefromTable(x,y,s))  ~(u=y)  clear(u,movefromTable(x,y,s)) Frame axioms for “move x from y to z” Frame axioms for “move x from y to table” Frame axioms for “move x from Table to y”

31. 31 Difference with previous representation? on(B,A,S 0 ) on(A,Table,S 0 ) clear(B,S 0 )  x,y,z,s on(x,y,s)  ~y=Table  ~x=z  ~on(z,y,s)  x,y,z,s on(x,y,s)  ~y=z  ~on(x,z,s)  x,y,s on(x,y,s)  ~y=Table  ~clear(y,s) Initial situation: Consistency axioms: No difference here!

32. 32 Difference with previous representation (2):  x,y,z,s on(x,y,s)  clear(x,s)  clear(z,s)  on(x,z,move(x,y,z,s))  clear(y,move(x,y,z,s)) on(x,z,move(x,y,z,s))  clear(y,move(x,y,z,s))  x,y,s on(x,Table,s)  clear(x,s)  clear(y,s)  on(x,y,movefromT(x,y,s))  x,y,s on(x,y,s)  clear(x,s)  on(x,Table,movetoT(x,y,s))  clear(y,movetoT(x,y,s)) New initiated properties for each action: Each of these for EVERY instance of x,y and z ! For N blocks: N ! times all these rules. Also for the frame axioms!

33. Meta-representation The Situation Calculus

34. 34 Meta-representation Is the frame axiom for on/2 and for clear/1 combined in one ! on(x,y,s) holds(on(x,y),s)  Basic change in representation:  Why is this useful?  on/2 now is a term  FOL allows to abstract terms by variables (and quantify over them)  FOL doesn’t allow abstraction of atoms or predicates !  x,y,p,s holds(p,s)  ~p = on(x,y)  holds(p,movetoT(x,y,s))  holds(p,movetoT(x,y,s))  Example:  Could not be done with extra-argument representation.

35. 35 Meta-representation: initial situation Meta-representation: initial situation  p holds(p,S 0 )  p=on(B,A)  p=on(A,Table)  p=on(A,Table)  p=clear(B)  p=clear(B)  The initial situation: holds(p,S 0 )  initially(p) initially(p)  p=on(B,A)  p=on(A,Table)  p=on(A,Table)  p=clear(B)  p=clear(B)  A slightly better representation:

36. 36 Situation Names: final version S0S0 A B AB move B from A to Table 1 2 moveBATable( moveBATable(S 0 ) movetoT(B,A, movetoT(B,A,S 0 ) 3 result(movetoT(B,A), result(movetoT(B,A),S 0 ) Reason: The name of the operator is now a term that we can abstract with a variable movetoT(B,A)

37. 37 Formally: situation names: move(x,y,z)movetoT(x,y)movefromT(x,y)  We introduce a separate term-representation for operator patterns:  Situations are now represented as: result(o,s), where o is an operator pattern and s a situation. S 0 result(move(C,B,A),S 0 )) result(movefromT(A,B), result(movetoT(B,A),S 0 )) …  Examples:

38. 38 Improvement ? move(x,y,z)movetoT(x,y)movefromT(x,y)  Ontological improvement:  Now we can not only name SITUATIONS, we can ALSO name OPERATOR PATTERNS. AND abstract them by variables: holds(p,result(o,s))  …  Use: we can now define if list, add list and delete list of each operator pattern explicitly with new predicates !

39. 39 If-list: legal/2 predicate if on(x,y) clear(x) clear(x) clear(z) clear(z) add clear(y) on(x,z) on(x,z) delete on(x,y) clear(z) clear(z) move x from y to z legal(move(x,y,z),s)  holds(on(x,y),s)  holds(on(x,y),s)  holds(clear(x),s)  holds(clear(x),s)  holds(clear(z),s) holds(clear(z),s)  Move x from y to z: legal(movetoT(x,y),s)  holds(on(x,y),s)  holds(on(x,y),s)  holds(clear(x),s) holds(clear(x),s) legal(movefromT(x,y),s)  holds(on(x,Table),s)  holds(on(x,Table),s)  holds(clear(x),s)  holds(clear(x),s)  holds(clear(y),s) holds(clear(y),s)  Others:

40. 40 Add-list: initiates/2 predicate if on(x,y) clear(x) clear(x) clear(z) clear(z) add clear(y) on(x,z) on(x,z) delete on(x,y) clear(z) clear(z) move x from y to z initiates(move(x,y,z),p)  p = clear(y)  p = on(x,z) p = clear(y)  p = on(x,z)  Move x from y to z: initiates(movetoT(x,y,z),p)  p = on(x,Table)  p = clear(y) p = on(x,Table)  p = clear(y) initiates(movefromT(x,y,z),p)  p = on(x,y) p = on(x,y)  Others:

41. 41 Delete-list: terminates/2 predicate: if on(x,y) clear(x) clear(x) clear(z) clear(z) add clear(y) on(x,z) on(x,z) delete on(x,y) clear(z) clear(z) move x from y to z terminates(movetoT(x,y),p)  p = on(x,y) p = on(x,y) terminates(movefromT(x,y),p)  p = on(x,Table)  p = clear(y) p = on(x,Table)  p = clear(y)  Others: terminates(move(x,y,z),p)  p = on(x,y)  p = clear(z) p = on(x,y)  p = clear(z)  Move x from y to z:

42. 42 What is gained ? The Situation Calculus! only 1 rule ! holds(p,S 0 )  initially(p)  + the initialization rule: holds(p,result(o,s))  holds(p,s)  legal(o,s)  ~terminates(o,p) holds(p,s)  legal(o,s)  ~terminates(o,p)  The frame axiom: holds(p,result(o,s))  legal(o,s)  initiates(o,p)  Positive effects of actions:

43. 43 Problem independence holds(p,S 0 )  initially(p) holds(p,result(o,s))  legal(o,s)  initiates(o,p) holds(p,result(o,s))  holds(p,s)  legal(o,s)  ~terminates(o,p) holds(p,s)  legal(o,s)  ~terminates(o,p)  Observe that these axioms defining ‘holds’ are problem independent  and can be applied to any planning problem.  For each new planning problem: only initially, legal, initiates and terminates need to be defined !  These correspond to the initial situation and to the if- add- and delete-lists of operator patterns.

44. 44 Planning: false  holds(p 1,s)  …  holds(p n,s)  A goal of the type:  should be added, where p 1,…,p n are the properties that should hold in the goal situation.  Deduction proceeds similarly to what was presented in the extra-argument representation.  In particular; the final value for s in the unifier is the plan !

45. 45 Completion and consistency  Note that all formulae are essentially Horn clauses (possibly extended with negation in the bodies - case: ~terminates). holds(p,result(o,s))  holds(p,s)  legal(o,s)  ~terminates(o,p) holds(p,s)  legal(o,s)  ~terminates(o,p)  Disjunctions in bodies are readily transformed to a set of Horn clauses. terminates(move(x,y,z),p)  p = on(x,y)  p = clear(z) p = on(x,y)  p = clear(z) terminates(move(x,y,z),p)  p = on(x,y) terminates(move(x,y,z),p)  p = clear(z)

46. 46 Completion and consistency (2)  Assume that this description is complete (anything else than described is false !), then we can interpret these Horn clauses as a Logic program.  Consequence: the consistency axioms are no longer needed:  Requires that we understand ~holds(p,s) as holds(~p,s),  then information such as holds(~on(B,Table),S 0 ) follows from our specification.

47. Meta-representation (2) The Event Calculus

48. 48 Using time points: Time points Situation names Extra argument Meta-predicate on(A,B,T) on(A,B,S) on(A,B,S) holds(on(A,B),T) holds(on(A,B),S) holds(on(A,B),S)  Alternative to situations:  Ontology: something holds on a specific moment in time.

49. 49 The main axioms: Where the concepts: initially/1, initiates/2 and terminates/2 are the same as before (and problem dependent), and legal/2 is completely similar as before,but defined in terms of holds(p,t) instead of holds(p,s). holds(p,t)  initiated(p,t’)  t’ < t  ~clipped(p,t’,t) initiated(p,t 0 )  initially(p) initiated(p,t)  event(o,t)  legal(o,t)  initiates(o,p) clipped(p,t’,t)  t’ < s  s < t  event(o,s)  legal(o,s)  terminates(o,p)  legal(o,s)  terminates(o,p) t 0 < t  ~t 0 =t

50. 50 Event calculus pictured: t0t0t0t0 on(A,B) t on(A,B) is still true if it was not undone. nothing here ‘clips’ t0t0t0t0 on(A,B) t on(A,B) is still true if it was not undone. t’ nothing ‘clips’  When does something hold at time t ?  If is was true from the beginning and not ‘clipped’:  If something happened to make it true and not ‘clipped’:

51. 51 The event calculus  The new elements in this ontology:  Temporal identification through time points instead of situations.  The relation </2 should be defined as a (strict) partial order (the temporal order on time points).  There is now only 1 definition of holds !!  This includes: the initial situation + relating situations + the frame axioms!  In situations calculus situations are related to the previous situation In event calculus they are related to some previous moment in time that initiated something (which hasn’t been clipped).  Allows to make bigger steps than just 1 at a time.

52. 52 holds(p,t)  initiated(p,t’)  t’ < t  ~clipped(p,t’,t) initiated(p,t 0 )  initially(p) initiated(p,t)  event(o,t)  legal(o,t)  initiates(o,p) clipped(p,t’,t)  t’ < s  s < t  event(o,s)  legal(o,s)  terminates(o,p)  legal(o,s)  terminates(o,p) Relation to planning?  A plan in this representation is a set of atoms: event(O 1,T 1 ) event(O 2,T 2 ) event(O 3,T 3 ).. where each O i is an operator, each T i a time point, T i < T i+1 and executing O 1, O 2, O 3,… in sequence in the initial state gives the goal state.  But how can we get this plan from a goal like:  t holds(on(A,Table),t)  We CAN NOT (deductively) !!!!

53. 53 Relation to planning (2)  Deduction from a goal:  needs to go through initiated/2.  Unless the goal was already satisfied in the initial state, this requires event/2 facts to hold !  But there is NO definition for event/2:  This predicate is the object of our search !!  SO:  is not a logical consequence of the event calculus theory.  It is not true in ALL models. holds(p,t)  initiated(p,t’)  t’ < t  ~clipped(p,t’,t)  ~clipped(p,t’,t) initiated(p,t 0 )  initially(p) initiated(p,t)  event(o,t)  legal(o,t)  initiates(o,p)  initiates(o,p)  t holds(on(A,Table),t)

54. 54  SO: the goal:  is not true in all models.  BUT: if there exists a model (with some event/2 facts true) in which the goal is also true, then the true event/2 atoms in that model give us the plan.  Model-generation techniques!  Or alternatively: abductive reasoning techniques.  Meaning: find a set of hypothesis  of event/2 atoms such that: Model generation  t holds(on(A,Table),t)   Theory entails