1. CISC453 Winter 2010 Classical Planning AIMA3e Ch 10

2. What is Planning?  generate sequences of actions to perform tasks and achieve objectives  involving states, actions & goals  a search for a solution over an abstract space of plans  the assumptions for classical planning problems  fully observable  we see everything that matters  deterministic  the effects of actions are known exactly  static  no changes to environment other than those caused by agent actions  discrete  changes in time and space occur in quantum amounts  single agent  no competition or cooperation to account for Classical Planning 2

3. Background: Planning  real systems help humans in practical applications  design and manufacturing environments  military operations, games, space exploration  notes:  the methods of Ch 10 (Classical Planning)  assume the classical environment assumptions  in Ch 11 (Planning & Acting in the Real World)  we'll introduce methods to handle real-world situations where the classical assumptions may not hold Classical Planning 3

4. Planning: Language  What is a good language?  expressive enough to describe a wide variety of problems  restrictive enough for efficient algorithms to operate on it  planning algorithm should be able to take advantage  of the logical structure of the problem  historical AI planning languages  STRIPS was used in classical planners  Stanford Research Institute Problem Solver  ADL addresses expressive limitations of STRIPS  Action Description Language  adds features not in STRIPS  negative literals, quantified variables, conditional effects, equality  we'll use one version of the now de facto standard PDDL Classical Planning 4

5. Planning language  there's now wide agreement on some planning language basics  one key is adopting of a factored representation for states  each state is represented as a collection of variables  this contrasts with  the atomic representation typical of our previous search algorithms (recall the Romanian driving problem: state = city)  the language we'll use is a variant of PDDL  Planning Domain Definition Language  to see its expressive power, recall propositional agent in the Wumpus World, which requires 4Tn 2 actions to describe a movement of 1 square  PDDL captures this with a single Action Schema Classical Planning 5

6. State & Action Representations  each state is represented as a conjunction of fluents  these are ground, functionless atoms  in addition, we'll use Database semantics  1. the Close World Assumption  fluents not explicitly mentioned are false  2. the Unique Names Assumption  different ground terms are different objects: Plane1, Plane2  this state representation allows alternative algorithms  it can be manipulated either by logical inference techniques or by set operations  actions are defined by a set of action schemas  these implicitly define the ACTIONS(s) & RESULT(s, a) functions required to apply search techniques Classical Planning 6

7. Action Schemas  PDDL & the Frame Problem  recall the representational issue of capturing what stays the same given some action  in PDDL we specify what changes, and if something is not mentioned, it stays the same  Action Schemas  are a lifted representation (recall Generalized Modus Ponens)  lifts from propositional logic to a restricted subset of FOL  each one stands for a set of variable-free actions  each includes the schema name, list of variables used, preconditions & effects  we consider variables as universally quantified, choose values to instantiate them  PRECOND: defines states in which an action can be executed  EFFECT: defines the result of executing the action Classical Planning 7

8. Action Schemas  each represents a set of variable-free actions  form: Action Schema = predicate + preconditions + effects  example: Action(Fly(p, from, to), PRECOND: At(p, from)  Plane(p)  Airport(from)  Airport(to) EFFECT: ¬AT(p, from)  At(p, to))  an action schema in which (p, from, to) need to be instantiated  the action name and its parameter list  its preconditions  a conjunction of function-free literals  its effects  a conjunction of function-free literals Classical Planning 8

9. Applying Action Schemas  action a is applicable in state s  if its preconditions are satisfied in s  its preconditions are entailed by s  given variables, there can be multiple applicable instantiations  for v variables in a domain with k unique object names, worst case time to find applicable ground actions is O(v k )  leads to 1 approach for solving PDDL planning problems  propositionalize by replacing action schemas with sets of ground actions & applying a propositional solver like SATPlan  impractical for large v & k  result of executing a in s is s'  s' is derived from s & action a's EFFECTs  remove -ve literal fluents (the delete list, DEL(a))  add +ve literals in EFFECTs (the add list, ADD(a))  RESULT(s, a) = (s - DEL(a))  ADD(a) Classical Planning 9

10. Action Schemas  PDDL schemas  1. variables & ground terms  variables in effects must also be in precondition  so matching to state s yields results with all variables bound  i.e. that contain only ground terms  2. handling of time  no explicit time terms, unlike axioms we saw for logical agents  instead time is implicitly represented in PDDL schemas  preconditions always refer to time: t  effects always refer to time: t + 1  3. a set of schemas defines a planning domain  a specific problem adds initial & goal states Classical Planning 10

11. Initial States, Goals, Solutions  initial state  conjunction of ground terms  goal  conjunction of +ve & -ve literals  both ground terms & those containing variables  variables are treated as existentially quantified  solution  a sequence of actions ending in s that entails the goal  example:  Plane(P1)  At (P1, SFO) entails At(p, SFO)  Plane (p)  defines planning as a search problem Classical Planning 11

12. Some Language Semantics  How do actions affect states?  an action is applicable in any state that satisfies its precondition(s)  applicability involves a substitution  for the variables in the PRECOND  example State: At(P1,JFK)  At(P2,SFO)  Plane(P1)  Plane(P2)  Airport(JFK)  Airport(SFO) Satisfies PRECOND of Fly action: At(p, from)  Plane(p)  Airport(from)  Airport(to) With substitution  ={p/P1, from/JFK, to/SFO} Thus the action is applicable. Classical Planning 12

13. Language Semantics  result of executing action a in state s is the state s’  s’ is same as s except  any positive literal P in the effect of a is added to s’  any negative literal ¬P is removed from s’  example Fly(p, from, to) EFFECT: ¬AT(p, from)  At(p, to),  ={p/P1, from/JFK, to/SFO} s: At(P1,JFK)  At(P2,SFO)  Plane(P1)  Plane(P2)  Airport(JFK)  Airport(SFO) s': At(P1,SFO)  At(P2,SFO)  Plane(P1)  Plane(P2)  Airport(JFK)  Airport(SFO) Classical Planning 13

14. Example: air cargo transport  simple actions to illustrate  moving cargo between New York(JFK) & San Francisco(SFO) Init(At(C1, SFO)  At(C2,JFK)  At(P1,SFO)  At(P2,JFK)  Cargo(C1)  Cargo(C2)  Plane(P1)  Plane(P2)  Airport(JFK)  Airport(SFO)) Goal(At(C1,JFK)  At(C2,SFO)) Action(Load(c, p, a) PRECOND: At(c, a) At(p, a) Cargo(c) Plane(p) Airport(a) EFFECT: ¬At(c, a) In(c, p)) Action(Unload(c, p, a) PRECOND: In(c, p) At(p, a) Cargo(c) Plane(p) Airport(a) EFFECT: At(c, a)  ¬In(c, p)) Action(Fly(p, from, to) PRECOND: At(p, from) Plane(p) Airport(from) Airport(to) EFFECT: ¬ At(p, from)  At(p, to)) Classical Planning 14

15. Air Cargo Example  this planning domain involves 3 actions  and 2 affects predicates, In(c, p) & At(x, a)  properly maintaining At predicates could be an issue  where is cargo that is in a plane?  without quantifiers, problem of expressing what happens to cargo that is in a plane  solve by treating At predicate as applying only to cargo after it is unloaded, in effect not At anywhere when it is in a plane  another issue is that this representation allows "empty" actions that produce contradictory effects  Fly(P1, JFK, JFK) yields At(P1, JFK)  ¬At(P1, JFK)  can ignore issues like this, rarely produce incorrect plans, or  add inequality precondition: from  to Classical Planning 15

16. Air Cargo Example  the solution is pretty obvious Init(At(C1, SFO)  At(C2,JFK)  At(P1,SFO)  At(P2,JFK)  Cargo(C1)  Cargo(C2)  Plane(P1)  Plane(P2)  Airport(JFK)  Airport(SFO)) Goal(At(C1,JFK)  At(C2,SFO)) Action(Load(c, p, a) PRECOND: At(c, a) At(p, a) Cargo(c) Plane(p) Airport(a) EFFECT: ¬At(c, a) In(c, p)) Action(Unload(c, p, a) PRECOND: In(c, p) At(p, a) Cargo(c) Plane(p) Airport(a) EFFECT: At(c, a)  ¬In(c, p)) Action(Fly(p, from, to) PRECOND: At(p, from) Plane(p) Airport(from) Airport(to) EFFECT: ¬ At(p, from)  At(p, to)) [Load(C1, P1, SFO), Fly(P1, SFO, JFK), Unload (C1, P1, JFK), Load(C2, P2, JFK), Fly(P2, JFK, SFO), Unload(C2, P2, SFO)] Classical Planning 16

17. Example: flat tire problem  simple actions to illustrate  changing a flat tire (in a bad neighbourhood) Init(At(Flat, Axle)  At(Spare, Trunk)) Goal(At(Spare, Axle)) Action(Remove(Spare, Trunk) PRECOND: At(Spare, Trunk) EFFECT: ¬At(Spare, Trunk)  At(Spare, Ground)) Action(Remove(Flat, Axle) PRECOND: At(Flat, Axle) EFFECT: ¬At(Flat, Axle)  At(Flat, Ground)) Action(PutOn(Spare, Axle) PRECOND: At(Spare, Ground) ¬At(Flat, Axle) EFFECT: At(Spare, Axle)  ¬At(Spare, Ground)) Action(LeaveOvernight PRECOND: EFFECT: ¬ At(Spare, Ground)  ¬ At(Spare, Axle)  ¬ At(Spare, Trunk)  ¬ At(Flat, Ground)  ¬ At(Flat, Axle) ) Classical Planning 17

18. Flat Tire example  highly simplified, abstracted version of the problem of constructing a plan to fix a flat tire  4 actions  note capturing of "bad neighbourhood" as tires "disappearing" if the car is left overnight  the simple solution [Remove (Flat, Axle), Remove(Spare, Trunk), PutOn(Spare, Axle)] Classical Planning 18

19. Example: Blocks World  simple actions to illustrate  building a tower in Blocks World Init(On(A, Table)  On(B, Table)  On(C,A)  Block(A)  Block(B)  Block(C)  Clear(B)  Clear(C)) Goal(On(A,B)  On(B,C)) Action(Move(b, x, y) PRECOND: On(b, x)  Clear(b)  Clear(y)  Block(b)  (b  x)  (b  y)  (x  y) EFFECT: On(b, y)  Clear(x)  ¬ On(b, x)  ¬ Clear(y)) Action(MoveToTable(b, x) PRECOND: On(b, x)  Clear(b)  Block(b)  (b  x) EFFECT: On(b, Table)  Clear(x)  ¬ On(b, x))  simple plan given the initial state  note PRECOND of Move uses inequality to prevent empty moves  Move(B,C,C) Classical Planning 19

20. Blocks World example  illustration of the problem Classical Planning 20

21. Blocks World example  problem domain:  stack cube-shaped blocks that are on a table  Notes:  only 1 block can fit on top of another  moves can be onto another block or onto the table  FOL would use quantifiers to express  there's no block on top of some other block  without quantifiers, PDDL requires Clear(x) predicate  Move action schema must be complemented by MoveToTable to avoid errors with the Clear predicate  since Clear(Table) is always true  residual problem if bind y to Table in Move(b, x, y)  search space grows, though answers still correct  fix with a Block(m) predicate & adding Block(b)  Block(y) to the preconditions for Move Classical Planning 21

22. Complexity & Classical Planning  first, we distinguish 2 versions of the planning problems  PlanSAT:  is there a plan that solves the problem?  Bounded PlanSAT:  is there a solution of k or fewer steps?  this can be used to find optimal plans  so long as we don't allow functions the number of states is finite & both categories are decidable  if we allow functions, PlanSAT becomes semi-decidable (may not terminate on unsolvable problems) though Bounded PlanSAT remains decidable even with functions Classical Planning 22

23. Complexity & Classical Planning  sadly, in general, both problems are NP-hard  putting restrictions on the expressiveness may help  for example, disallowing -ve preconditions lets PlanSAT reduce to P  fortunately for planners, many useful problems are easier than the worst case  for blocks world & air cargo domains, Bounded PlanSAT is NP- Complete, while PlanSAT is in P!  so, optimal planning is hard, sometimes sub-optimal planning is easy  reminder of complexity category relationships 23

24. Planning with state-space search  given the problem formulation  we can use search algorithms to generate plans  systematic heuristic search  local search (while recording trail of states)  search can be  in either the forward or the backward direction  recall the bi-directional example from search topic  problems related to non-invertible successor functions  Progression planners  do a forward state-space search  consider the effects of all possible actions in a given state  Regression planners  do a backward state-space search  to achieve a goal  consider what must have been true in the previous state  from effects through actions to preconditions 24

25. Progression & Regression Classical Planning 25  partial samples: plane moving example

26. Progression algorithm  formulation as a state-space search problem  Initial state = initial state of the planning problem  literals not appearing are false  Actions = apply those whose preconditions are satisfied  add positive effects, delete negative  Goal test = does the state satisfy the goal  Step cost = for simplicity, each action costs 1  recall, functions are not permitted in the representation  any graph search that is complete is a complete planning algorithm  for example, A* Classical Planning 26

27. Progression algorithm  Progression approach  while complete is inefficient  any applicable action is a possible next step  typically its problems are related to branching factor  (1) irrelevant actions  just because the preconditions of an action are satisfied  does not mean that it should be done  (2) requirement for a good heuristic  in order for improved efficiency of search Classical Planning 27

28. Progression: issues  illustrate with the air cargo example  assume 10 airports, 5 planes & 20 pieces of cargo at each  problem: move all cargo at airport A to airport B  solution is pretty simple, but uninformed search has serious complexity difficulties  just considering planes: 50 x 9 = 450 possible flights  suppose all planes & packages were at 1 airport: 10450 possible actions (200 x 50 + 50 x 9)  if 2000 actions available on average, 2000 41 nodes in search graph  fortunately, good (domain-independent) heuristics are possible, & can be derived automatically Classical Planning 28

29. Regression algorithm  what about Regression planners  issues:  How do we determine predecessors?  as we saw for bi-directional search, the problem may not allow predecessor determination (n-Queens)  fortunately PDDL formulation facilitates it  terminology: predecessor is g', action a, goal g  g' = (g - Add(a))  Precond(a), since any effects added by a might not have been true previously & preconditions must have been met or could not execute a  this process requires partially uninstantiated actions & states  use standardized variable names (avoid clashes, retain generality of variables)  implicitly quantify over these so 1 description summarizes possibility of using any appropriate ground term Classical Planning 29

30. Regression: relevant actions  which action?  forward uses applicable actions (their preconditions are met)  backward uses relevant actions (they could be the last step leading to current state)  relevant: at least 1 effect (+ve or -ve) must unify with a term in the (current) goal & there can't be any effect that negates a term in the goal  process: use the action formed by substituting the most general unifier into the standardized action schema  more formal definition of relevant actions  if goal g contains literal g i & action schema A (standardized to produce A') has effect literal e j ' where Unify (g i, e j ') =  & a' = SUBST(, A') & there's no effect in a' that is the negation of a literal in g, then a' is a relevant action toward g  terminates when a predecessor is satisfied by the initial state Classical Planning 30

31. Progression & Regression  status of the regression approach  1. there's a lower branching factor in the backward direction, the result of only considering relevant actions  2. but each state is really a set of states  the state defined by true or false for the ground fluents, and the states not mentioned (see mid p 374)  this representation of stages in the backward search as sets of states complicates the process of developing heuristics  progression versus regression  the greater availability of accurate heuristics for forward search has resulted in it being used in many more planning systems Classical Planning 31

32. Heuristics for planning  the complexity of planning problems  means that we need good heuristics  below: 2 general approaches to finding admissible heuristics  recall that an admissible h(n) (i.e. one that never overestimates cost) allows A* search for optimal solutions  recall from our (informed search) discussion of finding heuristics  as the optimal solution to a relaxed version of the problem  from problem decomposition  if we can make the subgoal independence assumption we can approximate the cost of solving a conjunction of subgoals as the sum of the costs of solving the corresponding subproblems  (1) relaxation: search problem viewed as a graph allows  (a) adding edges to make a path to a solution easier to find  (b) merging states in an abstraction of the original problem so searching is in a space with fewer nodes Classical Planning 32

33. Heuristics by problem relaxation  (a) adding edges to the search space  (1) ignore all preconditions  assume all actions are applicable in all states, so any goal fluent takes just a single step  apparently #steps ≈ #unsatisfied goals, but  note: some actions may achieve > 1 goal, & others may undo goals  ignoring the latter, h(n) = min # actions for which the union of their effects satisfies the goal  this is a version of the "set-cover problem", known to be NP-hard!  a greedy algorithm yields set covering within log n of the true min (n is # literals in goal), but is not guaranteed admissible  (2) ignore selected PRECONDs of actions  as in the N-puzzle heuristics: misplaced tiles, manhattan distance  the factored representation in action schemata for planning problems allows automated derivation of such heuristics  (3) ignore delete lists  if all goals & preconditions have only +ve literals (true for many problems anyway & others can be converted to this form) Classical Planning 33

34. Heuristics by problem relaxation  (a) adding edges to search space  (3) ignore delete lists  with only +ve literals in all goals & preconditions  then remove -ve literals from effects  now steps are monotonic progress towards the goal w/o undoing  but, it's still NP-hard to find an optimal solution, though it can be approximated in polynomial time with hill climbing  summary  adding edges relaxes a planning problem to yield heuristics but they're still expensive to calculate  (b) reduce nodes by abstraction to merge states  return to the air cargo example  10 airports, 50 planes, 200 packages means the general case has 50 10 x 200 50+10 or 10 155 states  a specific problem: say all pkgs are at 5 airports & all pkgs at any airport have 1 destination Classical Planning 34

35. Heuristics by problem relaxation  (b) reduce nodes by abstraction to merge states  in the air cargo example, in general, 10 155 states  a specific problem has all pkgs at 5 airports & pkgs at any airport have 1 destination  an abstraction of this version omits At fluents except those re: 1 plane & 1 pkg at each of 5 airports, reducing # states to 5 10 x 5 5+10 or 10 17 states  so a solution is shorter & an admissible h(n) & is extensible to the original problem by adding more Load & Unload actions  problem decomposition approaches for h(n)'s  the relevant pattern is:  subdivide problem  solve subproblems  combine solutions  the subgoal independence assumption  is useful in developing a heuristic as a sum of subgoal costs  is too optimistic if there are -ve interactions between subplans  is too pessimistic (inadmissible) when there are redundant actions in subplans 35

36. Heuristics by decomposition  problem decomposition approaches for h(n)'s  notation: represent the goal as a set of fluents G, where disjoint subsets of G are G 1, …G n, for which we find plans P 1, … P n & then use their costs to estimate cost for G  Cost(P i ) is an estimate, a possible heuristic  max i Cost(P i ) is admissible, overly optimistic  sum i Cost(P i ) is not generally admissible unless all G i & G j are independent (P i does not affect preconds or goals of P j ) in which case the sum is admissible & more accurate Classical Planning 36

37. Heuristics by decomposition  the challenge of abstraction is  to find one that has a significantly lower total cost (defining the abstraction + doing the abstracted search + mapping back to original problem) than the original problem  the pattern database techniques may be useful  recall them from informed search discussion of heuristics  amortize the cost of building the database over multiple problem solution instances  the FF (FASTFORWARD) planning system is a hybrid  it uses heuristics from a planning graph (our next discussion)  it uses local search (storing plan steps)  at a plateau or local maximum it does iterative deepening systematic search for a better state or gives up & restarts 37

38. Planning Graphs  Planning Graphs  are an alternative intermediate representation/data structure  they are polynomial complexity approximations of full (exponential) trees of all states & actions  they can't answer the question: is G reachable?  though they are correct when they say it's not reachable  they provide an estimate of the number of steps to G  any such estimate is optimistic, so is an admissible heuristic  planning graphs  provide a possible basis for better search heuristics  and they can also be used directly, for extracting a solution to a planning problem, by applying the GRAPHPLAN algorithm Classical Planning 38

39. Planning Graphs  a planning graph  is a directed graph organized in levels  the levels of nodes correspond to time steps in a plan  they consist of alternating S levels (in which the nodes are fluents that hold) & A levels (in which the nodes are actions that might be applicable)  level S 0 is the initial state  each level is a set of literals S i or a set of actions A i  S i : all the literals that could be true at that time step  depending on the actions executed at the previous steps  A i : all actions that could have PRECONDs satisfied at that step  depending on which of the literals actually hold Classical Planning 39

40. Planning Graphs  explanation regarding could on the previous slide  a planning graph only captures a restricted subset of the possible -ve interactions  so a literal might appear at a level earlier than it actually would (if it would at all), though it never appears too late  despite this error, the level j of the first appearance is a good estimate of how difficult it is to achieve from the initial state  refers to the approximate nature of the state/action lists  in part, this allows efficient construction by recording a restricted subset of possible -ve interactions among actions  note:  planning graphs apply only for propositional problems  action schemas for problems with variables can be propositionalized  their advantages may make this worthwhile, despite the increased size of the problem description Classical Planning 40

41. Planning Graphs  our simple example: "have your cake & eat it too"  the problem description Init(Have(Cake)) Goal(Have(Cake)  Eaten(Cake)) Action(Eat(Cake) PRECOND: Have(Cake) EFFECT: ¬Have(Cake)  Eaten(Cake)) Action(Bake(Cake) PRECOND: ¬ Have(Cake) EFFECT: Have(Cake))  the corresponding planning graph 41

42. Planning Graph Cake Example  start at level S 0, determine action level A 0 & next level S 1  A 0 : all actions whose preconditions are satisfied in the previous level (initial state)  actions are shown in rectangular boxes  lines connect PRECONDs at S 0 to EFFECTs at S 1  also, for each literal in S i, there's a persistence action (square box) & line to it in the next level S i+1  level A 0 contains the actions that could occur  conflicts between actions are represented by arcs: mutual exclusion or mutex links 42

43. Planning Graph Cake Example  level S 1 contains all the literals that could result  from picking any subset of actions in A 0  so S 1 is a belief state consisting of the set of all possible states  each is a subset of literals with no mutex links between members  conflicts between literals that cannot occur together are represented by the mutex links.  the level generation process is repeated  eventually consecutive levels are identical: leveling off 43

44. Planning Graph Cake Example  mutex relation holds between 2 actions at a level when  1. inconsistent effects  one action negates the effect of another  2. interference  an effect of one action negates a precondition of the other  3. competing needs  a precondition of one action is mutex with a precondition of the other Classical Planning 44

45. Planning Graph Cake Example  mutex relation holds between 2 literals at a level when  1. one is the negation of the other  2. if each possible action pair that could achieve the literals is mutex (Have(Cake) & Eaten(Cake) at S 1 )  this is termed inconsistent support Classical Planning 45

46. Planning Graphs & Heuristics  Planning Graphs  construction has complexity polynomial in the size of the planning problem  given l literals, a actions, & a PG of n levels: O(n(a + l) 2 )  the completed PG  provides information about the problem & candidate heuristics  1. a goal literal g that does not appear in the final level cannot be achieved by any plan  2. the level cost, the level at which a goal literal first appears, is useful as a cost estimate of achieving that goal literal  note that level cost is admissible, though possibly inaccurate since it counts levels, not actions  we could find a better alternative level cost by using a serial planning graph variation, restricted to one action per level  mutex links between every pair of actions except persistence actions Classical Planning 46

47. Planning Graphs & Heuristics  Planning Graph provides  possible heuristics for the cost of a conjunction of goals  1. max-level: highest level of any conjunct in the goal  admissible, possibly not accurate  2. level sum: the sum of level costs of conjuncts in the goal  incorporates the subgoal independence assumption  so may be inadmissible to degree the assumption does not hold  3. set-level: level where all goal conjuncts are present without mutex links  this one dominates the max level heuristic  & is good where there are interactions among subplans Classical Planning 47

48. Planning Graphs & Heuristics  a Planning Graph is a relaxed version of the problem  if a goal literal g does not appear, no plan can achieve it, but if it does appear, is not guaranteed to be achievable  why?  the PG only captures pairwise conflicts & there could be higher order conflicts  likely not worth the computational expense of checking for them  similar to Constraint Satisfaction Problems where arc consistency was a valuable pruning tool  3-consistency or even higher order consistency would have made finding solutions easier but was not worth the additional work  an example where PG fails to indicate unsolvable problem  blocks world problem with goal of A on B, B on C, C on A  any pair of subgoals are achievable, so no mutexes  problem only fails at stage of searching the PG Classical Planning 48

49. The GRAPHPLAN Algorithm  the GRAPHPLAN algorithm  generates the Planning Graph & extracts a solution from it function GRAPHPLAN(problem) return solution or failure graph  INITIAL-PLANNING-GRAPH(problem) goals  CONJUNCTS(problem. GOAL) nogoods  an empty hash table for tl = 0 to  do if goals all non-mutex in S t of graph then solution  EXTRACT-SOLUTION(graph, goals, NUMLEVELS(graph), nogoods) if solution  failure then return solution if graph and nogoods have both leveled off then return failure graph  EXPAND-GRAPH(graph, problem) Classical Planning 49

50. Example: Spare Tire Problem  recall the spare tire problem Init(At(Flat, Axle)  At(Spare, Trunk)) Goal(At(Spare, Axle)) Action(Remove(Spare, Trunk) PRECOND: At(Spare, Trunk) EFFECT: ¬At(Spare, Trunk)  At(Spare, Ground)) Action(Remove(Flat, Axle) PRECOND: At(Flat, Axle) EFFECT: ¬At(Flat, Axle)  At(Flat, Ground)) Action(PutOn(Spare, Axle) PRECOND: At(Spare, Ground) ¬At(Flat, Axle) EFFECT: At(Spare, Axle)  ¬At(Spare, Ground)) Action(LeaveOvernight PRECOND: EFFECT: ¬ At(Spare, Ground)  ¬ At(Spare, Axle)  ¬ At(Spare, Trunk)  ¬ At(Flat, Ground)  ¬ At(Flat, Axle) ) Classical Planning 50

51. GRAPHPLAN Spare Tire Example  Notes:  this figure shows the complete Planning Graph for the problem  arcs show mutex relations  but arcs between literals are omitted to avoid clutter  it also omits unchanging +ve literals (for example, Tire(Spare)) & irrelevant -ve literals  bold boxes & links indicate the solution plan Classical Planning 51

52. GRAPHPLAN Spare Tire Example  S 0 is initialized to 5 literals  from the problem initial state and the CWA literals  no goal literal in S 0 so EXPAND-GRAPH add actions  those with preconditions satisfied in S 0  also adds persistence actions for literals in S 0  adds the effects at level S 1, analyzes & adds mutex relations  repeat until the goal is in level S i or failure Classical Planning 52

53. GRAPHPLAN Spare Tire Example  EXPAND-GRAPH adds constraints: mutex relations  inconsistent effects (action x vs action y)  Remove(Spare, Trunk) & LeaveOvernight: At(Spare, Ground) & ¬At(Spare, Ground)  interference (effect negates a precondition)  Remove(Flat, Axle) & LeaveOvernight: At(Flat, Axle) as PRECOND & ¬At(Flat, Axle) as EFFECT  competing needs (mutex preconditions)  PutOn(Spare, Axle) & Remove(Flat, Axle): At(Flat, Axle) & ¬At(Flat, Axle)  inconsistent support (actions to produce literals are mutex)  in S2, At(Spare, Axle) & At(Flat, Axle) 53

54. Reminder: GRAPHPLAN Algorithm  the GRAPHPLAN algorithm both generates the Planning Graph & extracts a solution from it function GRAPHPLAN(problem) return solution or failure graph  INITIAL-PLANNING-GRAPH(problem) goals  CONJUNCTS(problem. GOAL) nogoods  an empty hash table for tl = 0 to  do if goals all non-mutex in S t of graph then solution  EXTRACT-SOLUTION(graph, goals, NUMLEVELS(graph), nogoods) if solution  failure then return solution if graph and nogoods have both leveled off then return failure graph  EXPAND-GRAPH(graph, problem) Classical Planning 54

55. GRAPHPLAN Spare Tire Example  in S2, the goal literals exist, & are not mutex  just 1 goal literal so obviously not mutex with any other goal  since a solution may exist, EXTRACT-SOLUTION tries to find it  EXTRACT-SOLUTION may search backwards for a solution  initial state: last level of the PG + goals of the planning problem  actions from S i  select a set of conflict-free actions in A i-1 with effects covering the goals  conflict free = no 2 actions are mutex & no pair of preconditions are mutex  goal: reach a state at level S 0 such that all goals are satisfied  cost: 1 for each action 55

56. GRAPHPLAN Solutions  if EXTRACT-SOLUTION fails  at that point it records (level, goals) as a "no-good"  subsequent calls can fail immediately if they require the same goals at that level  we already know planning problems are computationally hard  require good heuristics  greedy search with level cost of literals as a heuristic works well  1. pick literal with highest level cost  2. to achieve it, pick actions with easier preconds  action with smallest sum (or max) of level costs for its preconds 56

57. GRAPHPLAN Solutions  alternative to backward search for a solution  EXTRACT-SOLUTION could formulate a Boolean CSP  variables are actions at each level  values are Boolean: an action is either in or out of the plan  constraints are mutex relations & the need to satisfy each goal & precondition Classical Planning 57

58. GRAPHPLAN Termination  does GRAPHPLAN terminate?  does it stop & return failure if there's no solution  the general rationale is presented here  for some proof details see the textbook & its Ghallab reference  background  recall that levelled off means consecutive PG levels are identical  now note that a graph may level off before a solution can be found, on a problem for which there is a solution  example of 2 airports, 1 plane & n pieces of cargo at one of them, plan required to move all n to the other airport  but only 1 piece at a time fits in the plane  graph levels off at level 4, from which full solution can't be extracted (that would require 4n - 1 steps)  we need to take account of the no-goods (goals that were not achievable) as well  these may decline even after the graph levels off 58

59. GRAPHPLAN Termination  does GRAPHPLAN terminate?  given that no-goods might continue to decrease after the graph levels off  we now require a proof that both the graph & no-goods will level off  literals increase monotonically  once a literal appears, its persistence action causes it to stay  actions increase monotonically  once a literal that is a precondition appears, the action stays  mutexes decrease monotonically  though the graph simplifying conventions may not show it  if 2 actions are mutex at A i, they are also mutex at all previous levels where they appear 59

60. GRAPHPLAN Termination  does GRAPHPLAN terminate?  no-goods decrease monotonically  if a set of goals is not achievable at level i, they are not achievable at any previous level  so literals & actions increase monotonically  given a finite number of actions and literals, they will level off  and mutexes & no-goods decrease monotonically  since there cannot be fewer than 0, they must level off  if the graph & no-goods have levelled off & a goal is missing or is mutex with another goal, the algorithm can stop & return failure 60

61. Alternative Planning Approaches  the following slides outline brief discussions of 4 alternative approaches to solving planning problems  1. convert the PDDL description into a form solvable by a propositional logic satisfiability algorithm: SATPlan  2. convert into a situational calculus representation solvable by First-Order Logic inference techniques  3. represent & solve as a Constraint Satisfaction Problem  4. use a "plan refinement" approach to searching in a space of partially ordered plans Classical Planning 61

62. Planning with Propositional Logic  1. form a propositional logic satisfiability problem for submission to SATPlan  iteratively tries plans of increasing length (increasing number time steps) up to the T max parameter value, so finds the shortest plan, if one exists  the SATPlan algorithm  finds models for a (very long) PL sentence that includes initial state, goal, successor-state axioms, precondition axioms, & action-exclusion axioms  assigns true to the actions that are part of the correct plan & false to the others  if the planning is unsolvable the sentence will be unsatisfiable  any model satisfying the sentence will be a valid plan  an assignment that corresponds to an incorrect plan will not be a model because of inconsistency with the assertion that the goal is true Classical Planning 62

63. Planning with Propositional Logic  translation from PDDL to PL form for SATPlan  the process involves 6 steps  step 1: propositionalize actions, replacing each schema with a set of ground actions (constants substituted for variables)  step 2: define an initial state asserting F 0 for all fluents in the initial state, ¬F 0 for all fluents not in the initial state  step 3: propositionalize the goal - for each variable replace literals containing it with a disjunction over constants  step 4: include successor-state axioms for each fluent F t+1  ActionCausesF t  (F t  ¬ActionCausesNotF t ), where  ActionCausesF is disjunction of all ground actions with F in their add List  ActionCausesNotF is disjunction of all ground actions with F in their delete list  air cargo example: axioms are required for each plane, airport and time step! Classical Planning 63

64. Planning with Propositional Logic  the 6 steps from PDDL to SATPlan format  step 5: add precondition axioms (for every ground action A, add A t  PRE(A) t  step 6: add action exclusion axioms to say every action is distinct from every other action  submit to SATPlan Classical Planning 64

65. The SATPLAN Algorithm  the algorithm  note that in AIMA 3 e, this is first presented in the earlier section on propositional logic agents function SATPLAN(init, transition, goal, T max ) returns solution or failure inputs: init, transition, goal form a description of the planning problem T max, an upper limit to the plan length for t = 0 to T max do cnf  TRANSLATE-TO-SAT(init, transition, goal, t ) model  SAT-SOLVER(cnf) if model is not null then return EXTRACT-SOLUTION(model) return failure Classical Planning 65

66. Planning with Propositional Logic  some explanatory notes  distinct propositions for assertions about each time step  superscripts denote the time step: At(P1,SFO) 0  At(P2,JFK) 0  PDDL includes the Closed World Assumption, so conversion involves the need to specify which propositions are not true  ¬At(P1,JFK) 0  ¬At(P2,SFO) 0  unknown propositions are left unspecified  goal is associated with some particular time-step: which one? Classical Planning 66

67. Planning with Propositional Logic  some explanatory notes  we can't know in advance how many steps are required to achieve a goal so, to determine the time step T where the goal will be reached, the algorithm loops until success or some arbitrary time step limit  repeat  start at T = 0  Assert At(P1,JFK) 0  At(P2,SFO) 0  on failure: try T = 1  Assert At(P1,JFK) 1  At(P2,SFO) 1  add terms for next time step  until a solution is found or some max path length T max is exceeded: setting T max assures termination Classical Planning 67

68. Models  the pre-condition axioms  avoid models with illegal actions  e.g. for T = 1  Fly(P1, SFO, JFK) 0  Fly(P1, JFK, SFO) 0  Fly(P2, JFK, SFO) 0  the second action is infeasible, precluded if we include the appropriate pre-condition axiom: Fly(P1,JFK,SFO) 0  At(P1,JFK) 0  since At(P1, JFK) 0 false in initial state, Fly(P1, JFK, SFO) 0 false in any model Classical Planning 68

69. Models  without action-exclusion axioms  a model might include actions, for example having 1 plane fly to two destinations in a time step  Fly(P1, SFO, JFK) 0  Fly(P2, JFK, SFO) 0  Fly(P2, JFK, LAX) 0  though the second P2 action is infeasible, it requires the added action-exclusion axioms to avoid such solutions  ¬(Fly(P2,JFK,SFO) 0  Fly(P2,JFK,LAX) 0 ) Classical Planning 69

70. Planning in First-Order Logic  alternative 2: convert to the Situation Calculus format of FOL & apply FOL inferencing  recall Situation Calculus as a topic in the 352 Knowledge Representation discussion of time & action  PDDL omits quantifiers which limits expressiveness but helps control complexity of algorithms applied to it  PL representation also has limitations like the requirement that a "timestamp" becomes part of each fluent  Situation Calculus replaces explicit timestamps by using situations that correspond to a sequence or history of actions  1. situations  initial state is a situation  if s is a situation & a an action, RESULT(s, a) is also a situation  situations correspond to sequences or history of actions  2 situations are the same only if their start & actions are the same Classical Planning 70

71. Planning in First-Order Logic  convert to FOL Situation Calculus  2. fluents are functions or relations that vary over situations  syntax: situation is conventionally the last argument  At(x, l, s) is a relational fluent, true when object x is at location l in situation s & Location(x, s) is a functional fluent such that Location(x, s) = l in the same situation  3. a possibility axiom describes preconditions of each action  (s)  Poss(a, s) indicates when an action can be taken   is a formula giving preconditions  Alive(Agent, s)  Have(Agent, Arrow, s)  Poss(Shoot, s)  4. successor-state axioms for each fluent indicate what happens, depending on the action taken  here are the general form & an example from Wumpus World Action is possible  (Fluent is true in result state  Action's effect made it true V It was true before & the action left it unchanged) Poss(a, s)  (At(Agent, y, Result(a, s))  a = Go(x, y) V (At(Agent, y, s) Λ a  Go(y, z))) 71

72. Planning in First-Order Logic  express problem using FOL Situation Calculus  5. unique action axioms required to allow deductions such as  a  Go(y, z)  for each pair of action names A i & A j there's an axiom to say they are different: A i (x, …)  A j (y, …)  plus, for each action name A j an axiom indicates 2 uses are only equal if all arguments are equal  A i (x 1, …, x n ) = A i (y 1, …, y n )  x 1 = y 1  …  x n = y n  6. the solution is a situation (action sequence) that satisfies the goal  unfortunately, no major planning systems use situation calculus  basic efficiency issues of FOL inference  compounded by lack of good heuristics for planning with SC 72

73. Planning: Constraint Satisfaction  we noticed in discussions in 352  the similarities between Constraint Satisfaction Problems & boolean satisfiability  conversion to a CSP is similar to that of conversion to a propositional logic satisfiability problem  one difference is that it only requires a single variable Action t at each time step: its domain is the set of all possible actions  thus we also don't need the action exclusion axioms Classical Planning 73

74. Partial-order planning  plans may be developed through refinement of partially ordered plans  approaches describe so far, including both Progression and Regression planning, are totally ordered forms of plan search  they cannot take advantage of problem decomposition  decisions must be made on how to sequence the actions for all the subproblems though some are independent (in the air cargo problem, loading cargo at 2 different airports)  a least commitment strategy  would, where possible, delay choices during search  partially ordered planning implements this  adds actions to plans without committing to an absolute time/step, instead dealing mainly with relative constraints  action A must precede action B  partially ordered plans are generated by a search in the space of plans rather than through the state space  so the planning operators are different from the real-world actions Classical Planning 74

75. A plan to put your shoes on  an illustration from 2e notes Goal(RightShoeOn  LeftShoeOn) Init() Action(RightShoe, PRECOND: RightSockOn EFFECT: RightShoeOn) Action(RightSock, PRECOND: EFFECT: RightSockOn) Action(LeftShoe, PRECOND: LeftSockOn EFFECT: LeftShoeOn) Action(LeftSock, PRECOND: EFFECT: LeftSockOn) Classical Planning 75

76. A plan to put your shoes on  a partially-ordered planner can place two actions into a plan without specifying which comes first  sample POP and corresponding totally ordered plans Classical Planning 76

77. POP as a search problem  states are (mostly unfinished) plans  the empty plan contains only the special start & finish actions  each state (plan) is a partially ordered structure  it consists of a set of actions (the steps of the plan) and a set of ordering constraints of the form Before(a i, a j ) meaning that one action occurs before another  graphically, actions are shown as boxes, ordering constraints as arrows  the starting point is an empty plan consisting of the initial state & goal, with no actions between them  the search process adds an action to correct a flaw in the plan or if it can't, it backtracks  flaws keep a partial plan from being a solution  as an example, the next slide shows POPs corresponding to the "flat tire" planning problem, both the empty plan and a solution 77

78. POP for the flat tire problem  at the top the empty plan & below it, the solution plan  Remove(spare, trunk), Remove(flat, axle) can be done in either order, so long as both are done before PutOn(spare, axle) 78

79. POP for the flat tire problem  search in plan space seeks to add to the plan to correct a flaw  flaw: not achieving At(spare, axle) in the flat tire empty plan  adding PutOn(spare, axle) fixes that, but introduces new flaws in the form of its (unachieved) preconditions  process repeats, backtracking as necessary  each step makes the least commitment (ordering constraints & variable bindings) in order to fix a flaw  Remove(spare, trunk) must be before PutOn(spare, axle) but no other ordering requirements Classical Planning 79

80. Partial-Order Planning  was previously popular because of explicitly representing independent subproblems  but it lacks an explicit representation of states in the state- transition model & was superseded by progression planners with heuristics that allowed discovery of independent subproblems  nevertheless it's useful for tasks like operations scheduling, enhanced with domain specific heuristics  also appropriate where humans must understand the resulting plans  plan refinement approach is easier to understand & verify  example: spacecraft/rover plans are generated by this approach Classical Planning 80

81. Solving POP  POP in action  the initial plan contains  only the Start & Finish steps  Start step/action has the initial state description as its effect  Finish step/action has the goal description as its precondition  the single ordering constraint Before (Start, Finish)  there's a set of open preconditions: all those in Finish  the successor function  picks an open precondition p on an action B and generates a successor plan using an action A that achieves p  then does goal test  any remaining open preconditions? Classical Planning 81

82. Process summary  operators on partial plans may  add link from an existing action to an open precondition  add a step (a new action) to fulfill an open condition  order one step wrt another to remove possible conflicts  plan generation involves gradually moving  from incomplete/vague plans to complete/correct plans  plan construction may require backtracking  if an open condition is unachievable or conflict is unresolvable Classical Planning 82

83. Classical Planning Summary  approaches rely on techniques of search or logic  search to assemble a plan or to constructively prove one exists  handling the inherent complexity of these problems is a key  identifying independent subproblems provides potential for exponential speedup, but full decomposability is inconsistent with -ve interactions between actions  heuristically guided search can gain efficiency even if subproblems are not completely independent  some problems allow efficient solution by ruling out -ve interactions  serializable subgoals = there's a subgoal ordering for a solution that avoids having to undo any subgoals  tower problems in Blocks World are serializable if subgoals are ordered bottom to top 83

84. Classical Planning Summary  serializable subgoals  tower problems in Blocks World are serializable if subgoals are ordered bottom to top  NASA spacecraft control commands are serializable (unsurprising since the control systems are designed for easy control)  when problem is serializable, a planner can seriously prune the planning search  what's next? where does further progress come from?  probably not simply factored representations & propositional approaches  combine efficient heuristics + first-order, hierarchical representations Classical Planning 84

85. Classical Planning Summary  PDDL (Planning Domain Definition Language)  initial & goal states as conjunctions of literals  actions as lists of preconditions & effects  search in state space may be forward (Progression) or backward (Regression)  derive heuristics from subgoal independence assumption & relaxed versions of the problems  Planning Graphs  build from the initial state to the goal with alternating incremental sets of literals or actions representing possibilities for each time step, & encoding mutexes between inconsistent pairs of literals or actions  layers are supersets of literals/actions possible at that time step  the basis of good heuristics for state space searches or can yield a plan by application of the GRAPHPLAN algorithm Classical Planning 85

86. Classical Planning Summary  possible alternative approaches  FOL inference on a situation calculus knowledge base  conversion to boolean satisfiability problem for SATPlan  encoding as a Constraint Satisfaction Problem  search backwards over a space of Partially Ordered Plans  what's best?  there's no agreement on a single best representation or algorithm  we saw one hybrid that performs well  a particular domain may provide a best fit to one particular technique Classical Planning 86