1. Advanced Problem Solving Systems: Planning  In order to solve nontrivial problems, it is necessary to combine 1.Basic problem solving strategies 2.Knowledge representation mechanisms 3.Partial solutions and at the end combine into complete problem solution (decomposition)  Planning refers to the process of computing several steps of a problem solving before executing any of them.  Planning is useful as a problem solving technique for non decomposable problem.

2. Components of a Planning System  In any general problem solving systems, elementary techniques to perform following function are required 1. Choose the best rule (based on heuristics) to be applied 2. Apply the chosen rule to get new problem state 3. Detect when a solution has been found 4. Detect dead ends so that new directions are explored.  In more complex systems, we have to explore techniques for doing each of these tasks.  Detect when an almost correct solution has been found and employ special techniques to make it totally correct.  We will discuss planning methods using specific example of block world problem and explain ways in which each of above four things can be done.

3. 1. Choosing Rules to apply  Most widely used technique for selecting appropriate rules is to first isolate a set of differences between the desired goal state and current state.  Identify those rules that are relevant to reducing those difference.  If more rules are found then apply heuristic information to choose among them. 2. Apply Rules  In simple problem solving system, applying rules was easy as each rule specifies the problem state that would result from its application.  In complex problem we deal with rules that specify only a small part of the complete problem state.

4. Block World Problem Description Assumptions:  Flat surface on which blocks can be placed.  Number of square blocks of same size.  Blocks can be stacked one upon another.  Robot arm that can manipulate the blocks. It can hold only one block at a time.  In block world problem, the state is described by a set of predicates representing the facts that were true in that state.  One must describe for every action, each of the changes it makes to the state description.  In addition, some statements that everything else remains unchanged is also necessary.

5. Actions (Operations) performed by Robot arm UNSTACK (X, Y) : [US (X, Y)] Pick up X from its current position on block Y. The arm must be empty and X has no block on top of it. STACK (X, Y): [S (X, Y)] Place block X on block Y. Arm must holding X and the top of Y is clear. PICKUP (X): [PU (X) ] Pick up X from the table and hold it. Initially the arm must be empty and top of X is clear. PUTDOWN (X): [PD (X)] Put block X down on the table. The arm must have been holding block X.

6. Following predicates are used in order to perform above mentioned operations/actions ON (X, Y)-Block X is on block Y. ONTABLE(X)-Block X is on the table. [ONT (X)] CLEAR(X)-Nothing on top of X. [CL (X)] HOLDING(X)-arm is holding X. [HOLD (X)] ARMEMPTY-arm is empty. [AE] The following logical statements are true in this block world. 1. Holding X means, arm is not empty (  X) HOLD (X)  ~ AE 2. X is on a table means that X is not on the top of any block (  X) ONT (X)  ~ (  Y) ON (X, Y) 3. Any block with no block on has clear top (  X) (~ (  Y) ON (X, Y))  CLT (X)

7. The effect of US(X, Y) could be described by the following axiom [CL (X, State)  ON (X, Y, State)]  [HOLD(X, DO (US (X, Y), State))  CL (Y, DO (US(X, Y),State)) ]  DO is a function that generates a new state as a result of given action and a state.  Hence if we execute US (A,B) in S 0 then we can prove that HOLD (A, S 1 )  CLEAR (B, S 1 ) holds true, where S 1 is new state from doing Unstack operation.  But we do not know about other situations in new state S 1. For example, B is still on the table but we can’t derive it.  We have to provide a set of rules called frame axioms (components of the state that are not affected by US operator) such as ONT(Z, S)  ONT(Z, DO (US (A, B), S))

8. We also need to say that ON relation is only affected by US operator if the blocks involved in ON relation are the same involved in US operation. [ ON(M, N, S)  NE (M, X) ]  ON(M, N, DO (US (X, Y), S))  Advantage of this approach is that simple mechanism of resolution can perform all the operations that are required on the state descriptions.  Disadvantage is that number of axioms becomes very large for complex problem such that COLOR of block also does not change. So we have to specify rule for each attribute. COLOR (X, red, S)  COLOR (X, red, DO(US(Y, Z), s))

9.  To handle complex problem domain, we need a mechanism that does not require large number of explicit frame axioms.  One such mechanism was used in early robot problem solving system STRIPS (developed by Fikes, 1971).  In this approach, each operation is described by a list of new predicates that cause operator to be true and list of old predicates that it causes to become false.  These lists are called ADD and DELETE respectively.  A third list Pre_Cond is also specified for each operator that must be true for the operator to be applied.  Any predicate not included on either ADD or DELETE list of an operator is assumed to be unaffected by it.  Frame axioms are specified implicitly in STRIPS which greatly reduces amount of information stored.

10. STRIPS – Style Operators 1.S (X, Y) Pre:CL (Y)  HOLD (X) Del:CL (Y)  HOLD (X) Add:AE  ON (X, Y) 2.US (X, Y) Pre:ON (X, Y)  CL (X)  AE Del:ON (X, Y)  AE Add:HOLD (X)  CL (Y) 3.PU (X) Pre:ONT (X)  CL (X)  AE Del:ONT (X)  AE Add:HOLD (X) 4.PD (X) Pre:HOLD (X) Del:HOLD (X) Add:ONT (X)  AE

11. By making the frame axioms implicit we have greatly reduced the amount of information that must be provided for each operator. If a new attribute is introduced, it is not necessary to go back and add a new axioms for each operator. Here state is dropped for the sake of simplicity. If initial state is ON (X, Y)  ONT (Y)  CL (X)  AE then on applying US (X, Y), we get new state as ONT (Y)  CL (Y)  HOLD (X) Simply updating a state description works well as a way of keeping track of the effects of a given sequence of operators. If incorrect sequence is explored then it must be possible to return to the previous state so that different path can be explored.

12. Simple planning using a goal stack  One of the earliest techniques for solving compound goals that may interact, was the use of goal stack. This approach was used by STRIPS system.  Here problem solver makes use of a single stack that contains both goals and operators that have been proposed to satisfy those goals.  In Goal stack planning method, sub goals are solved linearly and then the conjoined sub goals is solved.  Plans generated by this method will contain complete sequence of operations for attaining one goal followed by complete sequence of operations for the next etc.  Problem solver also relies on  A database that describes the current situation.  Set of operators with precondition, add and delete lists.

13. Algorithm Let us assume that the goal to be satisfied is: GOAL = G 1  G 2  …  G n Subgoals G 1, G 2, … G n are stacked with compound goal G 1  G 2  …  G n at the bottom. TopG 1 G 2 : G n BottomG 1  G 2  …  G 4 At each step of problem solving process, the top goal on the stack is pursued.

14.  Find an operator that satisfies sub goal G 1 (makes it true) and replace G 1 by the operator.  If more than one operator satisfies the sub goal then apply some heuristic to choose one.  Now in order to execute the top most operations, its preconditions are added in the stack.  Once preconditions of an operator are satisfied, then we are guaranteed that operator can be applied to produce a new state.  New state is obtained by using ADD and DELETE lists of an operator to the existing database.  Problem solver keeps tract of operators applied.  This process is continued till the goal stack is empty and problem solver returns the plan of the problem.

15. Solving Block World problem using Goal stack method Consider the following initial and goal states Logical representation of Initial and Goal states are: Initial State: ON(B, A)  ONT(A)  ONT(C)  ONT(D)  CL(B)  CL(C)  CL(D)  AE Goal State: ON(C, A)  ON(B, D)  ONT(A)  ONT(D)  CL(C)  CL(B)  AE

16. We notice that ONT(A)  ONT(D)  CL(C)  CL(B)  AE in goal state is also true in initial state. Let us represent for the sake of simplicity, ONT(A)  ONT(D)  CL(C)  CL(B)  AE by TSUBG. We have to work only to satisfy ON(C, A) and ON(B, D) and in the process of satisfying, make sure that TSUBG remains true. There are two ways of solving it. Either start solving ON(C, A) or ON(B, D) first. Let us solve ON(C, A) first, so stack will look like as: Goal Stack: ON(C, A) ON(B, D) ON(C, A)  ON(B, D)  TSUBG

17. In order to solve ON(C, A), only operation S(C, A) could be applied. So replace ON(C, A) with S(C, A) in goal stack. Goal Stack: S (C, A) ON(B, D) ON(C, A)  ON(B, D)  TSUBG Stack operator S(C, A) can be applied only if its preconditions are true. So add its preconditions on the stack. Goal Stack: CL(A) HOLD(C)Preconditions of STACK CL(A) )  HOLD(C) S (C, A) Operator ON(B, D) ON(C, A)  ON(B, D)  TSUBG

18. State_0: ON(B, A)  ONT(A)  ONT(C)  ONT(D)  CL(B)  CL(C)  CL(D)  AE Next check if CL(A) is true in State_0. Since it is not true in State_0, only operator that could make it true is US(B, A). So replace CL(A) with US(B, A) and add its preconditions. Goal Stack: ON(B, A) CL(B) Preconditions of UNSTACK AE ON(B, A)  CL(B)  AE US(B, A)Operator HOLD(C) CL(A) )  HOLD(C) S (C, A) Operator ON(B, D) ON(C, A)  ON(B, D)  TSUBG

19. We see from State_0 that ON(B, A), CL(B) and AE are all true, so pop them along with its compound goal. Now pop US(B, A) and produce new state using its ADD and DELETE lists. Add US(B, A) in a queue of sequence of operators. SQUEUE = US (B, A) State_1: ONT(A)  ONT(C)  ONT(D)  HOLD(B)  CL(A)  CL(C)  CL(D) Goal Stack: HOLD(C) CL(A) )  HOLD(C) S (C, A) Operator ON(B, D) ON(C, A)  ON(B, D)  TSUBG Now satisfy the goal HOLD(C).

20. Two operators can be used 1.PU(C ) 2.US(C, X), where X could be any block Let us choose PU(C ) and proceed further. Goal Stack: ONT(C) CL(C) preconditions of AE ONT(C)  CL(C)  AE PU(C) Operator CL(A) )  HOLD(C) S (C, A) Operator ON(B, D) ON(C, A)  ON(B, D)  TSUBG

21. State_1: ONT(A)  ONT(C)  ONT(D)  HOLD(B)  CL(A)  CL(C)  CL(D) Here ONT(C ) and CL(C ) are true in State_1, so pop them off. AE is not true as arm is holding B, so in order to make it true do either S(B, X) or PD(B). Let us choose S(B, X). By looking ahead a bit, we see that we want B onto D. Let us unify X with D. Goal Stack: CL(D) HOLD (B)Preconditions of STACK CL(D)  HOLD(B) S (B, D)Operator ONT(C)  CL(C)  AE PU(C) Operator CL(A) )  HOLD(C) S (C, A) Operator ON(B, D) ON(C, A)  ON(B, D)  TSUBG

22. Now CL(D) and HOLD(B) are both true. Now pop S(B, D) and produce new state using its ADD and DELETE lists. Add S(B, D) in a queue of sequence of operators. Change the state. SQUEUE = US (B, A), S(B, D) State_2: ONT(A)  ONT(C)  ONT(D)  ON(B, D)  CL(A)  CL(C)  CL(B)  AE All conditions of PU (C ) are satisfied, so PU (C ) can be performed. POP it and add in queue of sequence of operators. Change the state. SQUEUE = US (B, A), S(B, D), PU(C ) A C BDBD

23. State_3: ONT(A)  HOLD(C)  ONT(D)  ON(B, D)  CL(A)  CL(B) Finally all preconditions of S(C, A) are true, so pop it and add to solution queue. SQUEUE = US (B, A), S(B, D), PU(C ), S(C, A) State_4: ONT(A)  ON(C, A)  ONT(D)  ON(B, D)  CL(C)  CL(B)  AE Goal Stack: ON(B, D) ON(C, A)  ON(B, D)  TSUBG From database, we see ON(B, D) is true and compound goal is still true, so pop these out. Problem solver returns the solution queue containing the sequence of operators to be applied as US (B, A), S(B, D), PU(C ), S(C, A) Heuristic information can be applied to guide the search process Some interaction among sub goals could help to produce a good overall solution.

24. Difficult Problem The Goal stack method is not adequate for more difficult problems. It fails to find good solution. Let us consider the following block world problem

25. Initial State: State_0: ON(C, A)  ONT(A)  ONT(B) Goal State: ON(A, B)  ON(B, C)  Let us remove CLEAR and AE predicates for the sake of simplicity.  For satisfying ON(A, B), we have to apply following operators US(C, A), PD(C), PU(A) and S(A, B)

26. State_1: ON(B, A)  ONT(C)  Now work for satisfying ON(B, C). The following operators are applied US(A, B), PD(A), PU(B) and S(B, C) State_2: ON(B, C)  ONT(A)

27.  Finally satisfy combined goal ON(A, B)  ON(B, C).  We fail as while satisfying ON(B, C), we have undone ON(A, B).  Difference in goal and current state is ON(A, B).  Operations required are PU(A) and S(A, B)

28.  Now combined goal is again checked and found to be satisfied.  The complete plan has the following operations in the solution sequence 1.US(C, A) 2.PD (C) 3.PU(A) 4.S(A, B) 5.US(A, B) 6.PD(A) 7.PU(B) 8.S(B, C) 9.PU(A) 10.S(A, B)  Although this plan will achieve the desired goal, but it is not efficient.

29.  In order to get efficient plan, either repair this plan or use some other method.  Repairing is done by looking at places where operations are done and undone immediately, such as S(A, B) and US(A, B).  By removing them, we get 1. US(C, A) 2.PD (C) 3.PU(A) 4.PD(A) 5.PU(B) 6.S(B, C) 7.PU(A) 8.S(A, B)  In new plan, we see that PU(A) and PD(A) are complimentary operations, so remove them also.

30.  We get the final plan as: 1.US(C, A) 2.PD (C) 3.PU(B) 4.S(B, C) 5.PU(A) 6.S(A, B)  Eventually, we got a final plan but wasted a great deal of problem solving efforts.  There exist another method called nonlinear planning which constructs efficient plans directly.

31. Nonlinear Planning using Goal Set  Generate a plan by doing some work on one goal, then some on another and then some more on the first one.  Such plans are called nonlinear plans as it is not composed of a linear sequence of complete sub plans.  Consider the same example done earlier using goal stack method.

32. A good plan for the solution of the above problem is the following. 1.Begin work on goal ON (A,B) by clearing A thus putting C on the table. 2.Achieve ON (B,C) by stacking B on C. 3.Complete the goal ON (A,B) by stacking A on B. One way to find such a plan is to consider the collection of desired goals as a set. Search procedure that operates is completely backward from the goal to the initial state, with no operators being actually applied along the way. Idea is to look first for the operator that will be applied last in final solution. It assumes that all but one of the sub goals have already been satisfied.

33.  Find all the operators that might satisfy the final sub goal (assumed all other sub goals are satisfied).  Each of them may have preconditions that must be met so that new combined goal must be generated including those preconditions as well as the original goals.  Many of the paths can be quickly be eliminated because of contradiction.  Consider the last sub goal to be proved is either ON(A, B) or ON(B, C).  If ON(A, B) is the last sub goal, then all its preconditions and ON(B, C) must be assumed to be true prior to this operation.  Backward chaining procedure is applied to this problem.  Whenever contradiction occurs in a goal set such as {HOLD (X), AE}, the path is pruned.  If the goal set contains false then also that path is pruned.

34.  When an operator is applied, it may cause the sub goals in a set no longer true.  So non selected goals are not directly copied to new goal set but a process called regression is applied.  Regression can be thought of as the backward application of operators.  Each goal is regressed through an operator whereby, we are attempting to determine what must be true before the operator is performed.  For example: i.Reg(ON(A, B), S(C, A)) = ON(A, B) ii.Reg(ON(A, B), S(A, B)) = true iii.Reg(ON(A, B), PU(C)) = ON(A, B) iv.Reg(AE, PD(A)) = true v. Reg(AE, S(X, Y)) = true vi.Reg(AE, PU(A)) = false vii.Reg(AE, US(A, B)) = false