onT-A onT-B onT-C cl-A cl-C cl-B he Pick-A Pick-B Pick-C onT-A onT-B onT-C cl-A cl-C cl-B he h-A h-B h-C ~cl-A ~cl-B ~cl-C ~he st-A-B st-A-C st-B-A st-B-C pdn-A pdn-B pdn-C Pick-A onT-A onT-B onT-C cl-A cl-C cl-B he h-A h-B h-C ~cl-A ~cl-B ~cl-C ~he on-A-B on-B-C

Lecture of October 11th

Blocks world State variables: Ontable(x) On(x,y) Clear(x) hand-empty holding(x) Stack(x,y) Prec: holding(x), clear(y) eff: on(x,y), ~cl(y), ~holding(x), hand-empty Unstack(x,y) Prec: on(x,y),hand-empty,cl(x) eff: holding(x),~clear(x),clear(y),~hand-empty Pickup(x) Prec: hand-empty,clear(x),ontable(x) eff: holding(x),~ontable(x),~hand-empty,~Clear(x) Putdown(x) Prec: holding(x) eff: Ontable(x), hand-empty,clear(x),~holding(x) Initial state: Complete specification of T/F values to state variables --By convention, variables with F values are omitted Goal state: A partial specification of the desired state variable/value combinations Init: Ontable(A),Ontable(B), Clear(A), Clear(B), hand-empty Goal: ~clear(B), hand-empty

Progression: An action A can be applied to state S iff the preconditions are satisfied in the current state The resulting state S’ is computed as follows: --every variable that occurs in the actions effects gets the value that the action said it should have --every other variable gets the value it had in the state S where the action is applied Ontable(A) Ontable(B), Clear(A) Clear(B) hand-empty holding(A) ~Clear(A) ~Ontable(A) Ontable(B), Clear(B) ~handempty Pickup(A) Pickup(B) holding(B) ~Clear(B) ~Ontable(B) Ontable(A), Clear(A) ~handempty

Regression: A state S can be regressed over an action A (or A is applied in the backward direction to S) Iff: --There is no variable v such that v is given different values by the effects of A and the state S --There is at least one variable v’ such that v’ is given the same value by the effects of A as well as state S The resulting state S’ is computed as follows: -- every variable that occurs in S, and does not occur in the effects of A will be copied over to S’ with its value as in S -- every variable that occurs in the precondition list of A will be copied over to S’ with the value it has in in the precondition list ~clear(B) hand-empty Putdown(A) Stack(A,B) ~clear(B) holding(A) clear(B) Putdown(B)??

Lecture of October 16th

Announcements Exam on Thursday –Homepage may be up Wednesday (may be) –Temporary homepage alive and well Will try to shift lecture notes there..(may be) –Homework 2 and 3 should be returned today (so the TA told me…) –Extended office hours Monday (today) I will be available until 2pm Revision class Wednesday? –I can do it at 1pm MAKE SURE YOU ARE GETTING S THE LAST COUPLE OF DAYS.. –If not, come and see me so I can add your address to the temporary list

Heuristics to guide Progression/Regression Set difference heuristic Intution: The cost of a state is the number of goals that are not yet present in it. Progression: The cost of a state S is | G \ S | The number of state-variable value pairs in G which are not present in S Regression: The cost of a state S is | S \ I | The number of state-variable value pairs in S that are not present in the initial state Problems with Set difference heuristic: 1. Every literal is given the same cost. Some literals are harder to achieve than others! 2. It is assumed that the cost of achieving n-literals together is n This ignores the interactions between literals (“subgoals”). -- It may be easier to achieve a set of literals together than to achieve each of them separately (+ve interactions) -- It may be harder to achieve a set of literals together than to achieve them separately. (-ve interactions)

Subgoal interactions: Suppose we have a set of subgoals G 1,….G n Suppose the length of the shortest plan for achieving the subgoals in isolation is l 1,….l n We want to know what is the length of the shortest plan for achieving the n subgoals together, l 1…n If subgoals are independent: l 1..n = l 1 +l 2 +…+l n If subgoals have +ve interactions alone: l 1..n < l 1 +l 2 +…+l n If subgoals have -ve interactions alone: l 1..n > l 1 +l 2 +…+l n

Estimating the cost of achieving individual literals (subgoals) Idea: Unfold a data structure called “planning graph” as follows: 1. Start with the initial state. This is called the zeroth level proposition list 2. In the next level, called first level action list, put all the actions whose preconditions are true in the initial state -- Have links between actions and their preconditions 3. In the next level, called first level propostion list, put: Note: A literal appears at most once in a proposition list All the effects of all the actions in the previous level. Links the effects to the respective actions. (If multiple actions give a particular effect, have multiple links to that effect from all those actions) 3.2. All the conditions in the previous proposition list (in this case zeroth proposition list). Put persistence links between the corresponding literals in the previous proposition list and the current proposition list. *4. Repeat steps 2 and 3 until there is no difference between two consecutive proposition lists. At that point the graph is said to have “leveled off” The next 2 slides show this expansion upto two levels

onT-A onT-B cl-A cl-B he Pick-A Pick-B onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he

onT-A onT-B cl-A cl-B he Pick-A Pick-B onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he St-A-B St-B-A Ptdn-A Ptdn-B Pick-A onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he on-A-B on-B-A Pick-B

Using the planning graph to estimate the cost of single literals: 1. We can say that the cost of a single literal is the index of the first proposition level in which it appears. --If the literal does not appear in any of the levels in the currently expanded planning graph, then the cost of that literal is: -- l+1 if the graph has been expanded to l levels, but has not yet leveled off -- Infinity, if the graph has been expanded (basically, the literal cannot be achieved from the current initial state) Examples: h({~he}) = 1 h ({On(A,B)}) = 2 h({he})= 0

Estimating the cost of a set of literals (e.g. a state in regression search) Idea 0. [Max Heuristic] H max ({p,q,r..}) = max{h(p),h(q),….} Admissible, but very weak in practice Idea 2. [Sum Heuristic] Make subgoal independence assumption h ind ({p,q,r,...}) = h(p)+h(q)+h(r)+… Much better than set-difference heuristic in practice. --Ignores +ve interactions h({~he,h-A}) = h(~he) + h(h-A) = 1+1=2 But, we can achieve both the literals with just a single action, Pickup(A). So, the real cost is 1 --Ignores -ve interactions h({~cl(B),he}) = 1+0 = 1 But, there is really no plan that can achieve these two literals in this problem So, the real cost is infinity!

We can do a better job of accounting for +ve interactions if we define the cost of a set of literals in terms of the level h lev ({p,q,r})= The index of the first level of the PG where p,q,r appear together so, h({~he,h-A}) = 1 Interestingly, h lev is an admissible heuristic, even though h ind is not! (Prove) To better account for -ve interactions, we need to start looking into feasibility of subsets of literals actually being true together in a proposition level. Specifically, in each proposition level, we want to mark not just which individual literals are feasible, but also which pairs, which triples, which quadruples, and which n-tuples are feasible. (It is quite possible that two literals are independently feasible in level k, but not feasible together in that level) --The idea then is to say that the cost of a set of S literals is the index of the first level of the planning graph, where no subset of S is marked infeasible --The full scale mark-up is very costly, and makes the cost of planning graph construction equal the cost of enumerating the full progression search tree. -- Since we only want estimates, it is okay if talk of feasibility of upto k-tuples -- For the special case of feasibility of k=2 (2-sized subsets), there are some very efficient marking and propagation procedures. This is the idea of marking and propagating mutual exclusion relations.

Rule 1. Two actions a1 and a2 are mutex if (a)both of the actions are non-noop actions or (b) a1 is a noop action supporting P, and a2 either needs ~P, or gives ~P. (c) some precondition of a1 is marked mutex with some precondition of a2 Rule 2. Two propositions P1 and P2 are marked mutex if all actions supporting P1 are pair-wise mutex with all actions supporting P2.

onT-A onT-B cl-A cl-B he Pick-A Pick-B onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he

onT-A onT-B cl-A cl-B he Pick-A Pick-B onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he St-A-B St-B-A Ptdn-A Ptdn-B Pick-A onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he on-A-B on-B-A Pick-B

Here is how it goes. We know that at every time step we are really only going to do one non-no-op action. So, at the first level either pickup-A, or pickup-B or pickup-C are done. If one of them is done, the others can’t be. So, we put red-arrows to signify that each pair of actions are mutually exclusive. Now, we can PROPAGATE the mutex relations to the proposition levels. Rule 1. Two actions a1 and a2 are mutex if (a)both of the actions are non-noop actions or (b) a1 is a noop action supporting P, and a2 either needs ~P, or gives ~P. (c) some precondition of a1 is marked mutex with some precondition of a2 By this rule Pick-A is mutex with Pick-B. Similarly, the noop action he is mutex with pick-A. Rule 2. Two propositions P1 and P2 are marked mutex if all actions supporting P1 are pair-wise mutex with all actions supporting P2. By this rule, h-A and h-B are mutex in level 1 since the only action giving h-A is mutex with the only action giving h-B. ~cl(B) and he are mutex in the first level, but are not mutex in the second level (note that ~cl(B) is supported by a noop and stack-a-b (among others) in level 2. he is supported by stack-a-b, noop (among others). At least one action –stack-a-b supporting the first is non-mutex with one action— stack-a-b-- supporting the second.

Some observations about the structure of the PG 1. If an action a is present in level l, it will be present in all subsequent levels. 2. If a literal p is present in level l, it will be present in all subsequent levels. 3. If two literals p,q are not mutex in level l, they will never be mutex in subsequent levels --Mutex relations relax monotonically as we grow PG 1,2,3 imply that a PG can be represented efficiently in a bi-level structure: One level for propositions and one level for actions. For each proposition/action, we just track the first time instant they got into the PG. For mutex relations we track the first time instant they went away. A PG is said to have leveled off if there are no differences between two consecutive proposition levels in terms of propositions or mutex relations. --Even if you grow it further, no more changes can occur..

Level-based heuristics on planning graph with mutex relations h lev ({p 1, …p n })= The index of the first level of the PG where p 1, …p n appear together and no pair of them are marked mutex. (If there is no such level, then h lev is set to l+1 if the PG is expanded to l levels, and to infinity, if it has been expanded until it leveled off) We now modify the h lev heuristic as follows This heuristic is admissible. With this heuristic, we have a much better handle on both +ve and -ve interactions. In our example, this heuristic gives the following reasonable costs: h({~he, cl-A}) = 1 h({~cl-B,he}) = 2 h({he, h-A}) = infinity (because they will be marked mutex even in the final level of the leveled PG) Works very well in practice

Now that we anyway have PGs…. If there exists a k-length plan, it will be a subgraph of the k-length planning graph. (see the highlighted subgraph of the PG for our example problem) --Consider extracting the plan from the PG directly -- This search can be cast as a CSP Variables: literals in proposition lists Values: actions supporting them Constraints: Mutex and Activation constraints

~cl-B-2: { #, St-A-B-2, Pick-B-2} he-2: {#, St-A-B-2, St-B-A-2,Ptdn-A-2,Ptdn-B-2} h-A-1: {#, Pick-A-1} h-B-1: {#,Pick-B-1 } …. Constraints: he-2 = St-A-B-2 => h-A-1 !=# {activation} On-A-B-2 = St-A-B-2 => On-B-A-2 != St-B-A-2 {mutex constraints} Goals: ~cl-B-2 != # he-2 !=#

onT-A onT-B cl-A cl-B he Pick-A Pick-B onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he St-A-B St-B-A Ptdn-A Ptdn-B Pick-A onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he on-A-B on-B-A Pick-B

onT-A onT-B onT-C cl-A cl-C cl-B he Pick-A Pick-B Pick-C onT-A onT-B onT-C cl-A cl-C cl-B he h-A h-B h-C ~cl-A ~cl-B ~cl-C ~he st-A-B st-B-C pdn-A pdn-B pdn-C Pick-A onT-A onT-B onT-C cl-A cl-C cl-B he h-A h-B h-C ~cl-A ~cl-B ~cl-C ~he on-A-B on-B-C