1. Planning AIMA: 10.1, 10.2, 10.3 Follow slides and use textbook as reference

2. Subgoal interactions Suppose we have a set of subgoals G1,….Gn
Suppose the length of the shortest plan for achieving
the subgoals in isolation is l1,….ln
We want to know what is the length of the shortest plan for
achieving the n subgoals together, l1…n
If subgoals are independent: l1..n = l1+l2+…+ln
If subgoals have + interactions alone: l1..n < l1+l2+…+ln
If subgoals have - interactions alone: l1..n > l1+l2+…+ln
If you made “independence” assumption, and added up the individual costs of
subgoals, then your resultant heuristic will be
perfect if the goals are actually independent
inadmissible (over-estimating) if the goals have positive interactions
admissible if the goals have negative interactions

3. Scalability came from sophisticated reachability heuristics based on
planning graphs..
..and not from any hand-coded
domain-specific
control knoweldge
Total cost
incurred in search
Cost of computing
the heuristic
Cost of searching
with the heuristic h0
hset-difference hC h* hP
The scalability came from sophisticated reachability heuristics based on planning graphs.. That sounds quite ominous—but it is really AI 101..
Not always clear where the total minimum occurs
Old wisdom was that the global min was closer to cheaper heuristics
Current insights are that it may well be far from the cheaper heuristics for many problems
E.g. Pattern databases for 8-puzzle
Plan graph heuristics for planning

4. Planning Graph and Projectionpq pr ps
pqr
pqs
psq
pst A1 A2 A3 A4
Envelope of Progression Tree (Relaxed Progression)
Proposition lists: Union of states at kth level
Mutex: Subsets of literals that cannot be part of any legal state
Lowerbound reachability information p q r s t A1 A2 A3 A4
Planning Graphs can be used as the basis for
heuristics!
[Blum&Furst, 1995] [ECP, 1997][AI Mag, 2007]

5. Planning Graph Basics pqr Can be used for estimating non-reachability
Envelope of Progression Tree (Relaxed Progression)
Linear vs. Exponential Growth
Reachable states correspond to subsets of proposition lists
BUT not all subsets are states
Can be used for estimating non-reachability
If a state S is not a subset of kth level prop list, then it is definitely not reachable in k steps A2 A1 pq pq A3 A1
pqs A2 p pr A1
psq A3 A3 ps ps A4
pst p p q r s p q r s t A1 A1 A2 A2 A3 A3 A4

6. Don’t look at curved lines for now…
Graph has leveled off, when the prop list has not changed from the previous iteration
Have(cake)
~eaten(cake)
~Have(cake)
eaten(cake)
Eat
No-op
No-op
Have(cake)
eaten(cake)
bake
~Have(cake)
~eaten(cake)
Eat
Have(cake)
~eaten(cake)
Don’t look at curved lines
for now…
The note that the graph has leveled off now since the last two
Prop lists are the same (we could actually have stopped at the
Previous level since we already have all possible literals by step 2)

7. A B Blocks world Init: Ontable(A),Ontable(B),
Clear(A), Clear(B), hand-empty
Goal:
~clear(B), hand-empty
State variables:
Ontable(x) On(x,y) Clear(x) hand-empty holding(x)
Initial state:
Complete specification of T/F values to state variables
--By convention, variables with F values are omitted
Goal state:
A partial specification of the desired state variable/value combinations
--desired values can be both positive and negative
Pickup(x)
Prec: hand-empty,clear(x),ontable(x)
eff: holding(x),~ontable(x),~hand-empty,~Clear(x)
Putdown(x)
Prec: holding(x)
eff: Ontable(x), hand-empty,clear(x),~holding(x)
Unstack(x,y)
Prec: on(x,y),hand-empty,cl(x)
eff: holding(x),~clear(x),clear(y),~hand-empty
Stack(x,y)
Prec: holding(x), clear(y)
eff: on(x,y), ~cl(y), ~holding(x), hand-empty

8. h-A
h-B
Pick-A
Pick-B
~cl-A
~cl-B
~he
onT-A
onT-A
onT-B
onT-B
cl-A
cl-A
cl-B
cl-B he he

9. h-A
on-A-B
St-A-B
on-B-A
h-B
Pick-A
h-A
h-B
Pick-B
~cl-A
~cl-B
~cl-A
~cl-B
St-B-A
~he
onT-A
~he
onT-A
Ptdn-A
onT-A
onT-B
onT-B
onT-B
Ptdn-B
cl-A
cl-A
cl-A
Pick-A
cl-B
cl-B
cl-B
Pick-B he he he

10. Estimating the cost of achieving individual literals (subgoals)
Idea: Unfold a data structure called “planning graph” as follows:
1. Start with the initial state. This is called the zeroth level proposition list
2. In the next level, called first level action list, put all the actions whose
preconditions are true in the initial state
-- Have links between actions and their preconditions
3. In the next level, called first level proposition list, put:
Note: A literal appears at most once in a proposition list.
3.1. All the effects of all the actions in the previous level.
Links the effects to the respective actions.
(If multiple actions give a particular effect, have multiple
links to that effect from all those actions)
3.2. All the conditions in the previous proposition list (in this case
zeroth proposition list).
Put persistence links between the corresponding literals in the
previous proposition list and the current proposition list.
*4. Repeat steps 2 and 3 until there is no difference between two consecutive
proposition lists. At that point the graph is said to have “leveled off”
The next 2 slides show this expansion upto two levels

11. Using the planning graph to estimate the cost of single literals:
1. We can say that the cost of a single literal is the index of the first proposition level
in which it appears.
--If the literal does not appear in any of the levels in the currently expanded
planning graph, then the cost of that literal is:
-- l+1 if the graph has been expanded to l levels, but has not yet leveled off
-- Infinity, if the graph has been expanded
(basically, the literal cannot be achieved from the current initial state)
Examples:
h({~he}) = 1
h ({On(A,B)}) = 2
h({he})= 0
How about sets of literals?
see next slide

12. Estimating reachability of sets
We can estimate cost of a set of literals in three ways:
Make independence assumption
hsum(p,q,r)= h(p)+h(q)+h(r)
Define the cost of a set of literals in terms of the level where they appear together
h-lev({p,q,r})= The index of the first level of the PG where p,q,r appear together
so, h({~he,h-A}) = 1
Compute the length of a “relaxed plan” to supporting all the literals in the set S, and use it as the heuristic: hrelax

13. Neither hlev nor hsum work well alwaysP0 A0 P1
True cost of {p1…p100} is 1
(needs just one action reach)
Hlev says the cost is 1
Hsum says the cost is 100
Hlev better than Hsum q q B1 p1 B2 p2 B3 p3
B99
p99
B100
p100 P0 A0 P1 q q
True cost of {p1…p100} is 100
(needs 100 actions to reach)
Hlev says the cost is 1
Hsum says the cost is 100
Hsum better than Hlev p1 p2 B* p3
p99
p100

14. h-sum; h-lev; H-lev is admissible H-sum in not admissible
H-sum is larger than or equal to H-lev

15. Slides after this one are NOT required for homework and exams

16. Goal Interactions
To better account for - interactions, we need to start looking into feasibility of subsets of literals actually being true together in a proposition level.
Specifically,in each proposition level, we want to mark not just which individual literals are feasible,
but also which pairs, which triples, which quadruples, and which n-tuples are feasible. (It is quite possible that two literals are independently feasible in level k, but not feasible together in that level)
The idea then is to say that the cost of a set of S literals is the index of the first level of the planning graph, where no subset of S is marked infeasible
The full scale mark-up is very costly, and makes the cost of planning graph construction equal the cost of enumerating the full progres sion search tree.
Since we only want estimates, it is okay if talk of feasibility of upto k-tuples
For the special case of feasibility of k=2 (2-sized subsets), there are some very efficient marking and propagation procedures.
This is the idea of marking and propagating mutual exclusion relations.

17. Level-off definition? When neither propositions nor mutexes change between levels

18. Mutex Propagation Rules
Two actions a1 and a2 are mutex if any of the following is true:
Inconsistent effects: one action negates the effect of the other.
Interference: one of the effects of one action is the negation of a prediction of the other
Competing needs: one of the predictions of one action is mutually exclusive with a prediction of the other
Two propositions P1 and P2 are marked mutex if:
all actions supporting P1 are pair-wise mutex with all actions supporting P2.

19. h-A
h-B
Pick-A
Pick-B
~cl-A
~cl-B
~he
onT-A
onT-A
onT-B
onT-B
cl-A
cl-A
cl-B
cl-B he he

20. h-A on-A-B St-A-B on-B-A h-B Pick-A h-A h-B Pick-B ~cl-A ~cl-B ~cl-A
St-B-A
~he
onT-A
~he
onT-A
Ptdn-A
onT-A
onT-B
onT-B
onT-B
Ptdn-B
We want to try and make the level of a set of literals a good indicator of the difficulty of achieving them. One way is to introduce and propagate mutual exclusion (mutex) relations.
Here is how it goes. We know that at every time step we are really only going to do one non-no-op action. So, at the first level either pickup-A, or pickup-B or pickup-C are done. If one of them is done, the others can’t be. So, we put red-arrows to signify that each pair of actions are mutually exclusive.
Now, we can PROPAGATE the mutex relations to the proposition levels.
Rule 1. Two actions a1 and a2 are mutex if
both of the actions are non-noop actions or
a1 is a noop action supporting P, and a2 either needs ~P, or gives ~P.
some precondition of a1 is marked mutex with some precondition of a2
By this rule Pick-A is mutex with Pick-B. Similarly, the noop action he is mutex with pick-A.
Rule 2. Two propositions P1 and P2 are marked mutex if all actions supporting P1 are pair-wise mutex with all actions supporting P2.
By this rule, h-A and h-B are mutex in level 1 since the only action giving h-A is mutex with the only action giving h-B.
~cl(B) and he are mutex in the first level, but are not mutex in the second level (note that
~cl(B) is supported by a noop and stack-a-b (among others) in level 2. he is supported by stack-a-b, noop (among others). At least one action –stack-a-b supporting the first is non-mutex with one action—stack-a-b-- supporting the second.
cl-A
cl-A
cl-A
Pick-A
cl-B
cl-B
cl-B
Pick-B he he he

21. Level-based heuristics on planning graph with mutex relations
We now modify the hlev heuristic as follows
hlev({p1, …pn})= The index of the first level of the PG where p1, …pn appear together
and no pair of them are marked mutex.
(If there is no such level, then hlev is set to l+1 if the PG is expanded
to l levels, and to infinity, if it has been expanded until it leveled off)
This heuristic is admissible.
With this heuristic, we have a much better handle on both +
and - interactions. In our example, this heuristic gives the
following reasonable costs:
h({~he, cl-A}) = 1
h({~cl-B,he}) = 2
h({he, h-A}) = infinity
(because they will be marked mutex even in the final level of the
leveled PG)
Works very well in practice
H({have(cake),eaten(cake)}) = 2

22. How lazy can we be in marking mutexes?
We noticed that hlev is already admissible even without taking negative interactions into account
If we mark mutexes, then hlev can only become more informed
So, being lazy about marking mutexes cannot affect admissibility
However, being over-eager about marking mutexes (i.e., marking non-mutex actions mutex) does lead to loss of admissibility

23. Some observations about the structure of the PG
1. If an action a is present in level l, it will be present in
all subsequent levels.
2. If a literal p is present in level l, it will be present in all
subsequent levels.
3. If two literals p,q are not mutex in level l, they will never
be mutex in subsequent levels
--Mutex relations relax monotonically as we grow PG

24. Summary Planning and search Progression Regression
Planning graph and heuristics
Goal interactions and mutual exclusion