1. Introduction Contents Sungwook Yoon, Postdoctoral Research Associate
Lecture material and Homework is available on course web page
The lectures are intended to give overview of planning research in AI
Smattering of recent progresses
Contents
Classical planning
Definitions
Basic algorithmic paradigms
Markov Decision Processes (MDPs)
Basic definitions and solution techniques

2. Homework 1
Due April 25
Available from Course Web-Page

3. Classical STRIPS Planning
Sungwook Yoon*
Classical planning assumptions
STRIPS action language
What is a planning problem?
complexity of planning
Planning as Search
State-space search
* Based in part on slides by Alan Fern and Daniel Weld.

4. World AI Planning Percepts Actions ????
sole source of change vs. other sources
perfect vs. noisy
????
fully observable vs. partially observable
deterministic vs. stochastic
instantaneous vs. durative

5. Classical Planning Assumptions
Percepts
Actions
World
sole source of change
perfect
????
deterministic
fully observable
instantaneous

6. Representing States World states are represented as sets of facts.
We will also refer to facts as propositions.
holding(A)
clear(B)
on(B,C)
onTable(C)
State 1
handEmpty
clear(A)
on(A,B)
on(B,C)
onTable(C)
State 2 A A B B C C
Closed World Assumption (CWA): Fact not listed in a state are assumed to be false. Under CWA we are assuming the agent has full observability.

7. Representing Goals Goals are also represented as sets of facts.
For example { on(A,B) } is a goal in the blocks world.
A goal state is any state that contains all the goal facts.
handEmpty
clear(A)
on(A,B)
on(B,C)
onTable(C)
State 1
holding(A)
clear(B)
on(B,C)
onTable(C)
State 2 A A B B C C
State 1 is a goal state for the goal { on(A,B) }. State 2 is not a goal state.

8. Representing Action in STRIPS
handEmpty
clear(A)
on(A,B)
on(B,C)
onTable(C)
State 2
holding(A)
clear(B)
on(B,C)
onTable(C)
State 1
Stack(A,B) A B A C B C
A STRIPS action definition specifies: 1) a set PRE of preconditions facts 2) a set ADD of add effect facts 3) a set DEL of delete effect facts
Stack(x,y):
PRE: { holding(x), clear(y) } ADD: { on(x,y), handEmpty } DEL: { holding(x), clear(y) }
Stack(A,B):
PRE: { holding(A), clear(B) } ADD: { on(A,B), handEmpty } DEL: { holding(A), clear(B) }
x←A
x←B

9. Semantics of STRIPS Actions
handEmpty
clear(A)
on(A,B)
on(B,C)
onTable(C)
S  ADD – DEL
holding(A)
clear(B)
on(B,C)
onTable(C) S
Stack(A,B) A A B B C C
A STRIPS action is applicable (or allowed) in a state when its preconditions are contained in the state.
Taking an action in a state S results in a new state S  ADD – DEL (i.e. add the add effects and remove the delete effects)
Stack(A,B):
PRE: { holding(A), clear(B) } ADD: { on(A,B), handEmpty } DEL: { holding(A), clear(B) }

10. STRIPS Planning Problems
A STRIPS planning problem specifies: 1) an initial state S 2) a goal G ) a set of STRIPS actions
Objective: find a “short” action sequence reaching a goal state, or report that the goal is unachievable
Example Problem:
Solution: (Stack(A,B))
holding(A)
clear(B)
onTable(B)
Initial State A
on(A,B)
Goal B
Stack(A,B):
PRE: { holding(A), clear(B) } ADD: { on(A,B), handEmpty } DEL: { holding(A), clear(B) }
Stack(B,A):
PRE: { holding(B), clear(A) } ADD: { on(B,A), handEmpty } DEL: { holding(B), clear(A) }
STRIPS Actions

11. Properties of Planners
A planner is sound if any action sequence it returns is a true solution
A planner is complete outputs an action sequence or “no solution” for any input problem
A planner is optimal if it always returns the shortest possible solution
Is optimality an important requirement?
Is it a reasonable requirement?

12. Complexity of STRIPS Planning
PlanSAT Given: a STRIPS planning problem Output: “yes” if problem is solvable, otherwise “no”
PlanSAT is decidable.
Why?
In general PlanSAT is PSPACE-complete! Just finding a plan is hard in the worst case.
even when actions limited to just 2 preconditions and 2 effects
Does this mean that we should give up on AI planning?
NOTE: PSPACE is set of all problems that are decidable in polynomial space PSPACE-complete is believed to be harder than NP-complete

13. Satisficing vs. Optimality
While just finding a plan is hard in the worst case, for many planning domains, finding a plan is easy.
However finding optimal solutions can still be hard in those domains.
For example, optimal planning in the blocks world is NP-complete.
In practice it is often sufficient to find “good” solutions “quickly” although they may not be optimal.
For example, finding sub-optimal blocks world solutions can be done in linear time. How? ?

14. Propositional Planners
For clarity we have written propositions such as on(A,B) in terms of objects (e.g. A and B) and predicates (e.g. on).
However, the planners we will consider ignore the internal structure of propositions such as on(A,B).
Such planners are called propositional planners as opposed to first-order or relational planners
Thus it will make no difference to the planner if we replace every occurrence of “on(A,B)” in a problem with “prop1” (and so on for other propositions)
It feels wrong to ignore the existence of objects. But currently propositional planners are the state-of-the-art.
holding(A)
clear(B)
onTable(B)
Initial State
prop2
prop3
prop4
Initial State
on(A,B)
Goal
prop1
Goal

15. STRIPS Action Schemas
For convenience we typically specify problems via action schemas rather than writing out individual STRIPS action.
Action Schema: (x and y are variables)
Stack(B,A):
PRE: { holding(B), clear(A) } ADD: { on(B,A), handEmpty } DEL: { holding(B), clear(A) }
Stack(x,y):
PRE: { holding(x), clear(y) } ADD: { on(x,y), handEmpty } DEL: { holding(x), clear(y) }
Stack(A,B):
PRE: { holding(A), clear(B) } ADD: { on(A,B), handEmpty } DEL: { holding(A), clear(B) }
Each way of replacing variables with objects from the initial state and goal yields a “ground” STRIPS action.
Given a set of schemas, an initial state, and a goal, propositional planners compile schemas into actions and then ignore the existence of objects thereafter.

16. Planning as Search Nearly all classical planners are search procedures
Different planners use different search spaces
Two examples:
State-space planning
Each search node represents a state of the world
Search finds a path through the space (a plan)
Plan-space planning
Each search node is a “partial plan” giving constraints on a plan
Search adds more constraints until we get a complete plan
We will cover classical planners in historical order.

17. State-Space Search Search Nodes: states Search Arcs: STRIPS actions
Solution: path from the initial state to one state that satisfies the goal
Initial state is fully specified
There can be many goal states

18. Search Space: Blocks World
Search space is finite.

19. Forward-Chaining Search
goal
initial state
Breadth-first and best-first search are sound and complete
Very large branching factor can cause search to waste time and space trying many irrelevant actions
O(bd) worst-case where b = branching factor, d = depth limit
Need a good heuristic function and/or pruning procedure
Early AI researchers gave up on forward search.
But there has been a recent resurgence. More on this later in the course.

20. Backward-Chaining Search
goal
initial state
Backward search can focus on more “goal relevant” actions, but still the branch factor is typically huge
Again a good heuristic function and/or pruning procedure
Early AI researchers gave up on forward and backward search.
But there has been recent progress in developing general planning heuristics leading to a resurgence. More on this later in the course.

21. Total-Order vs. Partial-Order Planning (POP)? A C B D B D A C
There are many possible plans:
move(A, B, TABLE) ; move(B, TABLE, A) ; move(C, D, TABLE) ; move(D, TABLE, C)
move(A, B, TABLE) ; move(C, D, TABLE) ; move(D, TABLE, C) ; move(B, TABLE, A)
move(C, D, TABLE) ; move(D, TABLE, C) ; move(A, B, TABLE) ; move(B, TABLE, A)
ect . . .
State-space planning techniques produce totally-ordered plans, i.e. plans consisting of a strict sequence of actions.
Often, however, there are many possible orderings of actions than have equivalent effects.
However, often many orderings of the actions have equivalent effects.

22. Total-Order vs. Partial-Order Planning (POP)? A C B D B D A C
There are many possible plans:
move(A, B, TABLE) ; move(B, TABLE, A) ; move(C, D, TABLE) ; move(D, TABLE, C)
move(A, B, TABLE) ; move(C, D, TABLE) ; move(D, TABLE, C) ; move(B, TABLE, A)
move(C, D, TABLE) ; move(D, TABLE, C) ; move(A, B, TABLE) ; move(B, TABLE, A)
ect . . .
These plans share some common structure. They are all different interleavings of two separate plans:
1) move(A, B, TABLE) ; move(B, TABLE, A)
2) move(C, D, TABLE) ; move(D, TABLE, C)
A partial-order plan is one which specifies only the necessary ordering information. One partial-order plan may have many total-orderings

23. Total-Order vs. Partial-Order Planning (POP)

24. (Partial-Order) Plan-Space Search
Search nodes are partial plans
some plan steps may be missing
some orderings of actions may not be finalized
Search arcs are plan refinements
Solution is a node (not a path) giving the a partial order on actions.
Any linearization of the plan is a total-order solution to the problem
It follows the least-commitment principle:
Do not add constraints (e.g. action ordering) to a plan until it becomes necessary to ensure the correctness of the plan.

25. Representing Partial-Order Plans
Plan = (A, O, L, P), where
A: set of actions in the plan
O: temporal orderings between actions (a < b)
L: causal links linking actions via a literal (used for book keeping)
P: open preconditions that need to be satisfied
Causal Link:
Action A2 has precondition Q that is established in the plan by action A1. A1 A2 Q
clear(b)
Unstack(C,B)
Putdown(A,B)

26. Threats to a Causal Link
A causal link (A1, Q, A2) represents the assertion that the role of A1 is to establish proposition Q for A2
This tells future search steps to “protect” Q in the interval between A1 and A2
Action B threatens causal link (A1, Q, A2) if:
1. B has Q as a delete effect, and
2. B could come between A1 and A2, i.e.
O  (A1 < B < A2) is consistent
What is a threat to the following causal link?
PutDown(C,B)
clear(B)
Unstack(C,B) PutDown(A,B)

27. POP example: Blocks World
Initial plan includes only START and FINISH actions. The open preconditions correspond to the goal.
START
on(C, A)
ontable(A)
ontable(B)
clear(C)
clear(B) C B A
on(A, B)
on(B, C)
FINISH A B C

28. Work on open precondition on(B, C)
and clear(B)
START
on(C, A)
ontable(A)
ontable(B)
clear(C)
clear(B) C B A
preconditions
A1: move(B, TABLE, C)
on(B, C)
~ontable(B)
~clear(C)
clear(B)
clear(C)
ontable(B)
effects
on(A, B)
on(B, C)
FINISH A B C

29. Work on open precondition on(A, B)
START
on(C, A)
ontable(A)
ontable(B)
clear(C)
clear(B) C B A
A1: move(B, TABLE, C)
on(B, C)
~ontable(B)
~clear(C)
clear(B)
clear(C)
ontable(B)
clear(A)
clear(B)
ontable(A)
A2: move(A, TABLE, B)
~ontable(A)
~clear(B)
on(A, B)
on(A, B)
on(B, C)
FINISH A B C

30. Protect causal links (START, clear(B), A1) theatened by A2
on(C, A)
ontable(A)
ontable(B)
clear(C)
clear(B) C B A
(START, clear(B), A1) theatened by A2
--> Promote A2
A1: move(B, TABLE, C)
on(B, C)
~ontable(B)
~clear(C)
clear(B)
clear(C)
ontable(B)
clear(A)
clear(B)
ontable(A)
A2: move(A, TABLE, B)
~ontable(A)
~clear(B)
on(A, B)
on(A, B)
on(B, C)
FINISH A B C

31. Work on open precondition clear(A)
and protect links
START
on(C, A)
ontable(A)
ontable(B)
clear(C)
clear(B) C B A
A3: move(C, A, Table)
clear(C)
on(C, A)
~on(C, A)
ontable(C)
clear(A)
A1: move(B, TABLE, C)
on(B, C)
~ontable(B)
~clear(C)
clear(B)
clear(C)
ontable(B)
clear(A)
clear(B)
ontable(A)
A2: move(A, TABLE, B)
~ontable(A)
~clear(B)
on(A, B)
on(A, B)
on(B, C)
FINISH A B C

32. Final Plan
All preconditions are supported and plan is consistent. So this partial-order plan is a solution.
START
on(C, A)
ontable(A)
ontable(B)
clear(C)
clear(B) C B A
on(C, A)
clear(C)
A3: move(C, A, Table)
~on(C, A)
ontable(C)
clear(A)
clear(B)
clear(C)
ontable(B)
A1: move(B, TABLE, C)
clear(A)
clear(B)
ontable(A)
A2: move(A, TABLE, B)
~ontable(B)
~clear(C)
on(B, C)
~ontable(A)
~clear(B)
on(A, B)
on(A, B)
on(B, C)
FINISH A B C