1. Control Rules in Planning
Stephen M. Lee-Urban
March 26, 2007

2. References
Chapter 10, “Control Rules in Planning”, in Automated Planning Theory and Practice
F. Bacchus and F. Kabanza. “Using Temporal Logics to Express Search Control Knowledge for Planning”, in: Artificial Intelligence, vol 116, 2000
Planner: TLPlan
(For)Warning: Understanding temporal logic will be necessary for encoding control rules in your domains

3. u is a structured collection of actions and constraints
Motive
Classical planning efficiency suffers from combinatorial complexity (intractable)
Most earlier planners fit in Abstract-Search Procedure
ND search in node space of set of solution plans (set of all reachable solutions from n)
Prune function detects and cuts unpromising nodes
Can improve solving: exponential to polynomial
Same idea used in F.E.A.R.
Uses domain specific rules to guide forward chaining algorithm
u is a structured collection of actions and constraints

4. Pruning Often involves domain-specific tests Motivating Example:
Identify less desirable solutions below node than solutions below a different node
Motivating Example:
Forward Search + Prune in Container-Stacking
“Consistent”  all containers below c consistent with g
State Space. Long plans worse than shorter
If container c position is consistent with goal, prune what?
Prune states resulting from apply action moving c
If c’s position inconsistent, action a yields state consistent with goal, and action b moves c to position inconsistent w/ g
Prune states resulting from applying action b
Finds near optimal solutions in low-order polynomial time
How do we express such relationships between states?

5. Simple Temporal Logic: υ □ ◊ ○
STL extends FOL to include “modal operators”
Modal Operators express relationships between current state and subsequent states
Syntax: If L is a func-free FOL, then Lt includes all of L plus:
true, false: constant atoms that are always true/false
Modal Ops: What do we need?
υ (until), □ (always), ◊ (eventually), ○ (next), GOAL
If Φ1 and Φ2 are formulas, then so are
Φ1υΦ2, □Φ1, ◊Φ1, ○Φ1
If Φ1 has no modal ops, then GOAL(Φ1) is a formula

6. Semantics of STL Interpreting requires triple (S, si, g)
S = <s0, s1, …> is an infinite sequence
si S is the current state
g is a goal formula
If Lt is based on L of planning domain, and S is sequence of states produced by finite plan, isn’t S finite?
Use NOPs. Append infinite final state.
In coming slides, let Φ be an STL formula. We now define whether (S, si, g) |= Φ

7. But First…. Entailment |=
Aka “logical implication”
A |= B
A entails B iff every model that makes A true also makes B true
Example
A := {“All horses are animals”, “All stallions are horses”}
B := {“All stallions are animals”}

8. Definitions Given ground atom Φ, (S, si, g) |= Φ iff si |= Φ
υ - until
□ - always
◊ - eventually
○ - next
Definitions
Given ground atom Φ, (S, si, g) |= Φ iff si |= Φ
Eg: (S, ((on a b) (on b c) …), g) |= (on a b)
Quantifiers and logical connectives have the same semantic rules as in FOL
Eg (S, si, g) |= Φ1Φ2 iff (S, si, g) |= Φ1 and (S, si, g) |= Φ2
(S, si, g) |= □Φ iff
(S, sk, g) |= Φ for k i
(S, si, g) |= ○Φ iff
(S, si+1, g) |= Φ
(S, si, g) |= ◊Φ iff
k | k  i, (S, sk, g) |= Φ
(S, si, g) |= Φ1υΦ2 iff
k | k  i, (S, sm, g) |= Φ1 for m = i, …, k-1 and (S, sk, g) |= Φ2
(S, si, g) |= GOAL(Φ) iff
Φg
For diamond, the book says in words “sk is after si” but their math says >=
If Φ contains no GOAL operators, g is irrelevant. Write simply (S, si) |= Φ

9. Simple Examples Each means the same for s2
υ - until
□ - always
◊ - eventually
○ - next
Simple Examples
Each means the same for s2
s2 |= on(c1,c2)  on(c2,c3)
(S,s0) |= ○○(on(c1,c2)  on(c2,c3))
(S,s1) |= ○(on(c1,c2)  on(c2,c3))
(S,s2) |= on(c1,c2)  on(c2,c3)
(S,si) |= □holding(crane1,c1)
Holding(crane1,c1) is false for all k  i
(S,si) |= x ( on(x,c1)  □on(x,c1) )
The same container is on c1 in all subsequent states
(S,si) |= x □( on(x,c1)  ○on(x,c1) )

10. Encoding our Motivating Ex.
υ - until
□ - always
◊ - event.
○ - next
Encoding our Motivating Ex.
“Don’t move container if position is consistent with goal”
Φ1(c,d,p) = [GOAL(in(c,p))  q GOAL(in(c,q))]  [GOAL(on(c,d)  e GOAL(on(c,e))]
Holds if acceptable when container c is on item d in pile p (no goal requiring c in another pile or on top of something else)
Φ2(c,p) = ok(c,p) [same(p,pallet)  d (Φ1(c,d,p)  ok(d,p))]
ok(c,p) holds iff c is in pile p and c’s position is consistent with the goal
Φ3(c) = p (Φ2(c,p)  ok(c,p))
holds iff c’s position is consistent with the goal
Φ = c [Φ3(c)  pd □(in(c,p)  in(c,d))]
holds iff for every container c whose position is consistent with g, c will never be moved (c always remains in same pile and on same item in that pile)
Φ1 Holds if acceptable when container c is on item d in pile p (no goal requiring c in another pile or on top of something else)
Φ2 says ok(c,p) holds iff c is in pile p and c’s position is consistent with the goal
Φ3 holds iff c’s position is consistent with the goal
Φ holds iff for every container c whose position is consistent with g, c will never be moved (c always remains in same pile and on same item in that pile)

11. Break
Yum, dinner 

12. Progression
Computing progression of a control formula Φ is essential for pruning
Formula progress(Φ,si) is true in si+1 iff Φ is true in si. This is called Φ’s progression.
progress(Φ,si) is the formula produced from progr(Φ,si) by performing usual simplifications
Replace true  d with d, true with false, etc.
Both functions can be computed in low-order polynomial time with algorithms directly implementing their definitions
(S,si,g) |= Φ iff (S,si+1,g) |= progress(Φ,si)

13. Definition of progr(Φ,si)
υ - until
□ - always
◊ - eventually
○ - next
Definition of progr(Φ,si)
If Φ contains no modal operators, then
progr(Φ,si) = true if si |= Φ, false if si | Φ
Logical connectives are as usual
progr(Φ1Φ2,si) = progr(Φ1,si)  progr(Φ2,si)
progr(Φ,si) = progr(Φ,si)
Modal operators are as follows:
progr(○Φ,si) = Φ
progr(Φ1υΦ2,si) = ( (Φ1υΦ2)  progr(Φ1,si) )  progr(Φ2,si)
progr(◊Φ,si) = (◊Φ)  progr(Φ,si)
progr(□Φ,si) = (□Φ)  progr(Φ,si)

14. Example of Progr Φ = □on(c1,c2) si = on(c1,c2) progress(Φ,si) =
υ - until
□ - always
◊ - eventually
○ - next
Example of Progr
progr(□Φ,si) = (□Φ)  progr(Φ,si)
Φ = □on(c1,c2)
si = on(c1,c2)
progress(Φ,si) =
= □on(c1,c2)  progress(on(c1,c2),si)
= □on(c1,c2)  true
= Φ
What if si = on(c1,c2)?

15. Using Control Formulas in Planning
Let S = <s0,s1,…> be infinite and Φ be an STL formula.
If (S,s0,g) |= Φ, then for every finite truncation S’ = <s0,s1,…,si> of S, progress(Φ,S’)  false
Let s0 be a state,  be a plan applicable to s0, and S = <s0,…,sn> be the seq. of states produced by applying  to s0.
If Φ is an STL formula and progress(Φ,S) = false, then S has no extension S’ = <s0,…,sn, sn+1,sn+2,…> such that (S’,s0,g) |= Φ
Modify Forward-search to prune any partial plan  such that progress(Φ,S) = false

16. υ - until
□ - always
◊ - eventually
○ - next
Our Handy Example
Let s0 and g be for a container-stacking problem with constant symbols c1, …, ck. Let Φ1, Φ2, Φ3, and Φ be as before.
progress(Φ,s0) =
= progress(c [Φ3(c)  pd □(in(c,p)  in(c,d))], s0)
= progress([Φ3(c1)  pd □(in(c1,p)  in(c1,d))], s0)  …  progress([Φ3(ck)  pd □(in(ck,p)  in(ck,d))], s0)
Suppose in s0, c1 is consistent with g and is on item d1 in pile p1. Then
s0 |= Φ3(c1)
s0 | Φ3(ci) for i = 2, …, k
Which means progress(Φ,s0) =
= progress( pd □(in(c1,p)  in(c1,d)), s0 )
= progress( □(in(c1,p1)  in(c1,d1)), s0 )
= □(in(c1,p1)  in(c1,d1))
If an applicable action a to state s0 moves c1 then
(s0,a) | progress(Φ,s0)
Thus (s0,a) can be pruned.

17. Finally, the Planning Procedure!

18. Planning Procedure - Comments
Sound and complete, if the problem is solvable and STL formula Φ is entailed for at least one solution of the problem
Soundness follows from soundness of Forward-Search
Completeness follows from what?
the condition on Φ
Control formulas are like specialized computer programs and must be debugged.

19. Extensions Function Symbols Axioms (Horn-clauses) Attached Procedures
Restrict axioms to Horn-clauses and use a Horn-clause theorem prover
Attached Procedures
Allow some func./predicate symbols to be evalutated as attached procedures
Time
Actions with time durations and overlapping
Extended Goals
Add control rules like Φ = □at(r1,bad-loc)
Reach goal in 2 actions or fewer Φ = g V (○g)

20. Defined in Problem File
TLPlan
On vega.cc.lehigh.edu: /home/sml3/planning/sys/tlplan
Requires one input “script” file referring to two others:
(load-file "BlocksWorld.tlp")
(load-file "BlocksProblems.tlp")
(set-statistics-file "BlocksProblems.csv")
(set-goal (goal0))
(set-initial-world (state0))
(set-plan-name "Problem0")
(plan) ;; try and solve the problem
(select-final-world) ;; needed for next line
(print-pddl-plan)
(exit) ;;remove this line for interactive mode
Running: from the directory containing your domain…
../runtlplan script.tlp
Domain File
Problem File
Defined in Problem File
Also in tlplan.log
or make a symbolic link in your domain directory via: ln –s <path>/runtlplan

21. Sample Domain File Problem File
(clear-world-symbols) ;Remove old dom symb
;;; WORLD SYMBOLS
(declare-described-symbols
(predicate on 2)
;…and so on
(predicate ontable 1))
(declare-defined-symbols
(predicate goodtower 1)
(predicate goodtowerabove 1)
(function depth 1))
;;; DEFINED PREDICATES
(def-defined-predicate (goodtower ?x)
(and (clear ?x)
(goodtowerbelow ?x)))
;;; TEMPORAL CONTROL FORMULA
(define (bw-control1)
(always
(forall (?x) (clear ?x)
(implies (goodtower ?x)
(next (goodtowerabove ?x)))
)))
;;; OPERATORS
(def-strips-operator (pickup ?x)
(pre
(handempty) (clear ?x) (ontable ?x))
(add
(holding ?x))
(del
(handempty) (clear ?x) (ontable ?x)))
;;; PRINT ROUTINES and FUNCTIONS
Problem File
(define (state0)
(clear a) (clear b) (clear c)
(ontable a) (ontable b) (ontable c)
(handempty))
(define (goal0)
(on a b) (on b c) (ontable c)) …

22. Defining a Domain: Init/Def
(clear-world-symbols)
Must call first in a new domain definition file
Clears the prev. domain's language definition
Resets temporal control formula and the print world command to their defaults
(declare-described-symbols (function|predicate name arity [no-cycle-check|rewritable]) ...)
must be declared prior to any other symbols
(declare-defined-symbols (function|predicate|generator name arity) ...)
must be declared after the described symbols
(def-defined-predicate (name parameters) (local-vars declarations) formula)
Includes a new predicate, defined in terms of a FO formula involving other predicates
Can be recursive
(define name list)
name is an abbreviation for list. Allows macro subst. in domain definition files.
Use to define temporal control formulas
(:= ?variable value)
assigns value to ?variable

23. Formula Syntax
Terms constant – symbol, number, or string ?variable – any symbol starting with “?” (?array index …) – dimension must agree with args (function term …) – built-in or declared w/ init. decl.
Atomic Formulas
(predicate term …) – predicate declared w/ init. decl. (= term1 term2) – predefined equality binary predicate (TRUE) – const. atomic formula; always true
(FALSE) – const. atomic formula; always false
First-Order Formulas
atomic-formula
(and formula …) -- 
(or formula …) – 
(xor formula …) – 
(not formula …) – 
(implies formula1 formula2) -- 
(if-then-else formula1 formula2 formula3)
(forall var-gen … [formula]) -- 
(exists var-gen … [formula]) – 
(exists! var-gen … [formula]) – unique  (exactly one)
Temporal Logic Formulas (TF) first-order-formula (next tf) -- ○ (eventually tf) – ◊ (t-eventually ispec tf) – ispec is an interval of states (always tf) – □ (t-always ispec tf) – ispec is again an interval (t-until ispec tf1 tf2) – υ
Modalities (goal formula | tf | generator) – eval in goal world (previous formula | tf | generator) – eval in prev. state (current formula | tf | generator) – eval in cur. state

24. Defining Operators
STRIPS Operator (def-strips-operator name pre del cost duration priority) ;; where… (name v …) – Declares op name and params (pre formula) – Precondition list (add p …) – Add list (del p …) – Delete list (cost n) – Cost of action, default is 1 (duration n) – Duration of action, default 1 (priority n) – Used in search to order successor states, default 0
(def-strips-operator (pickup ?x)
(pre
(handempty) (clear ?x) (ontable ?x))
(add
(holding ?x))
(del
(handempty) (clear ?x) (ontable ?x)))

25. Defining TL Control Formulas
(define name list)
(define (bw-control1)
(always
(forall (?x) (clear ?x)
(implies (goodtower ?x)
(next (goodtowerabove ?x)))
)))

26. Contact Me
Start early, defining control rules is tricky, REQUIRES DEBUGGING and tuning