Control Rules in Planning Stephen M. Lee-Urban March 26, 2007

References Chapter 10, “Control Rules in Planning”, in Automated Planning Theory and Practice F. Bacchus and F. Kabanza. “Using Temporal Logics to Express Search Control Knowledge for Planning”, in: Artificial Intelligence, vol 116, 2000 Planner: TLPlan (For)Warning: Understanding temporal logic will be necessary for encoding control rules in your domains

u is a structured collection of actions and constraints Motive Classical planning efficiency suffers from combinatorial complexity (intractable) Most earlier planners fit in Abstract-Search Procedure ND search in node space of set of solution plans (set of all reachable solutions from n) Prune function detects and cuts unpromising nodes Can improve solving: exponential to polynomial Same idea used in F.E.A.R. Uses domain specific rules to guide forward chaining algorithm u is a structured collection of actions and constraints

Pruning Often involves domain-specific tests Motivating Example: Identify less desirable solutions below node than solutions below a different node Motivating Example: Forward Search + Prune in Container-Stacking “Consistent”  all containers below c consistent with g State Space. Long plans worse than shorter If container c position is consistent with goal, prune what? Prune states resulting from apply action moving c If c’s position inconsistent, action a yields state consistent with goal, and action b moves c to position inconsistent w/ g Prune states resulting from applying action b Finds near optimal solutions in low-order polynomial time How do we express such relationships between states?

Simple Temporal Logic: υ □ ◊ ○ STL extends FOL to include “modal operators” Modal Operators express relationships between current state and subsequent states Syntax: If L is a func-free FOL, then Lt includes all of L plus: true, false: constant atoms that are always true/false Modal Ops: What do we need? υ (until), □ (always), ◊ (eventually), ○ (next), GOAL If Φ1 and Φ2 are formulas, then so are Φ1υΦ2, □Φ1, ◊Φ1, ○Φ1 If Φ1 has no modal ops, then GOAL(Φ1) is a formula

Semantics of STL Interpreting requires triple (S, si, g) S = <s0, s1, …> is an infinite sequence si S is the current state g is a goal formula If Lt is based on L of planning domain, and S is sequence of states produced by finite plan, isn’t S finite? Use NOPs. Append infinite final state. In coming slides, let Φ be an STL formula. We now define whether (S, si, g) |= Φ

But First…. Entailment |= Aka “logical implication” A |= B A entails B iff every model that makes A true also makes B true Example A := {“All horses are animals”, “All stallions are horses”} B := {“All stallions are animals”}

Definitions Given ground atom Φ, (S, si, g) |= Φ iff si |= Φ υ - until □ - always ◊ - eventually ○ - next Definitions Given ground atom Φ, (S, si, g) |= Φ iff si |= Φ Eg: (S, ((on a b) (on b c) …), g) |= (on a b) Quantifiers and logical connectives have the same semantic rules as in FOL Eg (S, si, g) |= Φ1Φ2 iff (S, si, g) |= Φ1 and (S, si, g) |= Φ2 (S, si, g) |= □Φ iff (S, sk, g) |= Φ for k i (S, si, g) |= ○Φ iff (S, si+1, g) |= Φ (S, si, g) |= ◊Φ iff k | k  i, (S, sk, g) |= Φ (S, si, g) |= Φ1υΦ2 iff k | k  i, (S, sm, g) |= Φ1 for m = i, …, k-1 and (S, sk, g) |= Φ2 (S, si, g) |= GOAL(Φ) iff Φg For diamond, the book says in words “sk is after si” but their math says >= If Φ contains no GOAL operators, g is irrelevant. Write simply (S, si) |= Φ

Simple Examples Each means the same for s2 υ - until □ - always ◊ - eventually ○ - next Simple Examples Each means the same for s2 s2 |= on(c1,c2)  on(c2,c3) (S,s0) |= ○○(on(c1,c2)  on(c2,c3)) (S,s1) |= ○(on(c1,c2)  on(c2,c3)) (S,s2) |= on(c1,c2)  on(c2,c3) (S,si) |= □holding(crane1,c1) Holding(crane1,c1) is false for all k  i (S,si) |= x ( on(x,c1)  □on(x,c1) ) The same container is on c1 in all subsequent states (S,si) |= x □( on(x,c1)  ○on(x,c1) )

Encoding our Motivating Ex. υ - until □ - always ◊ - event. ○ - next Encoding our Motivating Ex. “Don’t move container if position is consistent with goal” Φ1(c,d,p) = [GOAL(in(c,p))  q GOAL(in(c,q))]  [GOAL(on(c,d)  e GOAL(on(c,e))] Holds if acceptable when container c is on item d in pile p (no goal requiring c in another pile or on top of something else) Φ2(c,p) = ok(c,p) [same(p,pallet)  d (Φ1(c,d,p)  ok(d,p))] ok(c,p) holds iff c is in pile p and c’s position is consistent with the goal Φ3(c) = p (Φ2(c,p)  ok(c,p)) holds iff c’s position is consistent with the goal Φ = c [Φ3(c)  pd □(in(c,p)  in(c,d))] holds iff for every container c whose position is consistent with g, c will never be moved (c always remains in same pile and on same item in that pile) Φ1 Holds if acceptable when container c is on item d in pile p (no goal requiring c in another pile or on top of something else) Φ2 says ok(c,p) holds iff c is in pile p and c’s position is consistent with the goal Φ3 holds iff c’s position is consistent with the goal Φ holds iff for every container c whose position is consistent with g, c will never be moved (c always remains in same pile and on same item in that pile)

Break Yum, dinner 

Progression Computing progression of a control formula Φ is essential for pruning Formula progress(Φ,si) is true in si+1 iff Φ is true in si. This is called Φ’s progression. progress(Φ,si) is the formula produced from progr(Φ,si) by performing usual simplifications Replace true  d with d, true with false, etc. Both functions can be computed in low-order polynomial time with algorithms directly implementing their definitions (S,si,g) |= Φ iff (S,si+1,g) |= progress(Φ,si)

Definition of progr(Φ,si) υ - until □ - always ◊ - eventually ○ - next Definition of progr(Φ,si) If Φ contains no modal operators, then progr(Φ,si) = true if si |= Φ, false if si | Φ Logical connectives are as usual progr(Φ1Φ2,si) = progr(Φ1,si)  progr(Φ2,si) progr(Φ,si) = progr(Φ,si) Modal operators are as follows: progr(○Φ,si) = Φ progr(Φ1υΦ2,si) = ( (Φ1υΦ2)  progr(Φ1,si) )  progr(Φ2,si) progr(◊Φ,si) = (◊Φ)  progr(Φ,si) progr(□Φ,si) = (□Φ)  progr(Φ,si)

Example of Progr Φ = □on(c1,c2) si = on(c1,c2) progress(Φ,si) = υ - until □ - always ◊ - eventually ○ - next Example of Progr progr(□Φ,si) = (□Φ)  progr(Φ,si) Φ = □on(c1,c2) si = on(c1,c2) progress(Φ,si) = = □on(c1,c2)  progress(on(c1,c2),si) = □on(c1,c2)  true = Φ What if si = on(c1,c2)?

Using Control Formulas in Planning Let S = <s0,s1,…> be infinite and Φ be an STL formula. If (S,s0,g) |= Φ, then for every finite truncation S’ = <s0,s1,…,si> of S, progress(Φ,S’)  false Let s0 be a state,  be a plan applicable to s0, and S = <s0,…,sn> be the seq. of states produced by applying  to s0. If Φ is an STL formula and progress(Φ,S) = false, then S has no extension S’ = <s0,…,sn, sn+1,sn+2,…> such that (S’,s0,g) |= Φ Modify Forward-search to prune any partial plan  such that progress(Φ,S) = false

υ - until □ - always ◊ - eventually ○ - next Our Handy Example Let s0 and g be for a container-stacking problem with constant symbols c1, …, ck. Let Φ1, Φ2, Φ3, and Φ be as before. progress(Φ,s0) = = progress(c [Φ3(c)  pd □(in(c,p)  in(c,d))], s0) = progress([Φ3(c1)  pd □(in(c1,p)  in(c1,d))], s0)  …  progress([Φ3(ck)  pd □(in(ck,p)  in(ck,d))], s0) Suppose in s0, c1 is consistent with g and is on item d1 in pile p1. Then s0 |= Φ3(c1) s0 | Φ3(ci) for i = 2, …, k Which means progress(Φ,s0) = = progress( pd □(in(c1,p)  in(c1,d)), s0 ) = progress( □(in(c1,p1)  in(c1,d1)), s0 ) = □(in(c1,p1)  in(c1,d1)) If an applicable action a to state s0 moves c1 then (s0,a) | progress(Φ,s0) Thus (s0,a) can be pruned.

Finally, the Planning Procedure!

Planning Procedure - Comments Sound and complete, if the problem is solvable and STL formula Φ is entailed for at least one solution of the problem Soundness follows from soundness of Forward-Search Completeness follows from what? the condition on Φ Control formulas are like specialized computer programs and must be debugged.

Extensions Function Symbols Axioms (Horn-clauses) Attached Procedures Restrict axioms to Horn-clauses and use a Horn-clause theorem prover Attached Procedures Allow some func./predicate symbols to be evalutated as attached procedures Time Actions with time durations and overlapping Extended Goals Add control rules like Φ = □at(r1,bad-loc) Reach goal in 2 actions or fewer Φ = g V (○g)

Defined in Problem File TLPlan On vega.cc.lehigh.edu: /home/sml3/planning/sys/tlplan http://www.cs.toronto.edu/~fbacchus/tlplan-manual.html Requires one input “script” file referring to two others: (load-file "BlocksWorld.tlp") (load-file "BlocksProblems.tlp") (set-statistics-file "BlocksProblems.csv") (set-goal (goal0)) (set-initial-world (state0)) (set-plan-name "Problem0") (plan) ;; try and solve the problem (select-final-world) ;; needed for next line (print-pddl-plan) (exit) ;;remove this line for interactive mode Running: from the directory containing your domain… ../runtlplan script.tlp Domain File Problem File Defined in Problem File Also in tlplan.log or make a symbolic link in your domain directory via: ln –s <path>/runtlplan

Sample Domain File Problem File (clear-world-symbols) ;Remove old dom symb ;;; WORLD SYMBOLS (declare-described-symbols (predicate on 2) ;…and so on (predicate ontable 1)) (declare-defined-symbols (predicate goodtower 1) (predicate goodtowerabove 1) (function depth 1)) ;;; DEFINED PREDICATES (def-defined-predicate (goodtower ?x) (and (clear ?x) (goodtowerbelow ?x))) ;;; TEMPORAL CONTROL FORMULA (define (bw-control1) (always (forall (?x) (clear ?x) (implies (goodtower ?x) (next (goodtowerabove ?x))) ))) ;;; OPERATORS (def-strips-operator (pickup ?x) (pre (handempty) (clear ?x) (ontable ?x)) (add (holding ?x)) (del (handempty) (clear ?x) (ontable ?x))) ;;; PRINT ROUTINES and FUNCTIONS Problem File (define (state0) (clear a) (clear b) (clear c) (ontable a) (ontable b) (ontable c) (handempty)) (define (goal0) (on a b) (on b c) (ontable c)) …

Defining a Domain: Init/Def (clear-world-symbols) Must call first in a new domain definition file Clears the prev. domain's language definition Resets temporal control formula and the print world command to their defaults (declare-described-symbols (function|predicate name arity [no-cycle-check|rewritable]) ...) must be declared prior to any other symbols (declare-defined-symbols (function|predicate|generator name arity) ...) must be declared after the described symbols (def-defined-predicate (name parameters) (local-vars declarations) formula) Includes a new predicate, defined in terms of a FO formula involving other predicates Can be recursive (define name list) name is an abbreviation for list. Allows macro subst. in domain definition files. Use to define temporal control formulas (:= ?variable value) assigns value to ?variable

Formula Syntax Terms constant – symbol, number, or string ?variable – any symbol starting with “?” (?array index …) – dimension must agree with args (function term …) – built-in or declared w/ init. decl. Atomic Formulas (predicate term …) – predicate declared w/ init. decl. (= term1 term2) – predefined equality binary predicate (TRUE) – const. atomic formula; always true (FALSE) – const. atomic formula; always false First-Order Formulas atomic-formula (and formula …) --  (or formula …) –  (xor formula …) –  (not formula …) –  (implies formula1 formula2) --  (if-then-else formula1 formula2 formula3) (forall var-gen … [formula]) --  (exists var-gen … [formula]) –  (exists! var-gen … [formula]) – unique  (exactly one) Temporal Logic Formulas (TF) first-order-formula (next tf) -- ○ (eventually tf) – ◊ (t-eventually ispec tf) – ispec is an interval of states (always tf) – □ (t-always ispec tf) – ispec is again an interval (t-until ispec tf1 tf2) – υ Modalities (goal formula | tf | generator) – eval in goal world (previous formula | tf | generator) – eval in prev. state (current formula | tf | generator) – eval in cur. state

Defining Operators STRIPS Operator (def-strips-operator name pre del cost duration priority) ;; where… (name v …) – Declares op name and params (pre formula) – Precondition list (add p …) – Add list (del p …) – Delete list (cost n) – Cost of action, default is 1 (duration n) – Duration of action, default 1 (priority n) – Used in search to order successor states, default 0 (def-strips-operator (pickup ?x) (pre (handempty) (clear ?x) (ontable ?x)) (add (holding ?x)) (del (handempty) (clear ?x) (ontable ?x)))

Defining TL Control Formulas (define name list) (define (bw-control1) (always (forall (?x) (clear ?x) (implies (goodtower ?x) (next (goodtowerabove ?x))) )))

Contact Me Start early, defining control rules is tricky, REQUIRES DEBUGGING and tuning sml3@lehigh.edu