1. Planning as Propositional Satisfiabililty Brian C. Williams Oct. 30 th, 2002 16.412J/6.834J GSAT, Graphplan and WalkSAT Based on slides from Bart Selman & Henry Kautz

2. Outline  Planning as local search  Fast state-space planning using Sat

3. GSAT Example  C1: Not A or B  C2: Not C or Not A  C3: or B or Not C C1, C2, C3 violated A True B False C True C3 violated False C2 violated True C1 violated False 1.Pick random assignment 2.Check effect of flipping each assignment, counting violated clauses 3.Pick assignment with fewest violations.

4. GSAT Example  C1: Not A or B  C2: Not C or Not A  C3: or B or Not C C1 violated A True B False C Satisfied False Satisfied True C1,C2,C3 violated True 1.Pick random assignment 2.Check effect of flipping each assignment, counting violated clauses 3.Pick assignment with fewest violations.

5. GSAT Example  C1: Not A or B  C2: Not C or Not A  C3: or B or Not C Satisfied A True B C False 1.Pick random assignment 2.Check effect of flipping each assignment, counting violated clauses 3.Pick assignment with fewest violations.

6. Review Satisfiability Testing Procedures  Reduce to CNF (Clausal Form) then:  Systematic, complete procedures  Depth-first backtrack search (Davis, Putnam, & Loveland 1961)  unit propagation, shortest clause heuristic  State-of-the-art implementations:  ntab (Crawford & Auton 1997)  itms (Nayak & Williams 1997)  many others! See SATLIB 1998 / Hoos & Stutzle  Stochastic, incomplete procedures  GSAT (Selman et. al 1993)  Walksat (Selman & Kautz 1993)  greedy local search + noise to escape local minima

7. Outline  Satisfiability as Local Search  Fast state-space planning using Sat  Direct encoding (Sat Plan)  Graphplan encoding (Blackbox)

8. SATPLAN axiom schemas instantiated propositional clauses satisfying model plan mapping length problem description SAT engine(s) instantiate interpret

9. Approach  Design SAT encodings so that plans correspond to satisfying assignments  Use recent efficient satisfiability procedures (systematic and stochastic) to solve  Evaluate performance on benchmark instances

10. SAT Encodings  Propositional CNF: no variables or quantifiers  Sets of clauses specified by axiom schemas  fully instantiated before problem-solving  Discrete time, modeled by integers  state predicates: indexed by time at which they hold  action predicates: indexed by time at which action begins  each action takes 1 time step  many actions may occur at the same step

11. Encoding Conventions  Actions imply preconditions and effects fly(x,y,i)  at(x,i) & route(x,y) & at(y,i+1)  Conflicting actions cannot occur at same time (A deletes a precondition of B) fly(x,y,i) & y  z   fly(x,z,i)  If something changes, an action must have caused it (Explanatory Frame Axioms) at(x,i) &  at(x,i+1)   y. route(x,y) & fly(x,y,i)  Initial and final states hold at(NY,0) &... & at(LA,9) &...

12. Modeling Tricks  Can often dramatically reduce size of problem by modeling techniques move(x,y,z,i) requires n 4 vars pickup(x,y,i), putdown(x,z,i) requires 2n 3 vars  State-based encodings: eliminate all action variables (“compile away”) at(x,i)  at(x,i+1)   y. route(x,y) & at(y,i+1) at(x,i) & x  y   at(y,i)

13. Solution to a Planning Problem  A solution is specified by any model (satisfying truth assignment) of the conjunction of the axioms describing the initial state, goal state, and operators  Easy to convert back to a STRIPS-style plan

14. SAT Algorithms  Systematic Search  DP (Davis Putnam Logemann Loveland) backtrack search + unit propagation  satz (Chu Min Li) - variable selection by forward checking: max unit props  relsat (Bayardo) - dependency directed backtracking: add new clauses at dead-ends  Local Search  Inspired by Mins-Conflict algorithm (Adorf, Johnson, Minton, Philips, & Laird)  GSAT (Selman), Walksat (Selman, Kautz & Cohen) greedy local search + noise to escape minima

15. Planning Benchmark Test Set  Extension of Graphplan test set  blocks world - up to 18 blocks, 10 19 states  logistics - complex, highly-parallel transportation domain. Logistics.d:  2,165 possible actions per time slot  10 16 legal configurations (2 2000 states)  optimal solution contains 74 distinct actions over 14 time slots  Problems of this size never previously handled by state-space planning systems

16. Scaling Up Logistics Planning

17. Unpredictability of Systematic Search

18. Randomized Restarts  Solution: randomize the systematic solver  Add noise to the heuristic branching (variable choice) function  Cutoff and restart search after a fixed number of backtracks  In practice: rapid restarts with low cutoff can dramatically improve performance  (Gomes 1996, Gomes, Kautz, and Selman 1997, 1998)

19. Increased Predictability

20. What SATPLAN Shows  General propositional theorem provers can compete with state of the art specialized planning systems  New, highly tuned variations of DP surprisingly powerful  result of sharing ideas and code in large SAT/CSP research community  specialized engines can catch up, but by then new general techniques  Radically new stochastic approaches to SAT can provide very low exponential scaling  2+ orders magnitude speedup on hard benchmark problems

21. Why SATPLAN Works  More flexible than forward or backward chaining  Systematic: most unit propagation at most highly constrained states  Stochastic: iterative repair  Space for time tradeoff  Less overhead since does not have to instantiate variable during search  Randomized algorithms less likely to get trapped along bad paths

22. Outline  Satisfiability as Local Search  Fast state-space planning using Sat  Direct encoding (Sat Plan)  Graphplan encoding (Blackbox)

23. Graphplan  Planning as graph search (Blum & Furst 1995) Set new paradigm for planning  Like SATPLAN...  Two phases: instantiation of propositional structure, followed by search  Unlike SATPLAN...  Interleaves instantiation and pruning of plan graph  Employs specialized search engine  Graphplan - better instantiation  SATPLAN - better search

24. Blackbox STRIPS Plan Graph Mutex computation CNF General Stochastic / Systematic SAT engines Solution Simplifier Translator CNF

25. The Plan Graph Facts Actions... Facts Actions... preconditionsadd effects mutually exclusive delete effects

26. Graph Pruning  Graphplan instantiates in a forward direction, pruning unreachable nodes  conflicting actions are mutex  if all actions that add two facts are mutex, the facts are mutex  if the preconditions for an action are mutex, the action is unreachable!  In logical terms: limited application of negative binary propagation  given:  P V  Q, P V R V S V...  infer:  Q V R V S V...

27. Bridging Paradigms  Both SATPLAN and Graphplan are disjunctive planners (Kambhampati 1996) Can the best features of each be combined?  IJCAI Challenge in Bridging Plan Synthesis Paradigms (Kambhampati 1997)  Our response: blackbox

28. Translation of Plan Graph  Fact  Act1  Act2  Act1  Pre1  Pre2  ¬Act1  ¬Act2 Act1 Act2 Fact Pre1 Pre2

29. Improved Encodings  Translations of Logistics.a:  STRIPS  Axiom Schemas  SAT (Medic system, Weld et. al 1997)  3,510 variables, 16,168 clauses  24 hours to solve  STRIPS  Plan Graph  SAT  2,709 variables, 27,522 clauses  5 seconds to solve!

30. Limited Inference  SATPLAN used only a single general theorem-prover  What role can limited (polytime) reasoning algorithms play?  Two kinds of limited deduction  Planning specific (mutex computation)  General polytime simplification  apply to all CNF formulas, may or may not be designed with planning in mind

31. General Limited Inference  Generated wff can be further simplified by consistency propagation techniques  Compact (Crawford & Auton 1996)  unit propagation: is Wff inconsistent by resolution against unit clauses? O(n)  failed literal rule: is Wff + { P } inconsistent by unit propagation? O(n 2 )  binary failed literal rule: is Wff + { P V Q } inconsistent by unit propagation? O(n 3 )

32. General Limited Inference

33. Intuition  Many real-world problems not tractable, but are nearly so  polytime inference takes advantage of special kinds of structure  structure may be visible at the level of a domain specific representation, or only after the problem is encoded  small numbers of practical methods for combinatorial core  can be highly optimized

34. Blackbox Results

35. Staged Inference Domain specific model Polytime domain specific inference General language encoding Full general inference (NP complete) Solution Polytime general inference Abstract problem specification Encoding scheme Combinatorial CORE

36. Conclusions  Propositional approaches to Open-Loop planning using general SAT engines are highly competitive with specialized planning algorithms  Synergy with Plan Graph approaches  Biggest limitation: domains where number of objects is too large to instantiate