Planning as Propositional Satisfiabililty Brian C. Williams Oct. 30 th, J/6.834J GSAT, Graphplan and WalkSAT Based on slides from Bart Selman & Henry Kautz

Outline  Planning as local search  Fast state-space planning using Sat

GSAT Example  C1: Not A or B  C2: Not C or Not A  C3: or B or Not C C1, C2, C3 violated A True B False C True C3 violated False C2 violated True C1 violated False 1.Pick random assignment 2.Check effect of flipping each assignment, counting violated clauses 3.Pick assignment with fewest violations.

GSAT Example  C1: Not A or B  C2: Not C or Not A  C3: or B or Not C C1 violated A True B False C Satisfied False Satisfied True C1,C2,C3 violated True 1.Pick random assignment 2.Check effect of flipping each assignment, counting violated clauses 3.Pick assignment with fewest violations.

GSAT Example  C1: Not A or B  C2: Not C or Not A  C3: or B or Not C Satisfied A True B C False 1.Pick random assignment 2.Check effect of flipping each assignment, counting violated clauses 3.Pick assignment with fewest violations.

Review Satisfiability Testing Procedures  Reduce to CNF (Clausal Form) then:  Systematic, complete procedures  Depth-first backtrack search (Davis, Putnam, & Loveland 1961)  unit propagation, shortest clause heuristic  State-of-the-art implementations:  ntab (Crawford & Auton 1997)  itms (Nayak & Williams 1997)  many others! See SATLIB 1998 / Hoos & Stutzle  Stochastic, incomplete procedures  GSAT (Selman et. al 1993)  Walksat (Selman & Kautz 1993)  greedy local search + noise to escape local minima

Outline  Satisfiability as Local Search  Fast state-space planning using Sat  Direct encoding (Sat Plan)  Graphplan encoding (Blackbox)

SATPLAN axiom schemas instantiated propositional clauses satisfying model plan mapping length problem description SAT engine(s) instantiate interpret

Approach  Design SAT encodings so that plans correspond to satisfying assignments  Use recent efficient satisfiability procedures (systematic and stochastic) to solve  Evaluate performance on benchmark instances

SAT Encodings  Propositional CNF: no variables or quantifiers  Sets of clauses specified by axiom schemas  fully instantiated before problem-solving  Discrete time, modeled by integers  state predicates: indexed by time at which they hold  action predicates: indexed by time at which action begins  each action takes 1 time step  many actions may occur at the same step

Encoding Conventions  Actions imply preconditions and effects fly(x,y,i)  at(x,i) & route(x,y) & at(y,i+1)  Conflicting actions cannot occur at same time (A deletes a precondition of B) fly(x,y,i) & y  z   fly(x,z,i)  If something changes, an action must have caused it (Explanatory Frame Axioms) at(x,i) &  at(x,i+1)   y. route(x,y) & fly(x,y,i)  Initial and final states hold at(NY,0) &... & at(LA,9) &...

Modeling Tricks  Can often dramatically reduce size of problem by modeling techniques move(x,y,z,i) requires n 4 vars pickup(x,y,i), putdown(x,z,i) requires 2n 3 vars  State-based encodings: eliminate all action variables (“compile away”) at(x,i)  at(x,i+1)   y. route(x,y) & at(y,i+1) at(x,i) & x  y   at(y,i)

Solution to a Planning Problem  A solution is specified by any model (satisfying truth assignment) of the conjunction of the axioms describing the initial state, goal state, and operators  Easy to convert back to a STRIPS-style plan

SAT Algorithms  Systematic Search  DP (Davis Putnam Logemann Loveland) backtrack search + unit propagation  satz (Chu Min Li) - variable selection by forward checking: max unit props  relsat (Bayardo) - dependency directed backtracking: add new clauses at dead-ends  Local Search  Inspired by Mins-Conflict algorithm (Adorf, Johnson, Minton, Philips, & Laird)  GSAT (Selman), Walksat (Selman, Kautz & Cohen) greedy local search + noise to escape minima

Planning Benchmark Test Set  Extension of Graphplan test set  blocks world - up to 18 blocks, states  logistics - complex, highly-parallel transportation domain. Logistics.d:  2,165 possible actions per time slot  legal configurations ( states)  optimal solution contains 74 distinct actions over 14 time slots  Problems of this size never previously handled by state-space planning systems

Scaling Up Logistics Planning

Unpredictability of Systematic Search

Randomized Restarts  Solution: randomize the systematic solver  Add noise to the heuristic branching (variable choice) function  Cutoff and restart search after a fixed number of backtracks  In practice: rapid restarts with low cutoff can dramatically improve performance  (Gomes 1996, Gomes, Kautz, and Selman 1997, 1998)

Increased Predictability

What SATPLAN Shows  General propositional theorem provers can compete with state of the art specialized planning systems  New, highly tuned variations of DP surprisingly powerful  result of sharing ideas and code in large SAT/CSP research community  specialized engines can catch up, but by then new general techniques  Radically new stochastic approaches to SAT can provide very low exponential scaling  2+ orders magnitude speedup on hard benchmark problems

Why SATPLAN Works  More flexible than forward or backward chaining  Systematic: most unit propagation at most highly constrained states  Stochastic: iterative repair  Space for time tradeoff  Less overhead since does not have to instantiate variable during search  Randomized algorithms less likely to get trapped along bad paths

Outline  Satisfiability as Local Search  Fast state-space planning using Sat  Direct encoding (Sat Plan)  Graphplan encoding (Blackbox)

Graphplan  Planning as graph search (Blum & Furst 1995) Set new paradigm for planning  Like SATPLAN...  Two phases: instantiation of propositional structure, followed by search  Unlike SATPLAN...  Interleaves instantiation and pruning of plan graph  Employs specialized search engine  Graphplan - better instantiation  SATPLAN - better search

Blackbox STRIPS Plan Graph Mutex computation CNF General Stochastic / Systematic SAT engines Solution Simplifier Translator CNF

The Plan Graph Facts Actions... Facts Actions... preconditionsadd effects mutually exclusive delete effects

Graph Pruning  Graphplan instantiates in a forward direction, pruning unreachable nodes  conflicting actions are mutex  if all actions that add two facts are mutex, the facts are mutex  if the preconditions for an action are mutex, the action is unreachable!  In logical terms: limited application of negative binary propagation  given:  P V  Q, P V R V S V...  infer:  Q V R V S V...

Bridging Paradigms  Both SATPLAN and Graphplan are disjunctive planners (Kambhampati 1996) Can the best features of each be combined?  IJCAI Challenge in Bridging Plan Synthesis Paradigms (Kambhampati 1997)  Our response: blackbox

Translation of Plan Graph  Fact  Act1  Act2  Act1  Pre1  Pre2  ¬Act1  ¬Act2 Act1 Act2 Fact Pre1 Pre2

Improved Encodings  Translations of Logistics.a:  STRIPS  Axiom Schemas  SAT (Medic system, Weld et. al 1997)  3,510 variables, 16,168 clauses  24 hours to solve  STRIPS  Plan Graph  SAT  2,709 variables, 27,522 clauses  5 seconds to solve!

Limited Inference  SATPLAN used only a single general theorem-prover  What role can limited (polytime) reasoning algorithms play?  Two kinds of limited deduction  Planning specific (mutex computation)  General polytime simplification  apply to all CNF formulas, may or may not be designed with planning in mind

General Limited Inference  Generated wff can be further simplified by consistency propagation techniques  Compact (Crawford & Auton 1996)  unit propagation: is Wff inconsistent by resolution against unit clauses? O(n)  failed literal rule: is Wff + { P } inconsistent by unit propagation? O(n 2 )  binary failed literal rule: is Wff + { P V Q } inconsistent by unit propagation? O(n 3 )

General Limited Inference

Intuition  Many real-world problems not tractable, but are nearly so  polytime inference takes advantage of special kinds of structure  structure may be visible at the level of a domain specific representation, or only after the problem is encoded  small numbers of practical methods for combinatorial core  can be highly optimized

Blackbox Results

Staged Inference Domain specific model Polytime domain specific inference General language encoding Full general inference (NP complete) Solution Polytime general inference Abstract problem specification Encoding scheme Combinatorial CORE

Conclusions  Propositional approaches to Open-Loop planning using general SAT engines are highly competitive with specialized planning algorithms  Synergy with Plan Graph approaches  Biggest limitation: domains where number of objects is too large to instantiate