1. Planning as Satisfiability CSE 574 April 8, 2003 Dan Weld

2. Schedule BASICS Intro Graphplan SATplan State-space Refinement SPEEDUP TECHNIQUES SIMULTANEOUS + DURATIVE ACTIONS INCOMPLETE INFORMATION INTERLEAVED PLANNING + EXECUTION DECISION-THEORETIC PLANNING

3. The Idea Suppose a plan of length n exists Encode this hypothesis in SAT Init state true at t 0 Goal true at T n Actions imply effects, etc Look for satisfying assignment Decode into plan RISC: The Revolutionary Excitement

4. History Green IJCAI-69 STRIPS AIJ-71 Decades of work on “specialized theorem provers” Kautz+Selman ECAI-92 Rapid progress on SAT solving Kautz+Selman AAAI-96 Electrifying results (on hand coded formulae) Kautz, McAllester & Selman KR-96 Variety of encodings (but no compiler) CSE 573 => Ernst et al. IJCAI-97

5. Blackbox Blackbox solves planning problems by converting them into SAT. Very fast Tried different solvers Local search (GSAT) Systematic search with EBL (RelSAT) In 2000, GP-CSP could beat Blackbox –But in 2001, a newer “SUPER-DUPER” SAT solver called CHAFF was developed, that significantly speeds up Graphplan »By folks in CAD community (EE folks—see getting other people to do our work) – Currently, CSP people are trying to copy over the ideas from CHAFF to CSP.

6. Action Representation Regularfully-instantiated actionPaint-A-Red, Paint-A-Blue, Move-A-Table RepresentationOne Propositional Variable per Example Simply-splitfully-instantiated action’s argument Paint-Arg1-A  Paint-Arg2-Red Overloaded-splitfully-instantiated argument Act-Paint  Arg1-A  Arg2-Red more vars more clses BitwiseBinary encodings of actions Bit1  ~Bit2  Bit3 Paint-A-Red = 5

7. Small # Variables, Literals Variables for actions at t Regular = n Bitwise = log(n)

8. Main Ideas Clear taxonomy Utility of Explanatory frame axioms (most things don’t change) Parallelism & conflict exclusion Type inference Domain axioms Surprising Effectiveness of regular action encodings

9. Comparison Among Encodings Explanatory Frames are superior to classical -few actions affect each fluent -explanatory frames aid simplifications Parallelism is a major factor -fewer mutual exclusion clauses -fewer time steps Regular actions representation is smallest! -exploits full parallelism -aids simplification Overloaded, bitwise reps. are infeasible -prohibitively many clauses -sharing hinders simplification

10. Optimization 1: Factored Splitting - use partially-instantiated actions HasColor-A-Blue-(t-1) ^ Paint-Arg1-B-t ^ Paint-Arg2-Red-t  HasColor-A-Blue-(t+1).46.69.30.50.20.38 SimpleOverloaded Variables Clauses Literals factored unfactored Explanatory Frames

11. Optimization 2: Types A type is a fluent which no actions affects. type interference prune impossible operator instantiations type elimination.27.39.34.30.10.97.67.74 Classical Explanatory Type opts No type opts RegularSimpleOverloadedBitwise Literals

12. Domain-Specific Axioms Adding domain-specific axioms decreases solve time dramatically..861.53.26.881.84.38.862.24<.05 VarsClausesTime a b c domain info no domain info bw- large

13. Weaknesses Too much info, poor organization Graphs – lots of data, little insight Characterize best/worst domain / encoding No experiments on effects of type opt

14. Future Work Negation, disjunctive preconds,  Domain axioms  t clear(x, t)    y on(y, x, t)

15. Future Work Automatically choose best encoding Might do this for frame axioms Automating domain axioms (invariants) Use bounds on plan-length Rather than linear or binary search Resource Planning Compilation to …?

16. Domain Axioms Domain knowledge Synchronic vs. Diachronic constraints Speedup knowledge Action conflicts (=> by action schemata alone) Domain invariants (=> by initial state+schemata) Optimality heuristics Simplifying assumptions

17. Sample Metric Planning Problem Ship RedVines from one location to another Fuel used by moving Maximum fuel level in truck Maximum load for truck Limited amount of available RedVines

18. Sample Metric Planning Domain Action: MOVE-TRUCK(load) preconditions: load  45 ; max. avail. RedVines fuel  7 + load / 2 ; min. required fuel fuel  15 ; fuel capacity load  30 ; load capacity effects: (deliver) if (load = 45) then ; good-trip moves (good-trip) ; all the RedVines Steve Wolfman: I will use this as a running example Steve Wolfman: I will use this as a running example

19. LPSAT Architecture planning compiler LPSAT solver planning decoder input planning language LCNF intermediate language value assignment solution plan Systematic SAT Solver + Incremental LP Solver

20. LCNF Language CNF formula with triggered constraints Boolean variable triggers one (or no) constraint Constraint triggered iff its trigger variable’s truth assignment is true Truth/real-value assignment is a solution iff CNF formula is satisfied All triggered constraints are satisfied

21. Sample Problem in LCNF good-trip 1 deliver 1 move-truck 0  max-load move-truck 0  max-fuel move-truck 0  min-fuel move-truck 0  deliver 1 (move-truck 0  all-loaded)  good-trip 1... max-load: load  30 max-fuel: fuel  15 min-fuel: fuel  7 + load/2 all-loaded: load = 45... CNF formulaConstraint triggers Variable names and, in particular, trigger names added for clarity That’s the front end of the system; the back end simply reads off which actions are assigned true and what the constraint vars values are Set aside for later use!

22. LPSAT Inputs LCNF subset Supports operators +, -, *, and  Restricts constraints to linear (in)equalities Outputs truth/real-value assignment or reports failure Sound Complete Satisficing As opposed to optimizing

23. SOLVER Architecture Satisfiability Solver Linear Programming System new/revoked constraints consistency info, real variable values input problem solution

24. LPSAT Design SAT controller calls LP black box SAT solver based on RelSAT [Bayardo&Schrag] Systematic solver Sound and complete Gradual changes to constraint set Learning/backjumping based on conflict sets LP system is Cassowary [Badros&Borning] supports linear constraints over real variables fast updates to constraint set Explores all possible truth assignments without repeating any

25. LPSAT Algorithm Procedure LPSAT(φ: LCNF problem) If φ is satisfied, return YES Else if φ is inconsistent, return NO Else if there is a unit clause {  } or pure literal  in φ, return LPSAT(φ|  ) Else choose a variable in φ. If LPSAT(φ| ), return YES Else, return LPSAT(φ|  ) Put on other projector

26. Experimental Results Tested against Zeno planner Many times faster than Zeno Solved problems Zeno could not solve Tested conflict set discovery techniques Without learning/backjumping (slow) With global conflict sets (faster) With minimal conflict sets (fastest) Under resource bounds (memory)

27. Conflict Set Discovery Results The domain is a metric version of Kautz and Selman’s logistics domain. Log-c is difficult in the original domain even for modern planners

28. Alternative encodings.. The problem of finding a valid plan from the planning graph can be encoded on any combinatorial substrate Alternatives: CSP [GP-CSP] SAT [Blackbox; SATPLAN] IP [Vossen et al.] BDDs [Cimatti ….] SAT is CSP with Boolean Variables

29. Compilation to CSP [Do & Kambhampati, 2000] Variables: Propositions (In-A-1, In-B-1,..At-R-E-0 …) Domains: Actions supporting that proposition in the plan In-A-1 : { Load-A-1, #} At-R-E-1: {P-At-R-E-1, #} Constraints: Mutual exclusion ~[ ( In-A-1 = Load-A-1) & (At-R-M-1 = Fly-R-1)] ; etc.. Activation In-A-1 != # & In-B-1 != # (Goals must have action assignments) In-A-1 = Load-A-1 => At-R-E-0 != #, At-A-E-0 != # (subgoal activation constraints) [Corresponds to a regression-based proof] Goals: In(A),In(B) CSP: Given a set of discrete variables, the domains of the variables, and constraints on the specific values a set of variables can take in combination, FIND an assignment of values to all the variables which respects all constraints

30. CSP Encodings can be more compact: GP-CSP Do & Kambhampati, 2000

31. Compilation to LP Motivations Ability to handle numeric quantities, and optimize Heuristic value of the LP relaxation of ILP problems Conversion Convert a SAT/CSP encoding to ILP inequalities E.g. X v ~Y v Z => x + (1 - y) + z >= 1 Explicitly set tighter ILP inequalities (Cutting constraints) If X,Y,Z are pairwise mutex, we can write x+y+z <= 1 (instead of x+y <=1 ; y+z <=1 ; z +x <= 1) [ Walser & Kautz; Vossen et. al; Bockmayr & Dimopolous] Integer Linear Programming: Given a set of real valued variables, a linear objective function on the variables, a set of linear inequalities on the variables, and a set of integrality restrictions on the variables, Find the values of the feasible variables for which the objective function attains the maximum value -- 0/1 integer programming corresponds closely to SAT problem

32. Vossen/Ball/Lotem/Nau work Vossen et. Al. [IJCAI 99] tried two different IP encodings 1.Simple conversion of PG SAT encoding to IP (last slide) 2.Two new ideas Compile away proposition variables. The change of truth value of a proposition is equated to the occurrence of one of the actions that can give that proposition. This is basically a known transformation—see the McAllester/Kautz/Selman paper. (see next slide) Reformulate the constraints to make better use of expressiveness provided by IP E.g. IP is particularly good at compactly encoding n-ary mutex constraints—even for n>2 To say only one of X1..Xn can be true, we just say X1+X2+…Xn <= 1 In SAT, we have to write n constraints of the form –Xi => ~X1,~X2…~Xi-1,~Xi+1,….~Xn (IP, on the other hand, will have harder time representing “All- different” constraints)

34. Performance of IP Encodings As seen in the previous slide, IP encodings are not competitive with SAT/CSP encodings for classical planning Aren’t exercizing the “optimizing” properties of IP, Aren’t exercizing its ability to encode numerical variables IP may well do better if we are interested in finding “cost- optimal” plans of a given makespan (length) No one has confirmed this as yet… IP encodings may well be more useful when we have numerical quantities such as actions consuming metric resources (preconditions and effects involve numbers) See the work by Kautz and Walser(AAAI-99) or LPSat (Wolfman/Weld—IJCAI 99).

35. Compilation to (BDDs) Binary Decision Diagrams Idea: Represent sets of states as BDDs and refining the sets together as BDD operations –Operations such as “action projection” need to be modeled as BDD modifications [ Cimatti et. al., Fourman, Hans-Peter] X1X1 X2X2 BDD for X 1 & X 2 1 0 BDDs support compact representation and direct manipulation of boolean formulae on a finite set of propositions. (Popular in CAD community) Standard algorithms for converting a boolean formulae into BDDs and for supporting standard boolean operations on them [Bryant et. al.]

36. Tradeoffs as f(compilation substrates) CSP encodings support implicit representations More compact encodings [Do & Kambhampati, 2000] Easier integration with Scheduling techniques ILP encodings support numeric quantities Seamless integration of numeric resource constraints [Walser & Kautz, 1999] Not competitive with CSP/SAT for problems without numeric constraints SAT encodings support axioms in propositional logic form May be more natural to add (for whom ;-) Very active research on finding faster solvers.. Davis Putnam ( Satz; RelSat; Chaff) Gsat

37. Direct vs. compiled solution extraction LNeed to adapt CSP/SAT techniques JCan exploit approaches for compacting the plan JCan make the search incremental across iterations LCompilation stage can be time consuming, leads to memory blow-up LMakes it harder to exploit search from previous iterations JCan exploit the latest advances in SAT/CSP solvers JMakes it easier to add declarative control knowledge DIRECT Compiled

38. Resource Planning

39. Challenges in SAT