1. Planning as Satisfiability: Progress and Challenges Bart Selman Cornell University

2. 2 What is Planning? Planning (AI): Find a (partially) ordered set of actions that transforms a given initial state to a specified goal state. Can cover most forms of “problem solving” Long history in AI, since mid sixties --- general problem solving architecture --- robotics (“planning a sequence of actions to achieve a set of goals”) Scheduling (OR): Fixed set of actions, “just” need to find optimal ordering --- Significantly easier than general planning --- E.g. linear programming relaxations are quite effective. --- Planning problems typically highly non-linear, require combinatorial search

3. Some Applications of Planning Supply chain management: combines planning and scheduling [main industrial interest] Autonomous systems Many NASA applications: Deep Space One Remote Agent Long-range mission planning Communication planning & scheduling [DOD / Darpa] Softbots - software agents Internet agents, program assistants AI “characters” in games, entertainment Natural language / dialogue understanding and generation Speech act: The goal of the speaker is to convey info to the listener. Dialogue can be formulated as a plan generation problem. Listener: plan recognition poblem. User modeling: Plan recognition Theorem proving: Proof planning

4. 4 Core Computational Problem Focus: Classical state-space planning, extended with parallel actions Similar computational issues arise in other models Reactive plans Planning with uncertainty and utilities Continuous processes Metric time

5. 5 State-space Planning Find a sequence of operators that transform an initial state to a goal state State = complete truth assignment to a set of variables (“fluents”) Goal = partial truth assignment (set of states) Operator = a partial function State  State specified by three sets of variables: precondition, add list, delete list

6. 6 Parallelism Operators may be applied in parallel when all orderings are well defined and equivalent (Op1 || Op2)(s) = Op2(Op1(s)) = Op1(Op2(s)) A special form of non-linear plans Only allows parallel actions, not parallel action sequences Easy to serialize

7. 7 Abdundance of Negative Complexity Results I.Domain-independent planning: PSPACE-complete or worse (Chapman 1987; Bylander 1991; Backstrom 1993) II. Domain-dependent planning: NP-complete or worse operators fixed (Chenoweth 1991; Gupta and Nau 1992) III. Approximate planning: NP-complete or worse (Selman 1994)

8. Planning Practice: Phase I Till mid-nineties: Domain-independent planning systems could generate plans of only a few (< 10) steps! Prodigy, Nonlin, UCPOP,... Limited practical use Research focus: minimize or eliminate search by employing complex search control rules, hand-tailored to the search engine and the particular search space (Sacerdoti 1975, Slaney 1996, Bacchus 1996) pre-compiling entire state-space to a reactive finite-state machine (Agre & Chapman 1997, Williams & Nayak 1997) Drawback: Highly domain dependent.

9. Planning Practice: Phase II Since mid-nineties, three key advances: --- GraphPlan (Blum and Furst 1996) planning as a specialized graph search problem --- SATPLAN (Kautz and Selman 1996; foundation in ’92) planning as Boolean satisfiabilty (constraint reasoning) --- FF planner (Hoffmann 2000) non-optimal planning as specialized heuristic search (A*) Till mid-nineties: < 10 step optimal plans Current: we can now synthesize optimal 500+ step plans non-optimal: 2000+ steps [historical note: 1969 --- planning as first-order inference (Green ‘69)]

10. 10 Planning as Satisfiability: SATPLAN axiom schemas instantiated propositional clauses satisfying model plan mapping length problem description SAT engine(s) instantiate interpret Note: To deal with PSPACE completeness, we limit the plan length. Gives NP-completeness. Analogous to bounded model checking.

11. blackbox version 9B command line: blackbox -o logistics.pddl -f logistics_prob_d_len.pddl -solver compact -l -then satz -cutoff 25 -restart 10 ---------------------------------------------------- Converting graph to wff 6151 variables 243652 clauses Invoking simplifier compact Variables undetermined: 4633 Non-unary clauses output: 139866 ---------------------------------------------------- Invoking solver satz version satz-rand-2.1 Wff loaded [1] begin restart [1] reached cutoff 25 --- back to root [2] begin restart [2] reached cutoff 25 --- back to root [3] begin restart [3] reached cutoff 25 --- back to root [4] begin restart [4] reached cutoff 25 --- back to root [5] begin restart **** the instance is satisfiable ***** **** verification of solution is OK **** total elapsed seconds = 25.930000 ---------------------------------------------------- Begin plan 1 drive-truck_ny-truck_ny-central_ny-po_ny Example run

12. 100 step optimal plan

13. Biannual Planning competitions: --- All current competitors are derived from Graphplan, SATPLAN, or FF. Traditional planners no longer competitive Knowledge-intensive approaches require too much human effort Range of new techniques: Type-theoretic analysis of operators: can infer state invariants as extra constraints powerful, generally applicable pre-processor (Long et al. ‘01; Gerevini and Schubert ‘04) Compilation of more expressive languages (conditional effects) Recent extensions to Markov Decision Processes of FF (Geffner ‘02), Graphplan (Blum ‘02), SATPLAN (Littman ‘03) Deriving more complex plan constraints on the fly (Chen ’05)

14. 14 Research Challenges [planning & constraint reasoning in general] a) Automated structure discovery to further improve large-scale constraint solving [capture structure in additional constraints; use statistical and other machine learning techniques] b) Adversarial reasoning: Quantified Boolean Formulas c) Probabilistic reasoning: Solution Counting and the integration of probabilistic and logical reasoning

15. 15 Worst-case intractability but real-world has significant hidden tractable sub-structure. [aside: exponential behavior of SAT solvers easily demonstrated on synthetic instance classes. So, structure of naturally occurring problems is key.] Key notion: backdoor variables A small set of special variables that capture inherent combinatorics of a reasoning problem. In terms of human cognition, real-world reasoning problems can often be solved allow using clever shortcuts. Formal definition includes the notion of a “subsolver”: a polynomial simplification procedure with certain general characteristics found in current DPLL SAT solvers. I)Hidden Problem Structure Backdoors to tractability (Gomes et al. ’03. ’04)

16. Backdoors (wrt subsolver A; SAT case): Strong backdoors (wrt subsolver A; UNSAT case): Note: Notion of backdoor is related to but different from constraint-graph based notions such as cutsets.

17. Backdoors can be surprisingly small: Most recent: Other combinatorial domains. E.g. graphplan planning, near constant size backdoors (2 or 3 variables) and log(n) size in certain domains. (Hoffmann, Gomes, Selman ’04) Backdoors capture critical problem resources (bottlenecks).

18. Backdoors --- “seeing is believing” Logistics_b.cnf planning formula. 843 vars, 7,301 clauses, approx min backdoor 16 (backdoor set = reasoning shortcut) Constraint graph of reasoning problem. One node per variable: edge between two variables if they share a constraint. Visualization by Anand Kapur.

19. Logistics.b.cnf after setting 5 backdoor vars.

20. After setting just 12 (out of 800+) backdoor vars – problem almost solved.

21. MAP-6-7.cnf infeasible planning instances. Strong backdoor of size 3. 392 vars, 2,578 clauses. Another example

22. After setting 2 (out of 392) backdoor vars. --- reducing problem complexity in just a few steps!

23. Inductive inference problem --- ii16a1.cnf. 1650 vars, 19,368 clauses. Backdoor size 40. Last example.

24. After setting 6 backdoor vars.

25. After setting 38 (out of 1600+) backdoor vars: Some other intermediate stages: So: Real-world structure hidden in the network. Can be exploited by automated reasoning engines.

26. (Williams, Gomes, and Selman ’04) Current solvers Size backdoor n = num. vars. k is a constant Restarts, clause learning and local focus (Malik ’08) help in backdoor discovery.

27. II) Adversarial Reasoning Quantified Boolean Formulas (QBF) extend Boolean logic by allowing quantification over variables (exists and forall) QBF is satisfiable iff there exists a way of setting the existential vars such that for every possible assigment to the universal vars the clauses are satisfied. Literally a “game played on the clauses”: Existential player tries hard to satisfy all clauses in the matrix. Universal player tries hard to “spoil” it for the existential player: i.e., break (“unsatisfy”) one or more clauses. the clausesQuantifiers prefix

28. Formally: Problem is PSPACE- complete. Range of new applications: Multi-agent reasoning, unbounded planning, unbounded model-checking (verification), and certain forms probabilistic reasoning and contingency planning. Can we transfer successful SAT techniques to QBF? Cautiously optimistic. But very sensitive to problem encodings. (Antsotegui, Gomes, and Selman ’05) Related work: Walsh ’03; Gent, Nightingale, and Stergiou ’05; Pan & Vardi 04; Giunchiglia et al. 04; Malik and Zhang ‘04; Alur ’04; and Williams ’05.

29. Original Search Space 2 N Search Space SAT Encoding 2 N+M Space Searched by SAT Solvers 2 N/C ; N log(N) ; Poly(N ) Original 2 N Search Space for SAT Approaches

30. Original Search Space 2 N Search Space QBF Encoding 2 N+M’ Space Searched by COND QBF Solvers with Streamlining Search Space of QBF Search Space Standard QBF Encoding 2 N+M’’ Original 2 N

31. III) Combining Probabilistic and Logical Reasoning Survey Propagation (SP) Mezard et al. (Science ’02). Exciting new reasoning / combinatorial search method. Applies probabilistic reasoning technique for solving combinatorial search problems. Basic idea: Let N be the total number of satisfying assignments. N_x+ the number of satisfying assigns with x set to True. N_x- with x set to False. Define: P_x+ = N_x+ / N and P_x- = N_x- / N. I.e., P_x+ is “the probability of seeing x assigned True when randomly sampling satisfying assignments.”

32. SP, cont. Consider the following “decimation” strategy: If P_x+ >= P_x- then set x to True else set x to False. Simplify instance and repeat, until a satisfying assignment is reached. But, only a physicist would think of such a strategy! (Why?) Since computing the probabilities is much harder (#P-complete) than finding a satisfying assignment (NP-complete)!

33. SP, surprise But, perhaps one can efficiently compute good approximations of P_x+ and P_x- Strategy is to iteratively solve a set of recursive equations. Linear time. The so-called SP equations are quite involved. They are a form of probabilistic reasoning called Belief Propagation. Intuitively, the idea is to consider the effect of adding a clause (constraint) to a set of clauses. Example: start with the empty set of clauses over two variables p and q. So, P_p+ = P_p- = ½ and P_q_+ = P_q- = ½. Now add a clause (p OR (NOT q)). What happens to P_p+ and P_q+? First should go up a bit and the other down a bit…

34. (p OR (NOT q)) is satisfied by (T, F), (T, T), and (F,F). So, P_p+ = 2/3 and P_p- = 1/3 and P_q+ = 1/3 and P_q- = 2/3. Now consider adding ((NOT p) OR q OR r). P_p+ should go down a bit. P_q+ and P_r+ up a bit. Etc. Brute force enumeration becomes infeasible but SP equations model the changes in these probabilities directly to capture the addition of clauses/constraints. Clauses and variables interact, so we will have to look for a fixed point of a set of coupled recursive equations.

35. The CNF: The “Factor” Graph: (Graphical Model. Bayesian Net) The equations:

36. SP is surprisingly effective on hard random k-SAT and graph coloring. 10M instances with 42M clauses can be solved in linear time (around one hour of cpu time; sets batches of variables, never backtracks, finds satisfying assignment!) Walksat, a biased random walk strategy, is the next best but would require 100+ hrs of cpu time. Our formal understanding of SP is limited. SP is still a bit “magical”. Some insights: Zecchina et al. 2004; Wainright et al. 2006; Kroc et al. 2007, 2008.

37. (Kroc et al. 2007) Hard random 3-SAT. 5,000 var; 21,000 clauses SP Marginal Probabilities 0.0 1.0 True Marginal Probabilities 0.0 1.0 SP: marginals in < 30 seconds 100+ hrs (direct computation)

38. 38 Survey Propagation: final observations Next challenge for SP: use on structured (non-random) formulas. Significant potential. (preliminary results in Kroc et al. 2007b.) In general, probabilistic techniques for solving constraint problems provides an intriguing alternative to the existing two main paradigms: (1) complete, backtrack search, and (2) local search. Involves finding fixed points of coupled recursive equations over continuous variables. New perspective and potentially very fast.

39. P NP P^#P PSPACE NP-complete: SAT, scheduling, graph coloring, puzzles, … PSPACE-complete: QBF, adversarial planning, chess (bounded), … EXP-complete: games like Go, … P-complete: circuit-value, … Note: widely believed hierarchy; know P≠EXP for sure. In P: sorting, shortest path, … Complexity Hierarchy: Where We Are Easy PH EXP #P-complete/hard: #SAT, sampling, probabilistic inference, … SAT/SMT Solvers “Traditional” Algs. Extended SAT Solvers QBF Solvers Hard First Order Thm. Provers