1. Tradeoffs in Backdoors: Inconsistency Detection, Dynamic Simplification, and Preprocessing Bistra Dilkina, Carla Gomes, Ashish Sabharwal Cornell University ISAIM 2008 Fort Lauderdale, FL Note: Part of this work was presented at CP-07

2. ISAIM-082 Context  Constraint Satisfaction Problems (CSPs)  In particular, Boolean Satisfiability or SAT :  Given a Boolean formula F in conjunctive normal form e.g. F = (a or b) and (  a or  c or d) and (b or c) determine whether F is satisfiable  NP-complete  widely used in practice, e.g. in hardware & software verification, design automation, AI planning, …

3. ISAIM-083 From annual SAT competitions:  Large industrial benchmarks (10K+ vars) are solved within seconds by state-of-the-art complete SAT solvers  Good scaling behavior seems to defy “NP-completeness” ! How can we explain this gap between theory and practice? Real-world problems have tractable sub-structure “Backdoors” explain how solvers can get “smart” and solve very large instances SAT: Gap between theory & practice

4. ISAIM-084 Backdoors to tractability Informally [Gomes et al ’03]: A backdoor is a critical set of variables for a given problem such that, once assigned values, the remaining instance simplifies to a “tractable” class How is tractability captured formally in the notion of backdoors? sub-solver capturing poly-time mechanisms (not necessarily syntactically defined) e.g. Unit Propagation, Arc Consistency, LP. Note: concept of backdoor applicable to CSPs, MIP, …

5. ISAIM-085 Tractability: sub-solver Sub-solver S: An algorithm that, given formula F, satisfies four properties: 1.Efficiency: S runs in polynomial time (often linear or quadratic time) 2.Trivial solvability: S can determine whether F is trivially True (has no clauses) or trivially False (has an empty clause, or empty domain) 3.Trichotomy: reject, declare sat, declare unsat 4.Self-reducibility: also solves F[v/x]

6. ISAIM-086 Strong Backdoors A subset B of variables is a strong backdoor for a formula F w.r.t. a sub-solver S if: for every truth assignment  to B S solves the “simplified” formula F[  /B]  Note: have provided powerful insights, leading to techniques like randomization, restarts, and algorithm portfolios for SAT  Two key features A. Dynamic simplification based on truth values and constraint semantics (e.g. unit propagation) B. Trivial solvability / inconsistency detection

7. ISAIM-087 Our Results 1.Relaxing these two key features leads to exponentially larger backdoors 2.Finding strong backdoors can become harder* than NP just by adding inconsistency detection to tractable class (“static” notions trivially within NP, thus “weaker”) 3.State-of-the-art solvers do find very small strong backdoors (with inconsistency detection) 4.Preprocessing: small & mixed effect on backdoor size 5.Satisfiable instances: backdoors smaller w.r.t. PL than UP

8. ISAIM-088 Our Results 1.Relaxing these two key features leads to exponentially larger backdoors 2.Finding strong backdoors can become harder* than NP just by adding inconsistency detection to tractable class (“static” notions trivially within NP, thus “weaker”) 3.State-of-the-art solvers do find very small strong backdoors (with inconsistency detection) 4.Preprocessing: small & mixed effect on backdoor size 5.Satisfiable instances: backdoors smaller w.r.t. PL than UP

9. From algorithmic backdoors to syntactically defined tractable classes

10. ISAIM-0810 Horn/2CNF backdoors  Horn formula – every clause has at most one positive literal  2CNF formula – every clause is binary  “Strong” Backdoors w.r.t. Horn/2CNF a subset B of variables such that for every value assignment to B, the simplified sub-formula is Horn/2CNF (not including trivial inconsistency)  [Nishimura, Ragde, Szeider ’04] : “Strong” backdoors for Horn/2CNF equivalent to deletion backdoors

11. ISAIM-0811 Horn/2CNF backdoors  Horn formula – every clause has at most one positive literal  2CNF formula – every clause is binary  “Strong” Backdoors w.r.t. Horn/2CNF a subset B of variables such that for every truth assignment to B, the simplified sub-formula is Horn/2CNF (not including trivial inconsistency)  [Nishimura, Ragde, Szeider ’04] : “Strong” backdoors for Horn/2CNF equivalent to deletion backdoors  Aside: [Dechter ’92] Cycle cutset is a deletion backdoor w.r.t. the tractable class of acyclic CSPs Deletion once deleted from F, [Chandru, Hooker ’92]

12. ISAIM-0812 Deletion backdoors for Horn/2CNF  Deciding whether a formula has Horn/2CNF deletion backdoor of size k is fixed-parameter tractable (FPT)  O(f(k)n c ) – possibly exponential in k, but polynomial in n independent of k Why do we care about deletion backdoors? In general  Often easier to reason about and characterize (“syntactic”)  Deletion backdoors for a class are also “strong” backdoors  BUT, are deletion backdoors always as small as “strong” backdoors? (for Horn/2CNF: yes; what about richer classes?) [Nishimura, Ragde, Szeider ’04]

13. ISAIM-0813 Not The Case in General! Renamable Horn (RHorn) formulas  The formula is Horn up to renaming of the variables  Rename(v) : flip the sign of all literal occurrences of v Theorem. “Strong” backdoors w.r.t. RHorn can be exponentially smaller than deletion backdoors w.r.t. RHorn Proof idea:  given a strong backdoor set B,  for each assignment  to B, the simplified formula F[  /B] is RHorn  However, different  require different and mutually incompatible renamings to convert to Horn

14. ISAIM-0814  Deletion backdoors ignore the interplay between semantics of the constraints and value assignments  Thus, they capture only static tractable sub-structure  Dynamic structure analysis also much more powerful in other contexts, e.g.  dynamic caching for constraint reasoning beyond SAT [Bacchus CP-07 invited talk] Lesson? Ignoring truth value assignments may not always be a good idea

15. ISAIM-0815 How about inconsistency detection? What if the simplified formula is not Horn/2CNF but contains an empty clause?  Clearly unsatisfiable due to x 0  But smallest Horn backdoor has size n !!  Missing trivial solvability – empty clause detection

16. ISAIM-0816 Empty clause detection C {} : class of formulas containing an empty clause  Inconsistency detection is a key ingredient of backtrack solvers (distinguishes DPLL from naïve search)  size of C {} backdoors correlated with search effort by zChaff [Lynce et al ’04] Proposition: Strong backdoors w.r.t. C {} can be arbitrarily smaller than strong backdoors w.r.t. pure Horn/2CNF [from the example on previous slide]

17. ISAIM-0817 Our Results 1.Relaxing these two key features leads to exponentially larger backdoors 2.Finding strong backdoors can become harder* than NP just by adding inconsistency detection to tractable class (“static” notions trivially within NP, thus “weaker”) 3.State-of-the-art solvers do find very small strong backdoors (with inconsistency detection) 4.Preprocessing: small & mixed effect on backdoor size 5.Satisfiable instances: backdoors smaller w.r.t. PL than UP

18. ISAIM-0818 Backdoors w.r.t. Horn/2CNF  C {}  Horn/2CNF  C {}  sub-solver  Deletion backdoors  Strong backdoors  Deciding the existence of strong backdoor of size k w.r.t. pure Horn/2CNF is NP-complete [Nishimura et al ’04]  Intuitively speaking, not surprising since harder-to-find objects often capture richer structure more succinctly Theorem. Deciding the existence of a strong backdoor of size k w.r.t. Horn/2CNF  C {} is both NP-hard and coNP- hard.

19. ISAIM-0819 Our Results 1.Relaxing these two key features leads to exponentially larger backdoors 2.Finding strong backdoors can become harder* than NP just by adding inconsistency detection to tractable class (“static” notions trivially within NP, thus “weaker”) 3.State-of-the-art solvers do find very small strong backdoors (with inconsistency detection) 4.Preprocessing: small & mixed effect on backdoor size 5.Satisfiable instances: backdoors smaller w.r.t. PL than UP

20. ISAIM-0820 What matters in practice  Backdoors characterize hidden structure in interesting combinatorial problems  It is a key notion to understand the behavior of state-of-the- art solvers and their sophisticated propagation mechanisms  Backdoors provide powerful insights motivating new strategies for algorithm design (e.g., randomization, restarts, algorithm portfolios) Do problems exhibit small backdoors? Do SAT solvers exploit them?

21. An Experimental Study of Backdoors Finer grained insights into backdoors

22. ISAIM-0822 Experimental study Syntactic classes: Horn, Rhorn Algorithmic sub-solvers: UP (unit propagation), PL (pure literal elimination), UP+PL, UP+PL+probing (“branch-free Satz”, failed lits) Preprocessors: 2-Simplify, HyPre, SatELite, 3-Resolution Various problem domains: Graph coloring, logistics planning, car configuration, game theory (finding pure Nash eq.) Kilby et al. ’05: Satz-backdoor size correlated with problem hardness

23. ISAIM-0823 Finding minimum deletion backdoors  Decision version within NP (unlike strong backdoors)  For Horn (same as “strong” backdoors) and RHorn  Use Integer Programming: E.g for Horn, one {0,1} variable y i for each variable i in F Minimize size: Every clause should be Horn:

24. ISAIM-0824 Finding small strong backdoors  Not within NP  rely on indirect methods for finding a strong backdoor  Gives upper bound on the minimum strong backdoor size  Randomized version of Satz [Li and Anbulagan ’97]  a complete backtrack search solver that employs UP, PL, probing  can be easily configured to use selective propagation (e.g. only PL)  Using Satz to find (several) strong backdoors  Run Satz-Rand multiple times without restarts  Record the set B of vars not fixed by propagation mechanism S  Can verify: B is a strong backdoor w.r.t. S [for unsat instances]

25. ISAIM-0825 Results: graph coloring  3 binary variables per graph node  Planted clique = backdoor of size 4x3=12 w.r.t. C {} 12/n UP+PL+probing optimalupper bounds |B|/n

26. ISAIM-0826 Results: graph coloring  3 binary variables per graph node  Planted clique = backdoor of size 4x3=12 w.r.t. C {}  Propagation sub-solvers find even smaller backdoors  Horn and RHorn backdoors ignore constraint semantics  backdoor size linear in number of variables optimalupper bounds

27. ISAIM-0827 Results: logistics planning  SATZ backdoors of size 0 !!  UP+PL+probing solves these problems without branching  UP and UP+PL also excellent at finding the core inconsistencies  Horn and RHorn backdoors quite large  48%

28. ISAIM-0828 Results: car configuration  RHorn deletion backdoors are also very small  The original formulas are almost Horn  Again SATZ is excellent at finding small backdoor sets

29. ISAIM-0829 Results: game theory  Games on random graphs G(n,p) with avg. degree 3  SATZ backdoors again almost all of size 0  UP backdoors of size 6 to 8 critical players  Horn and RHorn backdoors quite large  66%

30. ISAIM-0830 Our Results 1.Relaxing these two key features leads to exponentially larger backdoors 2.Finding strong backdoors can become harder* than NP just by adding inconsistency detection to tractable class (“static” notions trivially within NP, thus “weaker”) 3.State-of-the-art solvers do find very small strong backdoors (with inconsistency detection) 4.Preprocessing: small & mixed effect on backdoor size 5.Satisfiable instances: backdoors smaller w.r.t. PL than UP

31. ISAIM-0831 Effect of Preprocessors How does preprocessing affect backdoor size? [Refer to paper for details]  Considered 2-Simplify, HyPre, SatELite, 3-Resolution  Compared to backdoor size without preprocessing  Key observations  Backdoor size in the same order of magnitude (e.g. 156-219 vars for UP+PL on an instance)  Mixed effect overall: often decreases but can increase  SatELite usually gave the smallest backdoors  Preprocessing often simplifies formula but sometimes obfuscates hidden structure

32. ISAIM-0832 Backdoors for Satisfiable Instances How do strong backdoors behave for satisfiable instances?  Satisfiable formulas mostly studied w.r.t. weak backdoors, although strong backdoors are also applicable [Refer to paper for details]  Considered satisfiable car configuration instances  Also tested with various preprocessing techniques  Key observations:  PL yields smaller strong backdoors than UP  Perhaps because pure literal elimination is a “streamliner”  Preprocessing does not have any significant effect

33. ISAIM-0833 Summary  The notion of backdoor sets takes into account two key features of state-of-the-art constraint solvers: 1.interplay between variable assignments and constraint semantics 2.inconsistency detection  These features are critical for small backdoors  In the worst-case, backdoor detection is not in NP (unless PH collapses to NP)  Despite this, in practice, state-of-the art solvers do find surprisingly small backdoors

34. ISAIM-0834 Future Directions  Study of backdoors in CSPs, MIP, …  Semantics of backdoors  Backdoors in the context of learning/caching, and restarts  Finer theoretical characterization of the complexity of backdoor detection