SAT 2009 Ashish Sabharwal Backdoors in the Context of Learning (short paper) Bistra Dilkina, Carla P. Gomes, Ashish Sabharwal Cornell University SAT-09 Conference Swansea, U.K., June 30,

SAT 2009 Ashish Sabharwal Boolean Satisfiability or SAT : –Given a Boolean formula F in conjunctive normal form e.g. F = (a or b) and (¬a or ¬c or d) and (b or c) determine whether F is satisfiable –NP-complete [note: “worst-case” notion] –widely used in practice, e.g. in hardware & software verification, design automation, AI planning, … Large industrial benchmarks (10K+ vars) are solved within seconds by state-of-the-art complete/systematic SAT solvers Even 100K or 1M not completely out of question Good scaling behavior seems to defy “NP-completeness”! Real-world problems have tractable sub-structure “Backdoors” help explain how solvers can get “smart” and solve very large instances SAT: Gap between theory & practice 2 not quite Horn-SAT or 2-SAT…

SAT 2009 Ashish Sabharwal (~500 vars) Informally: A backdoor to a given problem is a subset of its variables such that, once assigned values, the remaining instance simplifies to a tractable class. Formally: define a notion of a poly-time “sub-solver” handles tractable substructure of problem instance e.g. unit prop., pure literal elimination, CP filtering, LP solver, … Weak backdoors for finding feasible solutions Strong backdoors for finding feasible solutions or proving unsatisfiability Backdoors to Tractability 3 A notion to capture “hidden structure”

SAT 2009 Ashish Sabharwal The notion of backdoors has provided powerful insights, leading to techniques like randomization, restarts, and algorithm portfolios for SAT Are backdoors small in practice? 4 Enough to branch on backdoor variables to “solve” the formula  heuristics need to be good on only a few vars

SAT 2009 Ashish Sabharwal “Traditional” backdoors are defined for a basic tree-search procedure, such as pure DPLL –Oblivious to the now-standard (and essential) feature of learning during search, i.e, clause learning for DPLL Note: state-of-the-art SAT solvers rely heavily on clause learning, especially for industrial and crafted instances – provably leads to shorter proofs for many unsatisfiable formulas –significant speed-up on satisfiable formulas as well Does clause learning allow for smaller backdoors when capturing hidden structure in SAT instances? This Talk: Motivation 5

SAT 2009 Ashish Sabharwal Affirmative answer: 1.First, must extend the notion of backdoors to clause learning SAT solvers: take ‘order-sensitivity’ into account 2.Theoretically, learning-sensitive backdoors for SAT solvers with clause learning (“CDCL solvers”) can be exponentially smaller than traditional strong backdoors 3.Initial empirical results suggesting that in practice, –More learning-sensitive backdoors than traditional (of a given size) –SAT solvers often find much smaller learning-sensitive backdoors than traditional ones This Talk: Contribution 6

SAT 2009 Ashish Sabharwal Input: CNF formula F At every search node: –branch by setting a variable to True or False; current partial variable assignment:  –consider simplified sub-formula F|  –apply a poly-time inference procedure to F|  (e.g. unit prop., pure literal test, failed literal test / “probing”)  Contradiction  learn a conflict clause  Solution  declare satisfiable and exit  Not solved  continue branching “sub-solver” for SAT DPLL Search with Clause Learning 7

SAT 2009 Ashish Sabharwal Traditional Backdoor Backdoor { Sub-solver infers solution x y z w =1 =0 =1 { Backdoor? Search Tree to Solution Contradiction: Conflict clause learned Early contradiction due to previously learned clause Sub-solver infers solution with help from learned clauses x y y =0 =1 =0 Search order matters! Backdoors and Search with Learning 8

SAT 2009 Ashish Sabharwal Definition [Williams, Gomes, Selman ’03] : A subset B of variables is a strong backdoor (for F w.r.t. a sub-solver S) if for every truth assignment  to variables in B, S “solves” F| . Issue: oblivious to “previously” learned clauses; sub-solver must infer contradiction on F|  for every  from scratch. “Traditional” Backdoors 9 either finds a satisfying assignment for F or proves that F is unsatisfiable

SAT 2009 Ashish Sabharwal Definition: A subset B of variables is a learning-sensitive backdoor (for F w.r.t. a sub-solver S) if there exists a search order s.t. a clause learning solver –branching only on the variables in B –in this search order –with S as the sub-solver at each leaf “solves” F. New: Learning-Sensitive Backdoors 10 either finds a satisfying assignment for F or proves that F is unsatisfiable

Theoretical Results

SAT 2009 Ashish Sabharwal Setup: Sub-solver: unit propagation Clause learning scheme: 1-UIP Comparison w.r.t. traditional strong backdoors Theorem 1: There are unsatisfiable SAT instances for which learning-sensitive backdoors are exponentially smaller than the smallest traditional strong backdoors. Theorem 2: There are satisfiable SAT instances for which learning-sensitive backdoors are smaller than the smallest traditional strong backdoors. Learning-Sensitive Backdoors Can Provably be Much Smaller 12 used Rsat for experiments

SAT 2009 Ashish Sabharwal Proof Idea: Simple Example 13 { x } is a learning-sensitive backdoor (of size 1) : x=0 p1p1 p2p2 q ab contradiction Learn 1-UIP clause: (  q) x=1 ab contradiction qq r With clause learning, branching on x in the right order suffices to prove unsatisfiability (  x appears only in a “long” clause)

SAT 2009 Ashish Sabharwal Proof Idea: Simple Example 14 In contrast, without clause learning, must branch on at least 2 variables in every proof of unsatisfiability!  every “traditional” strong backdoor is of size ≥ 2 Why? every variable, in at least one polarity, only in “long” clauses e.g.,  p 1, q, r,  a do not appear in any 2-clauses therefore, no unit prop. or empty clause generation by fixing this variable to this value therefore, this variable by itself cannot be a strong backdoor

SAT 2009 Ashish Sabharwal Construct an unsatisfiable formula F on n vars. such that 1.certain long clauses must be used in every refutation (i.e., removing a long clause makes F satisfiable) 2.many variables in at least one polarity appear only in such long clauses with  (n) variables  Controlled unit propagation / empty clause generation  Must branch on essentially all variables of the long clauses to derive a contradiction  Such variables must be part of every traditional backdoor set 3.With learning: conflict clauses from previous branches on O(log n) “key variables” enable unit prop. in long clauses Proof Idea: Exponential Separation 15

SAT 2009 Ashish Sabharwal Corollary (follows from the proof of Theorem 1) : There are unsatisfiable SAT instances for which learning- sensitive backdoors w.r.t. one value ordering are exponentially smaller than the smallest learning-sensitive backdoors w.r.t. another value ordering. Order-Sensitivity of Backdoors 16

Experimental evaluation

SAT 2009 Ashish Sabharwal Learning-Sensitive Backdoors in Practice 18 Preliminary evaluation of smallest backdoor size Reporting “best found” backdoors over 5000 runs of Rsat (with clause learning) or Satz-rand (no learning) : up to 10x smaller than traditional on satisfiable instances often 2x or less smaller than traditional on unsatisfiable instances

SAT 2009 Ashish Sabharwal Considering only the size of the smallest backdoor does not provide much insight into this question One way to assess this difficulty: –How many backdoors are there of a given cardinality? Experimental setup: –For each possible backdoor size k, sample uniformly at random subsets of cardinality k from the (discrete) variables of the problem –For each subset, evaluate whether it is a backdoor or not How hard is it to find small backdoor sets with learning? 19 Recently reported in a paper at CPAIOR-09 (backdoors in the context of optimization problems)

SAT 2009 Ashish Sabharwal Backdoor Size Distribution 20 E.g., for a Mixed Integer Programming (MIP) optimization instance:

SAT 2009 Ashish Sabharwal Added Power of Learning 21 E.g., for a Mixed Integer Programming (MIP) optimization instance:

SAT 2009 Ashish Sabharwal Defined backdoors in the context of learning during search (in particular, clause learning for SAT solvers) Proved that learning-sensitive backdoors can be smaller than traditional strong backdoors –Exponentially smaller on unsatisfiable instances –Somewhat smaller on satisfiable instances (open?) Branching order affects backdoor size as well Future work: stronger separation for satisfiable instances; detailed empirical study Summary 22