1. 1 Paul Beame University of Washington Satisfiability and Unsatisfiability: Proof Complexity and Algorithms

2. 2 Outline Intro to proof complexity Proof complexity and complete SAT algs  Review Resolution/DPLL connection  Application to random k-SAT  Proof complexity and DPLL extensions Clause learning Caching Proof system survey

3. 3 Proof Complexity Study of the number of symbols required for proofs of unsatisfiability (or tautology) in propositional logic Does not address algorithmic issue  How would you find short proofs if they existed? Existence of short proofs for every unsatisfiable formula is equivalent to NP = co-NP (and is implied by P=NP)  Generally believed that such proofs don’t exist

4. 4 Propositional proof systems A propositional proof system is a polynomial time algorithm V s.t. for all formulas F F is unsatisfiable  there exists a string P s.t. V accepts input (P,F) Note:   direction is usually called soundness   direction is usually called completeness Proof complexity is size of P as a function of the size of F

5. 5 Sample propositional proof systems Truth tables  proof is a fully filled out truth table easy to verify that it is filled out correctly and all truth assignments yield T Frege systems  inference rules: e.g. modus ponens A, (A  B) | B  axioms: e.g. excluded middle | (A   A)  axioms & inference rules are schemas can make consistent substitution of arbitrary formulas for variables in schema e.g. excluded middle yields ((x  y)   (x  y))

6. 6 The graph of a proof Axioms/inputs are sources F7F7 F1F1 F 11 F3F3 F4F4 F8F8 F9F9 F 12 F2F2 F 10 F5F5 F6F6 F 13 Inference rule associated with each node Sink labelled by tautology (or  for refutation)

7. 7 Resolution Start with clauses of CNF formula F Resolution rule  Given (A  x), (B  x) derive (A  B) The empty clause is derivable  F is unsatisfiable Proof size = # of clauses used

8. 8 Restricted forms of Resolution Resolution  In general, graph of inferences is a directed acyclic graph Tree Resolution  Graph of inferences forms a binary tree i.e., if you want to use a clause more than once you need to re-derive it Regular Resolution  Once a variable is resolved out of a clause you can’t add it back later  More general than Tree Resolution

9. 9 Satisfiability Algorithms Incomplete Algorithms  Local search GSAT [Selman,Levesque,Mitchell 92] Walksat [Kautz,Selman 96]  Belief Propagation SP [Braunstein, Mezard, Zecchina 02] Complete Algorithms  Backtracking search DPLL [Davis,Putnam 60] [Davis,Logeman,Loveland 62] DPLL + clause learning GRASP, SATO, zchaff, berkmin  etc

10. 10 Proof Complexity and SAT Algorithms Proof Complexity  Unsatisfiable formulas  Nondeterministic algorithms SAT Algorithms  Satisfiable formulas  Deterministic or randomized algorithms

11. 11 Proof complexity and complete satisfiability algorithms In general  The transcript of the run of a complete satisfiability algorithm on an unsatisfiable formula is a proof of its unsatisfiability For backtracking style algorithms  Even runs on satisfiable formulas yield proofs of unsatisfiability of related formulas

12. 12 Backtracking search/DPLL DPLL(F) while F contains a 1-clause x F  F| x=1 if F is empty return 1 if F has an empty clause return 0 else select a literal x to branch on return DPLL(F| x=1 )  DPLL(F| x=0 ) Residual formula

13. 13 Resolution and DPLL DPLL tree with any variable selection rule on an unsatisfiable formula F generates a Resolution refutation of F  # of clauses  running time Note: Regular resolution corresponds to a similar DAG (read-once branching program) rather than a tree but general resolution has no such analogue

14. 14 DPLL Refutation Clauses 1. a  b  c 2. a  b 3.  c 4.  a  d 5.  d  b a bb cd 33 2145 aa a bb dd bb b d b a b ca b c a  c bb bb a da d d bd b cc c

15. 15 DPLL Refutation = Tree-like Resolution Proof Clauses 1. a  b  c 2. a  c 3.  b 4.  a  d 5.  a  b a:a: b: a b:  a d:a bd:a b 33 2 145 a b ca b c acac bb bb a da d d bd b c : a  b

16. 16 Random k-CNF formulas Randomly choose m clauses over n variables independently each of size k  Each size k clause is equally likely Key parameter r = m/n, the ratio of clauses to variables: Threshold value r k * of r Below, almost certainly satisfiable Above, almost certainly unsatisfiable Hardest problems near threshold

17. 17 DPLL on random 3-CNF* 0 1 probability satisfiable 4.267 ratio of clauses to variables # of DPLL backtracks * n = 50 variables Proof complexity shows 2  (n/r) time is required for unsatisfiable formulas for r  r 3 * [B,Karp,Saks,Pitassi 98] [Ben-Sasson 02] What about satisfiable formulas below threshold? r [Mitchell,Selman,Levesque 92]

18. 18 Some simple select choices for DPLL algorithms UC: Unit Clause/Ordered DLL  Choose variables in a fixed order  Always set True first UCwm: Unit Clause with majority  Choose variables in a fixed order  Apply a majority vote among 3-clauses for assigning each value GUC: Generalized Unit Clause  Choose a variable v in a shortest clause C  Set v to satisfy C

19. 19 Parameters of residual formulas follow trajectories 1 2/3 [Chao,Franco 88] [Frieze,Suen 95] [Achlioptas 00] [Achlioptas,Sorkin 00] UC GUC 3.52 3-clause ratio 2-clause ratio 8/3 [Kaporis et al 03] 0

20. 20 Satisfiability for mixed random formulas: proven properties 1 4.501 SAT UNSAT 3.52 2/3 ? ? ? ? ? ? ? ? ? ? ? ? 2.28 3-clause ratio 2-clause ratio [Achlioptas et al 96] [Kaporis et al 03] [Dubois 01]

21. 21 Resolution proof complexity of mixed random formulas Theorem: [Achlioptas, B, Molloy 01] For any constants r 2  1 and r 3  0 random formulas with r 2 n 2-clauses and r 3 n 3-clauses need exponential-size resolution proofs almost surely Extends [Chvatal-Szemeredi 88]

22. 22 Long-running DPLL Executions and Proof Complexity Residual formula at is unsatisfiable Algorithm’s proof of unsatisfiability is exponentially long Every resolution 2n2n Residual formula at each node is a mix of 2- and 3-clauses

23. 23 Trajectory on 3-CNF 1 UC Algorithm Trajectory 2-clause ratio 4.51 Provably UNSAT & Hard 3.524.267 Provably SAT & Easy 3-clause ratio 3.81 0

24. Exponential lower bounds for 3-CNF formulas below ratio 4.267 Corollary For almost all 3-CNF formulas, above ratio  3.81 UC takes exponential time  3.83 UCwm takes exponential time  4.01 GUC takes exponential time

25. 25 Exponential lower bound below the proven k-CNF threshold [Achlioptas, Peres 02] r k *  2 k ln 2 – (k+4)/2 [Achlioptas, B, Molloy 04] For k  4, for almost all k-CNF formulas UC takes exponential time above ratio c 2 k /k Note These formulas have huge numbers of satisfying assignments (more than 2 (1-  ) n out of a possible 2 n ) but still are hard

26. 26 Open Problem Closing gap for unsatisfiability of mixed formulas would yield sharp threshold behavior for each algorithm A  Below r A algorithm runs in linear time  Above r A algorithm requires exponential time Conjecture: no polynomial time selection heuristic will work up to the threshold  r A  r 3 * for all algorithms A Backtracking algorithms for other random problems with phase transitions?  e.g. k-colorability on random graphs G(n,r/n) Unsatisfiable phase exp(cn/r  k ) [B, Culberson, Mitchell, Moore 03]

27. 27 Outline Intro to proof complexity Proof complexity and complete SAT algs  Review Resolution/DPLL connection  Application to random k-SAT  Proof Complexity and DPLL extensions Clause learning Caching Proof system survey

28. 28 Clause Learning At each backtrack point in DPLL search add new clauses to the input formula that correspond to known causes of failure of the search  Critical to the good behavior of the best current algorithms What is the power of clause learning?  Intuitively: it makes DPLL trees DAG-like   regular resolution?  general resolution?

29. 29 Conflict Graph Decision scheme (p  q   b) 1-UIP scheme (t) pp qq b a x1x1 x2x2 x3x3 y yy false tt Known Clauses (p  q  a) (  a   b   t) (t   x 1 ) (t   x 2 ) (t   x 3 ) (x 1  x 2  x 3  y) (x 2   y) Current decisions p  false q  false b  true

30. 30 Proof complexity and clause learning Clause learning is no stronger than Resolution  Implication graph yields resolution derivation of each learned clause There are examples where clause learning works in polynomial time but any Regular Resolution proof is exponential [Sabharwal, B, Kautz 04]  Clause learning can sometimes be much stronger than Regular Resolution

31. 31 Proof complexity and clause learning Separation is true for any proper subsystem R of Resolution for which refuting F| x is no harder than F  Clause learning does not have this property unless it is as powerful as resolution Idea: Take formula F easy for Resolution but hard for R and create new formula F’ that is F plus clauses for (C  p C ) for each C in Resolution proof of F where each p C is a new propositional variable  Setting each p C to true yields original F but branching on sequence of  p C allows sequence of clauses C to be learned

32. 32 Still Open Can clause learning do everything regular resolution can?* Is clause learning as powerful as general resolution? * Note: Alan Van Gelder has pointed out that if one returns learned clauses up the tree and resolves them while backtracking then clause learning does efficiently simulate regular resolution

33. 33 Proof Complexity and Formula Caching Formula Caching: Memoization  Cache residual formulas that have been already found to be unsatisfiable  Do not make recursive call if the current residual formula is already cached

34. 34 DPLL(F) if F is empty report satisfiable and halt if F contains the empty clause  backtrack else select a literal x DPLL(F| x ) DPLL(F|  x ) DPLL

35. 35 FC(F,L) if F is empty report satisfiable and halt if F contains the empty clause  or F  L return else choose a literal x FC(F| x,L) FC(F|  x,L) add F to L Basic Formula Caching Start with FC( F,  )

36. 36 Proof Complexity of Formula Caching [B, Impagliazzo, Pitassi, Segerlind 03] Basic Formula Caching can exponentially improve DPLL Basic Formula Caching is not as powerful as regular resolution  but some of its natural extensions are These natural extensions can be exponentially more powerful than general resolution or even Res(k) for any k !  Res(k) is like resolution but uses k-DNFs instead of clauses

37. 37 Extensions of Formula Caching Don’t just check membership in the cache  F is trivially unsatisfiable given G iff every clause of G contains some clause of F Record the reasons why residual formulas are known to be unsatisfiable and check the reasons rather than the formulas

38. 38 FC*(F,L) if F is empty report satisfiable and halt if F is trivially unsatisfiable given L add F to L return else choose a literal x FC*(F| x,L) FC*(F|  x,L) add F to L More General Formula Caching

39. 39 FC reason (F,L) if F is empty report satisfiable and halt if F is trivially unsatisfiable based on G  L add F to L return(G) else choose a literal x G  FC reason (F| x,L) H  FC reason (F|  x,L) J  (  x  G) (x  H) add F,J to L return(J) Adding Reasons

40. 40 Combining Clause Learning and Formula Caching Cachet system used to count # of satisfying assignments [Sang, Bacchus, B., Kautz, Pitassi 04]  Built on zchaff’s clause learning  Uses decomposition of residual formula into components  Caches components rather than the whole formula  Subtle interactions

41. 41 Outline Intro to proof complexity Proof complexity and complete SAT algs  Resolution/DPLL review  Application to random k-SAT  DPLL extensions Clause learning Caching Proof system survey

42. 42 Proof Systems Hierarchy Truth Tables DPLL Nullstellensatz Polynomial Calculus Resolution Cutting Planes Frege AC 0 -Frege ZFC Ext-Frege PCR LS Res(k)

43. 43 AC 0 -Frege Constant-depth Frege (AC 0 -Frege)  Like Frege but formulas in proof only have constant # of alternations between  and   Resolution is depth-1 Frege

44. 44 Some Known Hard Problems for Resolution Exact/Modular Counting  Pigeonhole principle: n+1 pigeons can’t fit into n pigeonholes without getting cosy with each other  Parity Principle: e.g. can’t pair up an odd number of people  Tseitin Tautologies  These problems are also hard for AC 0 -Frege Random Unsatisfiable  k-CNF  Graph k-colorability  Existence of k-cliques in graphs

45. 45 Cutting Planes/PseudoBoolean Introduced to relate integer and linear programming [Gomory 59, Chvatal 73] :  Objects are linear integer inequalities  Clause (x 1   x 2  x 3 ) becomes inequality x 1 +(1-x 2 )+x 3  1  Add inequalities x i  0 and 1-x i  0 Goal: derive 0  1

46. 46 Cutting Planes rules addition: multiplication by positive integer: Division by positive integer: a 1 x 1 +... + a n x n  A b 1 x 1 +... + b n x n  B (a 1 +b 1 )x 1 +...+(a n +b n )x n  A+B a 1 x 1 +... + a n x n  A ca 1 x 1 +... + ca n x n  cA ca 1 x +... + ca n x n  B a 1 x 1 +... + a n x n   B/c 

47. 47 Limitations of Cutting Planes At least as good as resolution Can also handle  pigeonhole principle  parity principle But can’t handle counting when it is a derived property  Clique-Coloring A graph can’t have a k-clique and be k-1-colorable Tseitin tautologies still open

48. 48 Proof Systems Hierarchy Truth Tables DPLL Nullstellensatz Polynomial Calculus Resolution Cutting Planes Frege AC 0 -Frege ZFC Ext-Frege PCR LS Res(k)

49. 49 Hilbert’s Nullstellensatz S ystem of polynomials Q 1 (x 1,…,x n )=0,…,Q m (x 1,…,x n )=0 over field K has no solution in any extension field of K  there exist polynomials P 1 (x 1,…,x n ),…,P m (x 1,…,x n ) in K[x 1,…,x n ] s.t.

50. 50 Nullstellensatz proof system Clause (x 1   x 2  x 3 ) becomes equation (1-x 1 )x 2 (1-x 3 )=0 Add equations x i 2 -x i =0 for each variable  Guarantees only 0-1 solutions A proof is polynomials P 1,…, P m+n proving unsatisfiability: i.e. such that C QCQC

51. 51 Polynomial Calculus Similar to Nullstellensatz except:  Begin with Q 1,…,Q m+n as before  Given polynomials R and S can infer a  R + b  S for any a, b in K x i  R  Derive constant polynomial 1  Degree = maximum degree of polynomial appearing in the proof  Can find proof of degree d in time n O(d) using Groebner basis-like algorithm (linear algebra)

52. 52 PCR = PC + Resolution Two variables x and x’ for each proposition x  x’ stands for  x  include equations x+x’-1=0, x 2 -x=0, and (x’) 2 -x’=0 T ranslate (x 1   x 2  x 3 ) as x’ 1 x 2 x’ 3 =0 Same proof rules and proof search as polynomial calculus  Efficiently simulates resolution: clause  monomial  Degree-size relationship for PCR is same as width- size relationship of resolution

53. 53 Lovasz-Schrijver (LS) Proofs Linear inequalities Steps:  Lift to get degree 2 inequalities E.g. multiply inequalities  Project Combine degree 2 inequalities plus x 2 =x to cancel out degree 2 terms Captures properties of semi-definite programming

54. 54 Proof Systems Hierarchy Truth Tables DPLL Nullstellensatz Polynomial Calculus Resolution Cutting Planes Frege AC 0 -Frege ZFC Ext-Frege PCR LS Res(k)

55. 55 Conclusions Exchange of ideas between proof complexity and SAT algorithms can be fruitful  Better understanding of existing algorithms  Source of ideas for new algorithms and proof systems Potential to branch out beyond DPLL derivatives  Wide range of proof systems to explore

56. 56 Thank You Thank you to Toby Walsh Fahiem Bacchus Ian Miguel