1. 1 Paul Beame University of Washington Phase Transitions in Proof Complexity and Satisfiability Search Dimitris Achlioptas Michael Molloy Microsoft Research U. Toronto with

2. 2 Satisfiability F (x 1  x 2  x 4 ) (x 1  x 3 ) (x 3  x 2 ) (x 4  x 3 ) satisfying assignment for F: x 1, x 2, x 3, x 4 Given F does such an assignment exist?

3. 3 Satisfiability Algorithms Incomplete Algorithms  will (likely) find a satisfying assignment but will simply give up if one is not found Complete Algorithms  will either find a satisfying assignment or determine that no such assignment exists

4. 4 Satisfiability Algorithms Incomplete Algorithms  Local search GSAT [Selman,Levesque,Mitchell 92] Walksat [Kautz,Selman 96]  Belief Propagation SP [Braunstein, Mezard, Zecchina 02] Complete Algorithms  Backtracking search DPLL [Davis,Putnam 60] [Davis,Logeman,Loveland 62] DPLL + “clause learning” GRASP, SATO, zchaff

5. 5 Simplification and Satisfaction F (x 1  x 2  x 4 ) (x 1  x 3 ) (x 3  x 2 ) (x 4  x 3 ) satisfying assignment for F: x 1, x 2, x 3, x 4 Simplifying F after setting literal x 3 to true F (x 1  x 2  x 4 ) (x 1  x 3 ) (x 3  x 2 ) (x 4  x 3 ) F| x 3 (x 1  x 2  x 4 ) (x 2 ) (x 4 ) F is satisfied if all clauses disappear under simplification given the assignment 1-clauses

6. 6 Backtracking search/DPLL DPLL(F) while F contains a 1-clause l ’ F  F| l ’ if F has no clauses output ‘satisfiable’ halt if F has an empty clause backtrack else select a literal l = some x or x DPLL(F| l ) if backtrack then DPLL(F| l ) Residual formula

7. 7 Some standard select choices for DPLL algorithms UC: Unit Clause/Ordered DLL  Choose variables in a fixed order  Always set True first UCwm: Unit Clause with majority  Choose variables in a fixed order  Apply a majority vote among 3-clauses for assigning each value GUC: Generalized Unit Clause  Choose a variable v in a shortest clause C  Set v to satisfy C

8. 8 Random k-CNF formulas Distribution F k,n (r)  Randomly choose r  n clauses over n variables independently, each of size k  Each size k clause is equally likely Threshold value r k * r  r k *, almost certainly satisfiable r  r k *, almost certainly unsatisfiable Hardest problems near threshold

9. 9 DPLL on random 3-CNF* 0 1 probability satisfiable 4.267 ratio of clauses to variables # of DPLL backtracks * n = 50 variables Proof complexity shows 2  (n/r) time is required for unsatisfiable formulas for r  r 3 * [B,Karp,Saks,Pitassi 98] [Ben-Sasson 02] What about satisfiable formulas below threshold? r [Mitchell,Selman,Levesque 92]

10. Exponential lower bounds for 3-CNF formulas below ratio 4.267 r 3 UC = 3.81 r 3 UCwm = 3.83 r 3 GUC = 4.01 Theorem Let A  {UC, UCwm, GUC}. Let w.h.p. algorithm A takes exponential time on a random F  F 3,n (r) for r  r 3 A

11. 11 Exponential lower bounds for satisfiable formulas below the k-CNF threshold Theorem There exist l k  2 k /k and u k  2 k  s.t. for every k  4 and for F  F k,n (r) with l k  r  u k w.h.p. F is satisfiable UC takes exponential time on F Note These formulas have huge numbers of satisfying assignments (more than 2 (1-  ) n out of a possible 2 n ) but still are hard

12. 12 Ideas Part I: Use differential equations to analyze trajectory of algorithm as a function of the clause-variable ratio for r larger than l k Use resolution proof complexity to show that some residual formula along this trajectory requires large DPLL running time Part II: Show that formulas up to ratio u k are satisfiable [Achlioptas, Peres 03] u k =2 k ln 2 – (k+4)/2

13. 13 Algorithmic behavior using simple select choices On input F  F k,n (r) before the first backtrack occurs, the residual formula F’ is distributed as F 2  F k where  F j  F j,n’ (r j ) for j=2, ,k only has clauses of size k  F j are mutually independent Values of r j almost surely follow algorithm- dependent trajectories given by differential equations

14. 14 Proof Complexity Study of the number of symbols required for proofs of unsatisfiability (or tautology) in propositional logic Does not address algorithmic issue  How would you find short proofs if they existed? Existence of short proofs for every unsatisfiable formula is equivalent to NP = co-NP (and is implied by P=NP)  Generally believed that such proofs don’t exist Active research area with rich theory and many open questions

15. 15 Resolution Start with clauses of CNF formula F Resolution rule  Given (A  x), (B  x) can derive (A  B) The empty clause is derivable  F is unsatisfiable Proof size = # of clauses used

16. 16 Resolution and DPLL Running DPLL with any select rule on an unsatisfiable formula F generates a Resolution refutation of F  # of clauses  running time

17. 17 Backtracking search/DPLL DPLL(F) while F contains a 1-clause l ’ F  F| l ’ if F has no clauses output ‘satisfiable’ halt if F has an empty clause backtrack else select a literal l = some x or x DPLL(F| l ) if backtrack then DPLL(F| l ) Residual formula

18. Long-running DPLL Executions Residual formula at is unsatisfiable Algorithm’s proof of unsatisfiability is exponentially long Every resolution Residual formula at each node is a mix of 2- and 3-clauses 2n2n

19. 19 Satisfiability for mixed random formulas: proven properties 1 4.501 SAT UNSAT 3.52 2/3 ? ? ? ? ? ? ? ? ? ? ? ? 2.28 3-clause ratio 2-clause ratio [Achlioptas et al 96] [Kaporis et al 03] [Dubois 01]

20. 20 Resolution proof complexity of mixed random formulas Theorem A random CNF formula F  F 2,n (r 2 ) is  Satisfiable w.h.p. if r 2 <1  Unsatisfiable w.h.p if r 2 >1 and has linear size resolution proofs [Chvatal-Reed 91], [Goerdt 91], [De La Vega 91] Theorem For any constant r 3  0, w.h.p. G  F 3,n (r 3 ) requires an exponential-size resolution proof of unsatisfiability [Chvatal,Szemeredi 88] Theorem For any constants r 2  1 and r 3  0, w.h.p. for F  F 2,n (r 2 ) and G  F 3,n (r 3 ) the combined formula F  G requires an exponential-size resolution proof of unsatisfiability Easy Hard Easy  Hard = Hard

21. 21 Sharp Threshold in Resolution Proof Complexity Define distribution H n (r) on CNF formulas of the form H=F  G where  G  F 3,n (r 3 ) for some r 3  2.28 and  F  F 2,n (r). Then for H  H n (r) w.h.p.  H is unsatisfiable  For r  1, H has O(n) size resolution proofs  For r  1, H requires 2  (n) size resolution proofs

22. 22 Trajectory on 3-CNF 1 UC Algorithm Trajectory 2-clause ratio 4.51 Provably UNSAT & Hard 3.524.267 Provably SAT & Easy 3-clause ratio 3.81

23. 23 UC trajectory for k  4 Start with 2.75  2 k n/k k-clauses Wait until 3n/(k-1) variables remain With high probability:  The 2-clauses remained satisfiable throughout  The residual formula overall is unsatisfiable  Its resolution complexity is exponential

24. 24 Directions What price completeness? Closing gap for unsatisfiability of mixed formulas would yield an algorithm-dependent phase transition  Below r A algorithm runs in linear time  Above r A algorithm requires exponential time Backtracking algorithms for other random problems with phase transitions?  e.g. k-colorability on random graphs G(n,r/n) Unsatisfiable phase exp(cn/r  k ) [B, Culberson, Mitchell, Moore 03]