Analysis of Algorithms: Random Satisfiability Benny Applebaum Introduction

Department should schedule the following courses: Discrete Math, Physics1, Linear Algebra, Differential Methods, Matlab, Physics2, Intro to CS, Differential Equations, Data-Structures, Logic-Design Constraints: DiscMath, Phys1, LinAlg,DiffMeth can’t be at the same time Dr. No: wants to teach LinAlg and DiffEq at the same day Dr. Phil (Phys1, Phys2): hates Mondays Dr. Evil (LogDes) wants to arrive late on Wednesdays (after 2pm) …. Hard to satisfy all constraints…. (Is it possible at all?) The Time-Table Problem

Given: n variables x 1,…,x n over domain X and m constraints: f 1 (x 1,x 2,x 3 )=a 1 … f m (x 1,x 10,x 11,x 50 )=a m Goal: find a satisfying assignment Abstraction: Constraint Satisfaction Problems Time table example: For each course define 2 vars (Day, Time): - (D DiscMath, T DiscMath ), (D Phy1, T Phy1 ), (D LinAlg, T LinAlg ), (D DiffMet, T DiffMet ), … Constraints: -(D DiscMath, T DiscMath )  (D Phy1, T Phy1 ) - D LinAlg = D DiffMet AND D LinAlg  D DiffMet -…

Given: n variables x 1,…,x n over domain X and m constraints: f 1 (x 1,x 2,x 3 )=a 1 … f m (x 1,x 10,x 11,x 50 )=a m Goal: find a satisfying assignment The power of Constraint Satisfaction Problems CSPs can be found everywhere: circuit-design (placement, routing, testing) compiler optimization/scheduling computer graphics, image processing decoding error-correcting codes computational biology, economics, physics, chemistry, sociology Universality: solving CSP allows to solve all NP-problems !

k-CNF: n variables x 1,…,x n over binary domain {0,1} constraints in conjunctive normal form: x 1  x 2  x 3 (  x 2 )  x 4  (  x 5 ) … x 2   x 10  x 11 Each constraint contains at most k variables The special case of k-SAT K-SAT problem: given k-CNF find a satisfying assignment if it exists Special case of CSPs …. But still universal !!! Cook-Levin Thm: 3-SAT is NP-hard - efficiently solving 3-SAT  efficiently solving all NP Literal (variable or its negation) Clause

Coping with NP-hardness The P  NP assumption: There is no efficient algorithm that always solves 3-SAT It is widely believed that Assumption holds But proving it is one of the most important problems in science The assumption talks about worst-case behavior Q: Is there an efficient algorithm that solves 3-SAT on “most” instances? Depends on the distribution of instances Subject of our course !

Random CNFs F 3 (n,m)= uniform distribution over 3-CNFs: n variables x 1,…,x n m random constraints C 1,…C m - Choose C i independently and uniformly from all possible 3-clauses -(n choose 3)*2 3 options for each clause F k (n,m)= uniform distribution over k-CNFs - how many possibilities per clause ? Random Graph: n m variables clauses x1xnx1xn x1x2 x5x1x2 x

More Clauses  Less Solutions Clauses: 1. x 1  x 2  x 3 2. (  x 2 )  x 4  (  x 5 ) 3. x 2   x 10  x … 5.… 6.… 7.… 8.… 9.… 10.… All assignments {0,1} n Satisfying UnSatisfying 000********* *1*01****** *0*******10*

K-CNFs: Phase-Transition vs. Complexity Let r=m/n Clause-Variables ratio all assignments {0,1} n 2 k ln2±k Sat UnSat SAT threshold Satisfiable Pr[F k (n,rn) sat]  1 Algorithmic Barrier 2 k /k UnSatisfiable Pr[F k (n,rn) sat]  0 Efficient Algorithms Interesting connection: complexity depends on geometry of solutions?

Course Plan Let r=m/n Clause-Variables ratio all assignments {0,1} n 2 k ln2±k Sat UnSat SAT threshold Satisfiable Algorithmic Barrier 2 k /k UnSatisfiable (whp) Efficient Algorithms (1) Solving sparse instances (2) Solving dense satisfiable instances (3) Applications: Hard CSPs  fast crypto Easy CSPs  fast error-correction

Two Frameworks for Algorithms DPLL(CNF  ): While there exists an unassigned variable x i Try x i =1, recurse on  |x i =1 If “failed”, Try x i =0, recurse on  |x i =0 If “failed” output “unsat” Output assignment LocalSearch(CNF  ): Initialize assignment  For T iterations If  satisfies  output  Update  to  ’ via a “local rule” Output failed to find an assignment How to choose the next var? Which value (0,1) to try first? Where to start? How to modify assignment ?

Phase-Transition vs. Computational Complexity Let r=m/n Clause-Variables ratio {0,1} n r<4 r>4.5 r  4.2 Pr[F 3 (n,rn) sat]  1 Sat UnSat Pr[F 3 (n,rn) sat]  1/2 Pr[F 3 (n,rn) sat]  0 Efficient Algorithms Yes Unknown! (seems Hard…) Interesting connection: complexity depends on statistical structure?

Efficient Algorithms K-SAT: The Sparse Regime Let r=m/n Clause-Variables ratio {0,1} n 2 k ln2±k Sat UnSat SAT threshold Satisfiable But no algorithms Algorithmic Barrier 2 k /k UnSatisfiable (whp)

Efficient Algorithms K-SAT: The Dense Regime Let r=m/n Clause-Variables ratio {0,1} n 2 k ln2±k Sat UnSat SAT threshold Satisfiable But no algorithms Algorithmic Barrier 2 k ln(k)/k UnSatisfiable (whp)