1. Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems by Carla P. Gomes, Bart Selman, Nuno Crato and henry Kautz Presented by Yunho Kim Provable Software Lab, KAIST

2. Contents Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 2/28 Introduction Search procedures and problem domains Cost distributions of backtrack search Consequences for Algorithm Design Conclusion

3. Introduction(1/4) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 3/28 The DPLL algorithm is a complete algorithm for deciding the satisfiability of propositional logic formulas – It is guaranteed that eventually either the DPLL algorithm finds a satisfying model or proves the formula is unsatisfiable The iterative version of DPLL algorithm 1 status = preprocess(); 2 if (status!=UNKNOWN) return status; 3 while(1){ 4 decide_next_branch(); 5 while(1){ 6 status = deduce(); 7 if (status == CONFLICT){ 8 blevel = analyze_conflict(); 9 if (blevel == 0) 10 return UNSAT; 11 else backtrack(blevel); 12 } 13 else if (status == SAT) 14 return SAT; 15 else break; 16 } 17 }

4. Introduction(2/4) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 4/28 At each step a heuristic is used to select the next branch variable – A branch heuristic scores each variable in some manner and select the highest one Randomization can be used for tie-breaking – If several choices are ranked equally, choose among them at random – All variables that receive scores within H -percent of the highest score are considered equally good H is a heuristic equivalence parameter Empirically it is known that randomized branching heuristics is effective on hard instances

5. Introduction(3/4) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 5/28 Problem instance: quasigroup completion problem (N = 11, 30% pre-assignments) The sample mean of the number of backtracks does diverge Sample mean (number of backtracks) Number of runs

6. Introduction(4/4) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 6/28 The authors have observed the erratic behavior of the mean and the variance of the search cost distributions on a same instance The mean cost calculated over an increasing number of runs, on the same satisfiable problem instance, of a randomized backtrack search procedure does diverge. The authors have not found unsatisfiable instances with heavy-tailed behavior

7. Contents Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 7/28 Introduction Search procedures and problem domains Cost distributions of backtrack search Consequences for Algorithm Design Conclusion

8. Search Procedures Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 8/28 The authors modified two state-of-the-art SAT solvers(at that time), Satz and Relsat Both solvers hire similar occurrences-based decision heuristics Satz employs chronological backtracking while Relsat uses non-chronological backtracking

9. Quasigroup Completion Problem(1/2) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 9/28 A quasigroup is an ordered pair( Q, ¢ ), where Q is a set and ( ¢ ) is a binary operation on Q such that the equations a ¢ x = b and y ¢ a = b are uniquely solvable for every pair of elements a, b in Q The order N of the quasigroup is the cardinality of the set Q The multiplication table of a finite quasigroup is a Latin square – An N £ N table filled with n different symbols in such a way that each symbol occurs exactly once in each row and exactly once in each column Order 4 quasigroupOrder 10 quasigroup

10. Quasigroup Completion Problem(2/2) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 10/28 The quasigroup completion problem – determining whether the remaining entries of the partial Latin square can be filled in such a way that we obtain a complete Latin square The quasigroup completion problem is NP-complete 32% pre-assigned

11. Other Problems Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 11/28 Timetabling, planning and instances in the Dimacs Challenge benchmark are also considered Timetabling problem is to determine whether there exists a feasible schedule that consider a set of pairing and distribution constraints Planning is to find a sequence of actions that transform an initial state to a goal state

12. Contents Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 12/28 Introduction Search procedures and problem domains Cost distributions of backtrack search Consequences for Algorithm Design Conclusion

13. Cumulative Distribution(1/2) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 13/28 Data was produced by running the randomized backtrack search procedure 10,000 times on the same instance Even though 50% of the runs solve the instance in 1 backtrack or less, after 100,000 backtracks 0.5% of the runs were still not completed Number of backtracks Cumulative fraction of successful runs Completion of quasigroup

14. Cumulative Distribution(2/2) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 14/28 A solution is found in 1,000 backtracks or less in 80% of runs However, 5% of the runs do not result in a solution even after 1,000,000 backtracks Number of backtracks Cumulative fraction of successful runs Timetabling

15. Heavy-Tailed Distributions(1/3) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 15/28 Standard Exponential decay e.g. Normal: P{ X > x } ~ Ce - x 2 for some C > 0 Heavy-Tailed Power law decay e.g. Pareto-Levy: P{ X > x } ~ Cx - ® where for some 0 0 Power Law Decay Standard Distribution (finite mean & variance) Exponential Decay

16. Heavy-Tailed Distributions(2/3) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 16/28 We consider distributions that asymptotically have “heavy tails”, namely, where for some 0 0 The ® is referred to as the index of stability of the distribution – The lower the index, the heavier the tail Heavy-tailed distributions have finite/infinite mean and infinite variance P { X > x } ~ Cx - ® 0 < ® · 11 < ® MeanInfiniteFinite VarianceInfinite

17. Heavy-Tailed Distributions(3/3) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 17/28 Comparison of tail probabilities P{X > c} – Cauchy distribution is the heavy-tailed distribution which has ® = 1.0 – Levy distribution is the heavy-tailed-distribution which has ® = 0.5

18. Visual Check(1/3) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 18/28 Log-log plot of the cost distribution of the satisfiable completion of quasigroups instances 1- F ( x ) = P{ X > x } ~ Cx - ® N = 15, 40% pre-assignments Completion of quasigroups Log(1-F(x)) N = 15, 30% pre-assignments N = 11, 30% pre-assignments Log number of backtracks Log(1- F ( x )) ~ - ® Log( x ) + C ’

19. Visual Check(2/3) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 19/28 Log-log plot of the cost distribution of the satisfiable timetabling instance 1- F ( x ) = P{ X > x } ~ Cx - ® Completion of timetabling Log(1-F(x)) Log number of backtracks

20. Visual Check(3/3) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 20/28 Log-log plot of the cost distribution of the satisfiable logistics planning from two different SAT solvers 1- F ( x ) = P{ X > x } ~ Cx - ® Logistics planning Log(1-F(x)) Log number of backtracks Satz Relsat

21. Estimation of ® Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 21/28 Calculated maximum likelihood estimates of ® using Hill estimator – k is sample size Since ® · 1, mean and variance are infinite

22. Unsatisfiable Instance Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 22/28 Log-log plot of the cost distribution of unsatisfiable completion of quasigroups instnces 1- F ( x ) = P{ X > x } ~ Cx - ® Completion of quasigroups Log(1-F(x)) Log number of backtracks

23. Contents Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 23/28 Introduction Search procedures and problem domains Cost distributions of backtrack search Consequences for Algorithm Design Conclusion

24. Restarts Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 24/28 Restart after a fixed number of backtracks prevent a solver from entering pitfall Total number of backtracks Effect of restarts on a quasigroup instance N = 20, 5% pre-assignments Log(1-F(x)) No restarts With restarts Without restarts and given a total of 300 backtracks, 70% of runs failed With restarts, only 0.01% of runs failed

25. Restarts Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 25/28 Randomized rapid restarts(RRR) show better performance than deterministic

26. Contents Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 26/28 Introduction Search procedures and problem domains Cost distributions of backtrack search Consequences for Algorithm Design Conclusion

27. Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 27/28 The authors show the suitability of heavy-tailed distributions in modeling the runtime behavior of DPLL SAT solver with the random decision heuristic Restarts can exploit the mass of probability on the left of the cost distributions

28. Reference Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 28/28 Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems by Carla P. Gomes, Bart Selman, Nuno Crato and Henry Kautz in Journal of Automated Reasoning 24: 67-100, 2000

29. Lévy Distribution(1/2) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 29/28 Probability density function of Lévy distribution – Lévy have infinite mean and variance

30. Lévy Distribution(2/2) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST 30/28 Cumulative distribution function of Lévy distribution