1. 1 Exploiting Random Walk Strategies in Reasoning Wei

2. 2 Universal inference engine  One of the oldest AI dreams General Problem Solver (Newell & Simon, 1961) Use logic reasoning ( “ Program with Common Sense ”, McCarthy 1968)  Largely unsuccessful due to the tradeoff between computational complexity and representation power. => many successful systems are domain-specific (e.g. Dendral, 1968-85, organic chemistry)  Recent improvement in computer ’ s abilities to perform large-scale search => Revisit the original idea

3. 3 Computational Power - SAT  Hardware power  Average-case complexity  New algorithmic tools: randomization, learning  Increasing demand: verification  Research input: annual conference, competition year#variable 196250 1992300 19961,000 20021,000,000

4. 4 Boolean Satisfiability Problem  Boolean Satisfiability Problem (SAT) asks if a Boolean expression can be made true by assigning Boolean values to its variables.  The problem is well-studied in AI community with direct application in reasoning, planning, CSP, VLSI microprocessor verification etc.  Does statement s hold in world A (represented by a set of clauses)? A s  ( ¬ s) ^ A unsatisfiable

5. 5 SAT  SAT (even 3SAT) is NP-complete. Best theoretical bound so far is (2-2/k) N randomized (Schoening 1999) or (2-2/(k+1)) N deterministic (Dantsin et al 2002) for k-SAT  In practice, there are two different kinds of solvers DPLL (Davis, Logemann and Loveland 1962) Local Search (Selman et al 1992)

6. 6 DPLL (x 1   x 2  x 3 )  (x 1  x 2   x 3 )  (x 1  x 2 )  DPLL was first proposed as a basic depth- first tree search. x1x1 x2x2 FT T null F solution x2x2 Potential problem: early commitment

7. 7 DPLL  Recently (since late 90 ’ s), many improvements: Randomization restarts out-of-order backtracking clause learning

8. 8 Local Search (x 1   x 2  x 3 )  (x 1  x 2   x 3 )  (x 1  x 2 )  The idea: Start with a random assignment. And make local changes to the assignment until a solution is reached (010  011  001)  Pro: often efficient in practice. Sometimes the only feasible way for some problems  Con: Cannot prove nonexistence of solutions. Difficult to analyze theoretically.  Example GSAT (Selman et al. 1992)

9. 9 Local Search Schemes  local search schemes used: Simulated annealing Tabu search Genetic algorithms Random Walk and its variants  the most successful so far

10. 10 Unbiased (Pure) Random Walk for SAT Procedure Random-Walk (RW) Start with a random truth assignment Repeat c:= an unsatisfied clause chosen at random x:= a variable in c chosen at random flip the truth value of x Until a satisfying assignment is found

11. 11 Unbiased RW on any satisfiable 2SAT Formula  Given a satisfiable 2SAT formula with n variables, a satisfying assignment will be reached by Unbiased RW in O(n 2 ) steps with high probability. (Papadimitriou, 1991)

12. 12 We have an unbiased random walk with a reflecting barrier at distance N from T (max Hamming distance) and an absorbing barrier (satisfying assignment) at distance 0. We start at a Hamming distance of approx. ½ N. Property of unbiased random walks: after N^2 flips, with high probability, we will hit the origin (the satisfying assignment). (Drunkards walk) So, O(N^2) randomized algorithm (worst-case!) for 2- SAT. TA0 T reflecting

13. 13 Unbiased RW on 3SAT Formulas Random walk takes exponential number of steps to reach 0. T A0 reflecting

14. 14 Comments on RW 1) Random Walk is highly “ myopic ” does not take into account any gradient of the objective function (= number of unsatisfied clauses)! Purely “ local ” fixes. 2) Can we make RW practical for SAT? Yes --- inject greedy bias into walk  biased Random Walk.

15. 15 Biased Random Walk Procedure WalkSat Repeat c:= an unsatisfied clause chosen at random if there exist a variable x in c with break value = 0 // greedy bias flip the value of x ( “ freebie move ” ) else with probability p // pure walk x:= a variable in c chosen at random flip the value of x with probability (1-p) x:= a variable in c with smallest break value // more greedy bias flip the value of x Until a satisfying assignment is found Note: tune parameter p.

16. 16 Random Walk algorithms  Random walk algorithm (e.g. Walksat) offer significant improvement on performance over hill-climbing algorithms. IDVarsclausesGSAT+wWalksat Ssa7552-038150135751292.3 Ssa7552-15813633034902 Ssa7552-15913633032140.8 Ssa7552-16013913126181.5

17. 17 First, bringing out the worst in random walks… (2-SAT) X1  X2 X2  X3 X3  X4 Xn  X4 Xn  X1 Note: Only 2 satisfying assignments, all False and all True.

18. 18 Binary Chains Walk is exactly balanced.

19. 19 Results: Speeding up Random Walks on Binary Chains * : empirical results ** : proof available Pure binary chain Chain with redundancies RW (n 2 ) ** RWF (n 2 ) ** (n 1.2 ) * WalkSat (n 2 ) ** (n 1.1 ) * Becomes almost like unit prop.

20. 20 Ternary Chains In general, even a small bias in the wrong direction leads to exponential time to reach 0. reflecting

21. 21 What about 3-SAT? Again, consider “chain” formulas. X1 & X2  X3 X2 & X3  X4 X_(n-2) & X_(n-1)  X_n X1 & X2 X_floor(n/2) & X_(n-1)  X_n

22. 22 Theoretical Results on 3-SAT Chains Function low(i)Expected run time of pure RW i-2 (highly local) ~ Fib(n) (i.e., exp.) i/2 (interm. reach) ~ n log n (i.e., quasi-poly) log i (interm. reach) ~ n 2. (log n) 2 (i.e., poly) 1 (full back reach) ~ n 2 low(i) captures how far back the clauses reach.

23. 23 Recurrence Relations Our formula structure gives us: E(f(z i )) = (E(f(z low(i) ) + E(f(z i ) + 1) * 1/3 + (E(f(z i-1 ) + E(f(z i ) + 1) * 1/3 + 1 * 1/3  E(f(z i )) = E(f(z low(i) ) + E(f(z i-1 ) + 3

24. 24 Decompose: multiple “ 0 ” s 110101 010101010101 110001 010001 100001 111001 110001 101001 111001 111111 111101 110101 010101010101 110001 100001 111001 110001 110101 010001 101001 111001 111101 111111 110111 010111 110111 100111 111111 110111 111101 111001 101001 111001 111101 111111 110111  111111 111101  111111 Start Sat assign.

25. 25 Recap Chain Formula Results  Adding implied constraints capturing long-range dependencies speeds random walk on 2-Chain to near linear time.  Certain long-range dependencies in 3-SAT lead to poly-time convergence of random walks.  Can we take advantage of these results on practical problem instances? Yes! (next) Our methodology: Identify, analyze, and “ exploit ” special tractable structure in large practical reasoning problems.

26. 26 Empirical Results SSS-SAT-1.0 instances (Velev 1999). 100 total.  level of redundancy added (20% near optimal) Formulas (redun. level) <40 sec<400 sec <4000 sec  = 0.0 152642  = 0.2 8598100  = 1.0 133364

27. 27 Optimal Redundancy Rate Time vs Redundancy Rate Flips vs Redundancy Rate WalkSat(noise=50) on dlx2_cc_bug01.cnf from SAT-1.0 Suite

28. 28 Probabilistic Reasoning  Previously, we asked “ does statement s hold in world A (represented by a set of clauses)? ” A s  ( ¬ s) ^ A unsatisfiable  Now, what is the probability that statement s holds in world A? Pr = #( s ^ A ) / #A

29. 29 Probabilistic Reasoning  Close connection between counting and sampling (Jerrum et al, 1986).  Bayesian Net Queries can be encoded as #SAT. (Kautz, 2004)  How to Sample: By repeated counting using DPLL algorithms Monte Carlo Markov Chain Method Use state-of-the-art local search methods Note: Random Walk not uniform

30. 30 Characteristics of Solution space: Solution Clustering  Visualization with multi-dimensional scaling (MDS) Solutions to specific 75 variable, 325 clause 3-SAT instances 75 dimensional solution projected to two dimensions Distance between points approximates hamming distance

31. 31  Empirically determined each solution’s probability (uf75-01 - 75 variable, 325 clause 3-SAT instance)  WalkSat finds every solution, but with very large range of probabilities (1:10 4 )  Probability Clusters Solution Probability Using WalkSat Algorithm

32. 32 Probability Ranges in Different Domains InstanceRunsHits Rarest Hits Common Common- to -Rare Ratio Random50*10 6 539*10 5 1.7*10 4 Logistics1* 10 6 844*10 3 50

33. 33 Improving the Uniformity of Sampling - mixing sampling strategy  To reduce the range of probabilities, we propose a hybrid local search algorithm: With probability p, the algorithm makes a biased random walk move With probability 1-p, the algorithm makes a SA (simulated annealing) move  In our experiments, we used 50% WalkSat + 50% SA at a fixed temperature

34. 34 Results of the Hybrid Approach Our key figure.

35. 35 Solution Clusters Results on a random 3-SAT instance (70 vars, 301 clauses, 2531 solutions).

36. 36 Summary 1)WalkSAT does sample all solutions. 2)But, sampling can be highly biased. 3)Using a new hybrid strategy, we can obtain effective near-uniform sampling. Lesson: Hybrid of SA and biased walk, is a promising alternative to MCMC methods for sampling. Idea: Use SAT solvers to sample solutions from a combinatorial space. Findings:

37. 37 Research Directions  Can we exploit techniques from DPLL solvers, such as learned clauses (inferred structure) in local search?  Can we identify other classes of structure in real-world problem that can further accelerate random walk style SAT solvers?  Can we design better SAT encodings based on our insights about structure?

38. 38 Research Directions, cont.  Solution sampling Main Challenge: moving between clusters evaluation tools.  Randomize DPLL to sample solution space Early indicator of solution counts.  Compare with Bayesian/probabilistic state-of-the-art inference methods. Can we outperform them?