1 Exploiting Random Walk Strategies in Reasoning Wei

2 Universal inference engine  One of the oldest AI dreams General Problem Solver (Newell & Simon, 1961) Use logic reasoning ( “ Program with Common Sense ”, McCarthy 1968)  Largely unsuccessful due to the tradeoff between computational complexity and representation power. => many successful systems are domain-specific (e.g. Dendral, , organic chemistry)  Recent improvement in computer ’ s abilities to perform large-scale search => Revisit the original idea

3 Computational Power - SAT  Hardware power  Average-case complexity  New algorithmic tools: randomization, learning  Increasing demand: verification  Research input: annual conference, competition year#variable , ,000,000

4 Boolean Satisfiability Problem  Boolean Satisfiability Problem (SAT) asks if a Boolean expression can be made true by assigning Boolean values to its variables.  The problem is well-studied in AI community with direct application in reasoning, planning, CSP, VLSI microprocessor verification etc.  Does statement s hold in world A (represented by a set of clauses)? A s  ( ¬ s) ^ A unsatisfiable

5 SAT  SAT (even 3SAT) is NP-complete. Best theoretical bound so far is (2-2/k) N randomized (Schoening 1999) or (2-2/(k+1)) N deterministic (Dantsin et al 2002) for k-SAT  In practice, there are two different kinds of solvers DPLL (Davis, Logemann and Loveland 1962) Local Search (Selman et al 1992)

6 DPLL (x 1   x 2  x 3 )  (x 1  x 2   x 3 )  (x 1  x 2 )  DPLL was first proposed as a basic depth- first tree search. x1x1 x2x2 FT T null F solution x2x2 Potential problem: early commitment

7 DPLL  Recently (since late 90 ’ s), many improvements: Randomization restarts out-of-order backtracking clause learning

8 Local Search (x 1   x 2  x 3 )  (x 1  x 2   x 3 )  (x 1  x 2 )  The idea: Start with a random assignment. And make local changes to the assignment until a solution is reached (010  011  001)  Pro: often efficient in practice. Sometimes the only feasible way for some problems  Con: Cannot prove nonexistence of solutions. Difficult to analyze theoretically.  Example GSAT (Selman et al. 1992)

9 Local Search Schemes  local search schemes used: Simulated annealing Tabu search Genetic algorithms Random Walk and its variants  the most successful so far

10 Unbiased (Pure) Random Walk for SAT Procedure Random-Walk (RW) Start with a random truth assignment Repeat c:= an unsatisfied clause chosen at random x:= a variable in c chosen at random flip the truth value of x Until a satisfying assignment is found

11 Unbiased RW on any satisfiable 2SAT Formula  Given a satisfiable 2SAT formula with n variables, a satisfying assignment will be reached by Unbiased RW in O(n 2 ) steps with high probability. (Papadimitriou, 1991)

12 We have an unbiased random walk with a reflecting barrier at distance N from T (max Hamming distance) and an absorbing barrier (satisfying assignment) at distance 0. We start at a Hamming distance of approx. ½ N. Property of unbiased random walks: after N^2 flips, with high probability, we will hit the origin (the satisfying assignment). (Drunkards walk) So, O(N^2) randomized algorithm (worst-case!) for 2- SAT. TA0 T reflecting

13 Unbiased RW on 3SAT Formulas Random walk takes exponential number of steps to reach 0. T A0 reflecting

14 Comments on RW 1) Random Walk is highly “ myopic ” does not take into account any gradient of the objective function (= number of unsatisfied clauses)! Purely “ local ” fixes. 2) Can we make RW practical for SAT? Yes --- inject greedy bias into walk  biased Random Walk.

15 Biased Random Walk Procedure WalkSat Repeat c:= an unsatisfied clause chosen at random if there exist a variable x in c with break value = 0 // greedy bias flip the value of x ( “ freebie move ” ) else with probability p // pure walk x:= a variable in c chosen at random flip the value of x with probability (1-p) x:= a variable in c with smallest break value // more greedy bias flip the value of x Until a satisfying assignment is found Note: tune parameter p.

16 Random Walk algorithms  Random walk algorithm (e.g. Walksat) offer significant improvement on performance over hill-climbing algorithms. IDVarsclausesGSAT+wWalksat Ssa Ssa Ssa Ssa

17 First, bringing out the worst in random walks… (2-SAT) X1  X2 X2  X3 X3  X4 Xn  X4 Xn  X1 Note: Only 2 satisfying assignments, all False and all True.

18 Binary Chains Walk is exactly balanced.

19 Results: Speeding up Random Walks on Binary Chains * : empirical results ** : proof available Pure binary chain Chain with redundancies RW (n 2 ) ** RWF (n 2 ) ** (n 1.2 ) * WalkSat (n 2 ) ** (n 1.1 ) * Becomes almost like unit prop.

20 Ternary Chains In general, even a small bias in the wrong direction leads to exponential time to reach 0. reflecting

21 What about 3-SAT? Again, consider “chain” formulas. X1 & X2  X3 X2 & X3  X4 X_(n-2) & X_(n-1)  X_n X1 & X2 X_floor(n/2) & X_(n-1)  X_n

22 Theoretical Results on 3-SAT Chains Function low(i)Expected run time of pure RW i-2 (highly local) ~ Fib(n) (i.e., exp.) i/2 (interm. reach) ~ n log n (i.e., quasi-poly) log i (interm. reach) ~ n 2. (log n) 2 (i.e., poly) 1 (full back reach) ~ n 2 low(i) captures how far back the clauses reach.

23 Recurrence Relations Our formula structure gives us: E(f(z i )) = (E(f(z low(i) ) + E(f(z i ) + 1) * 1/3 + (E(f(z i-1 ) + E(f(z i ) + 1) * 1/3 + 1 * 1/3  E(f(z i )) = E(f(z low(i) ) + E(f(z i-1 ) + 3

24 Decompose: multiple “ 0 ” s   Start Sat assign.

25 Recap Chain Formula Results  Adding implied constraints capturing long-range dependencies speeds random walk on 2-Chain to near linear time.  Certain long-range dependencies in 3-SAT lead to poly-time convergence of random walks.  Can we take advantage of these results on practical problem instances? Yes! (next) Our methodology: Identify, analyze, and “ exploit ” special tractable structure in large practical reasoning problems.

26 Empirical Results SSS-SAT-1.0 instances (Velev 1999). 100 total.  level of redundancy added (20% near optimal) Formulas (redun. level) <40 sec<400 sec <4000 sec  =  =  =

27 Optimal Redundancy Rate Time vs Redundancy Rate Flips vs Redundancy Rate WalkSat(noise=50) on dlx2_cc_bug01.cnf from SAT-1.0 Suite

28 Probabilistic Reasoning  Previously, we asked “ does statement s hold in world A (represented by a set of clauses)? ” A s  ( ¬ s) ^ A unsatisfiable  Now, what is the probability that statement s holds in world A? Pr = #( s ^ A ) / #A

29 Probabilistic Reasoning  Close connection between counting and sampling (Jerrum et al, 1986).  Bayesian Net Queries can be encoded as #SAT. (Kautz, 2004)  How to Sample: By repeated counting using DPLL algorithms Monte Carlo Markov Chain Method Use state-of-the-art local search methods Note: Random Walk not uniform

30 Characteristics of Solution space: Solution Clustering  Visualization with multi-dimensional scaling (MDS) Solutions to specific 75 variable, 325 clause 3-SAT instances 75 dimensional solution projected to two dimensions Distance between points approximates hamming distance

31  Empirically determined each solution’s probability (uf variable, 325 clause 3-SAT instance)  WalkSat finds every solution, but with very large range of probabilities (1:10 4 )  Probability Clusters Solution Probability Using WalkSat Algorithm

32 Probability Ranges in Different Domains InstanceRunsHits Rarest Hits Common Common- to -Rare Ratio Random50* * *10 4 Logistics1* *

33 Improving the Uniformity of Sampling - mixing sampling strategy  To reduce the range of probabilities, we propose a hybrid local search algorithm: With probability p, the algorithm makes a biased random walk move With probability 1-p, the algorithm makes a SA (simulated annealing) move  In our experiments, we used 50% WalkSat + 50% SA at a fixed temperature

34 Results of the Hybrid Approach Our key figure.

35 Solution Clusters Results on a random 3-SAT instance (70 vars, 301 clauses, 2531 solutions).

36 Summary 1)WalkSAT does sample all solutions. 2)But, sampling can be highly biased. 3)Using a new hybrid strategy, we can obtain effective near-uniform sampling. Lesson: Hybrid of SA and biased walk, is a promising alternative to MCMC methods for sampling. Idea: Use SAT solvers to sample solutions from a combinatorial space. Findings:

37 Research Directions  Can we exploit techniques from DPLL solvers, such as learned clauses (inferred structure) in local search?  Can we identify other classes of structure in real-world problem that can further accelerate random walk style SAT solvers?  Can we design better SAT encodings based on our insights about structure?

38 Research Directions, cont.  Solution sampling Main Challenge: moving between clusters evaluation tools.  Randomize DPLL to sample solution space Early indicator of solution counts.  Compare with Bayesian/probabilistic state-of-the-art inference methods. Can we outperform them?