1. SampleSearch: A scheme that searches for Consistent Samples Vibhav Gogate and Rina Dechter University of California, Irvine USA

2. Outline  Background Bayesian Networks with Zero probabilities Importance Sampling Rejection Problem  The SampleSearch Scheme Algorithm Sampling Distribution and its Approximation  Experimental Results

3. Bayesian Networks: Representation (Pearl, 1988) lung Cancer Smoking X-ray Bronchitis Dyspnoea P(D|C,B) P(B|S) P(S) P(X|C,S) P(C|S) P(S, C, B, X, D) = P(S) P(C|S) P(B|S) P(X|C,S) P(D|C,B) (A) Probability of Evidence P(smoking=no, dyspnoea=yes)=? (B) Belief Updating: P (lung cancer=yes | smoking=no, dyspnoea=yes ) = ?

4. Complexity  Belief Updating NP-hard when zeros are present General case when all CPTs are positive, not known. Relative Approximation  Randomized Polynomial time algorithm when all CPTs are positive (Dagum and Luby 1997)  Probability of Evidence NP-hard when zeros are present Relative Approximation  Randomized Polynomial time algorithm when all CPTs are positive and (1/P(e)) is polynomial (Karp, Dagum and Luby 1993)

5. Importance Sampling (Rubinstein ’81)

6. Importance Sampling for Belief Updating

7. Generating i.i.d. samples from Q Q(A,B,C)=Q(A)*Q(B|A)*Q(C|A,B) Q(A)=(0.8,0.2) Q(B|A)=(0.4,0.6,0.2,0.8) Q(C|A,B)=Q(C)=(0.2,0.8) A=0 B=0B=1B=0B=1 A=1 C=1 C=0 C=1 Root 0.80.2 0.40.6 0.2 0.8 0.20.80.20.80.2 0.8 0.2 0.8 C=0

8. Rejection Problem  Importance Sampling requirement f(x i )>0 => Q(x i )>0 Conversely, Q(x i ) can be >0 even if f(x i )=0.  So if the probability of sampling ∑Q(x i |f(x i )>0) is very small A large number of assignments will have zero weight  Extreme case: Our approximation = zero.

9. Rejection Problem All Blue leaves correspond to solutions i.e. f(x) >0 All Red leaves correspond to non-solutions i.e. f(x)=0 A=0 B=0 C=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root 0.80.2 0.40.6 0.2 0.8 0.20.80.20.80.2 0.8 0.2 0.8

10. Example: map coloring Variables - countries (A,B,C,etc.) Values - colors (red, green, blue) Constraints: A Solution is an assignment that satisfies all constraints C A B D E F G Constraint Networks (Dechter 2003)

11. Constraint networks to model “zeros” A F C D B G ACP(C|A) 000 011 101 110 Constraints A=0, C=0 not allowed A=1, C=1 not allowed Or A≠C Why constraints? For a partial sample if a constraint is violated f(X=x)=0 for any full extension X=x of the sample. For every full assignment X=x solution implies f(X=x) >0 and non-solution f(X=x)=0

12. Using Constraints A=0 B=0 C=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root 0.80.2 0.40.6 0.2 0.8 0.20.80.20.80.2 0.8 0.2 0.8 Constraints A≠B, A≠C

13. Using Constraints A=0 B=0B=1 C=1 C=0 Root 0.8 0.40.6 0.2 0.8 0.20.8 C=0 Constraints A≠B, A≠C Constraint A≠B violated

14. Outline  Background Bayesian Networks Bayesian Networks Importance Sampling Importance Sampling Rejection Prblem Rejection Prblem  The SampleSearch Scheme Algorithm Sampling Distribution and Approximation  Experimental Results

15. Algorithm SampleSearch A=0 B=0 C=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root 0.80.2 0.40.6 0.2 0.8 0.20.80.20.80.2 0.8 0.2 0.8 Constraints A≠B, A≠C

16. Algorithm SampleSearch A=0 B=0 C=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root 0.80.2 0.40.6 0.2 0.8 0.20.80.20.80.2 0.8 0.2 0.8 Constraints A≠B, A≠C 1

17. Algorithm SampleSearch A=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root 0.80.2 0.80.20.80.2 0.8 0.2 0.8 Constraints A≠B, A≠C Resume Sampling 1

18. Algorithm SampleSearch A=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root 0.80.2 1 0.80.20.80.2 0.8 0.2 0.8 Constraints A≠B, A≠C Constraint ViolatedUntil Solution i.e. f(x)>0 found 1

19. Generate more Samples A=0 B=0 C=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root 0.80.2 0.40.6 0.2 0.8 0.20.80.20.80.2 0.8 0.2 0.8 Constraints A≠B, A≠C

20. Generate more Samples A=0 B=0 C=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root 0.80.2 0.40.6 0.2 0.8 0.20.80.20.80.2 0.8 0.2 0.8 Constraints A≠B, A≠C 1

21. Traces of SampleSearch A=0 B=1 C=1 Root A=0 B=0 B=1 C=1 Root A=0 B=1 C=1 Root C=0 A=0 B=0 B=1 C=1 Root C=0 Constraints A≠B, A≠C

22. The Sampling distribution Q R of SampleSearch A=0 B=0B=1 C=1 C=0 Root 0.8 01 0 0 01 C=0 What is probability of generating A=0? Q R (A=0)=0.8 Why? SampleSearch is systematic What is probability of generating B=1? Q R (B=1|A=0)=1 Why? SampleSearch is systematic What is probability of generating B=0? Simple: Q R (B=0|A=0)=0 All samples generated by SampleSearch are solutions  Did you generate samples from Q? -NO! Backtrack-free distribution

23. Computing Q R  Invoke an oracle or a complete search procedure O(n) times per sample A=0 B=1 C=1 Root ?? Solution

24. Approximation A R of Q R A=0 B=0B=1 C=1 C=0 Root 0.8 01 0 0 01 C=0 Hole Don’t know No solutions here IF Hole THEN A R =Q IF No solutions on the other branch THEN A R =1

25. Approximation A R of Q R A=0 B=1 C=1 Root A=0 B=0 B=1 C=1 Root C=0  Problem: Can’t guarantee convergence ? ? 0.8 0.61 0.8 A=0 B=0 B=1 C=1 Root 0.8 ? 1 A=0 B=1 C=1 Root C=0 0.8 ? 0.6 1 1

26. Guarantee convergence in the limit  Store all possible traces A=0 B=1 C=1 Root C=0 0.8 ? 0.6 1 A=0 B=0 B=1 C=1 Root 0.8 ? 1 Approximation A R N IF Hole THEN A R N =Q IF No solutions on other branch THEN A R N =1 A=0 B=1 C=1 Root 0.8 1 1 ?

27. Improving Naive SampleSeach  Handle Non-binary domains See the paper, Proof is complicated.  Better Search Strategy Can use any state-of-the-art CSP/SAT solver e.g. minisat (Sorrenson et al 2006)  All theorems and result hold  Better Importance Function Use output of generalized belief propagation to compute the initial importance function Q (Gogate and Dechter 2005)

28. Experimental Results  Previous Algorithms Likelihood weighting (LW)  Proposal=Prior IJGP-sampling (IJGP-S) (Gogate and Dechter 2005)  Proposal=Output of generalized belief propagation  Adding SampleSearch SampleSearch with LW (S+LW) SampleSearch with IJGP-sampling (S+IJGP-S)

31. Linkage BN_69

32. Linkage BN_73

33. Linkage BN_76

35. Conclusions Belief networks with zero probabilities lead to the Rejection problem in importance Sampling. We presented a SampleSearch scheme that works with any importance sampling scheme to circumvent the Rejection Problem. Sampling Distribution of SampleSearch is the backtrack- free distribution Q R –Expensive to compute –Approximation of Q R based on storing all traces that yields an asymptotically unbiased estimator Empirically, when a substantial number of zero probabilities are present, SampleSearch based schemes dominate their pure sampling counter-parts.