SampleSearch: A scheme that searches for Consistent Samples Vibhav Gogate and Rina Dechter University of California, Irvine USA

Outline  Background Bayesian Networks with Zero probabilities Importance Sampling Rejection Problem  The SampleSearch Scheme Algorithm Sampling Distribution and its Approximation  Experimental Results

Bayesian Networks: Representation (Pearl, 1988) lung Cancer Smoking X-ray Bronchitis Dyspnoea P(D|C,B) P(B|S) P(S) P(X|C,S) P(C|S) P(S, C, B, X, D) = P(S) P(C|S) P(B|S) P(X|C,S) P(D|C,B) (A) Probability of Evidence P(smoking=no, dyspnoea=yes)=? (B) Belief Updating: P (lung cancer=yes | smoking=no, dyspnoea=yes ) = ?

Complexity  Belief Updating NP-hard when zeros are present General case when all CPTs are positive, not known. Relative Approximation  Randomized Polynomial time algorithm when all CPTs are positive (Dagum and Luby 1997)  Probability of Evidence NP-hard when zeros are present Relative Approximation  Randomized Polynomial time algorithm when all CPTs are positive and (1/P(e)) is polynomial (Karp, Dagum and Luby 1993)

Importance Sampling (Rubinstein ’81)

Importance Sampling for Belief Updating

Generating i.i.d. samples from Q Q(A,B,C)=Q(A)*Q(B|A)*Q(C|A,B) Q(A)=(0.8,0.2) Q(B|A)=(0.4,0.6,0.2,0.8) Q(C|A,B)=Q(C)=(0.2,0.8) A=0 B=0B=1B=0B=1 A=1 C=1 C=0 C=1 Root C=0

Rejection Problem  Importance Sampling requirement f(x i )>0 => Q(x i )>0 Conversely, Q(x i ) can be >0 even if f(x i )=0.  So if the probability of sampling ∑Q(x i |f(x i )>0) is very small A large number of assignments will have zero weight  Extreme case: Our approximation = zero.

Rejection Problem All Blue leaves correspond to solutions i.e. f(x) >0 All Red leaves correspond to non-solutions i.e. f(x)=0 A=0 B=0 C=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root

Example: map coloring Variables - countries (A,B,C,etc.) Values - colors (red, green, blue) Constraints: A Solution is an assignment that satisfies all constraints C A B D E F G Constraint Networks (Dechter 2003)

Constraint networks to model “zeros” A F C D B G ACP(C|A) Constraints A=0, C=0 not allowed A=1, C=1 not allowed Or A≠C Why constraints? For a partial sample if a constraint is violated f(X=x)=0 for any full extension X=x of the sample. For every full assignment X=x solution implies f(X=x) >0 and non-solution f(X=x)=0

Using Constraints A=0 B=0 C=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root Constraints A≠B, A≠C

Using Constraints A=0 B=0B=1 C=1 C=0 Root C=0 Constraints A≠B, A≠C Constraint A≠B violated

Outline  Background Bayesian Networks Bayesian Networks Importance Sampling Importance Sampling Rejection Prblem Rejection Prblem  The SampleSearch Scheme Algorithm Sampling Distribution and Approximation  Experimental Results

Algorithm SampleSearch A=0 B=0 C=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root Constraints A≠B, A≠C

Algorithm SampleSearch A=0 B=0 C=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root Constraints A≠B, A≠C 1

Algorithm SampleSearch A=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root Constraints A≠B, A≠C Resume Sampling 1

Algorithm SampleSearch A=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root Constraints A≠B, A≠C Constraint ViolatedUntil Solution i.e. f(x)>0 found 1

Generate more Samples A=0 B=0 C=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root Constraints A≠B, A≠C

Generate more Samples A=0 B=0 C=0 B=1B=0B=1 A=1 C=1 C=0 C=1 Root Constraints A≠B, A≠C 1

Traces of SampleSearch A=0 B=1 C=1 Root A=0 B=0 B=1 C=1 Root A=0 B=1 C=1 Root C=0 A=0 B=0 B=1 C=1 Root C=0 Constraints A≠B, A≠C

The Sampling distribution Q R of SampleSearch A=0 B=0B=1 C=1 C=0 Root C=0 What is probability of generating A=0? Q R (A=0)=0.8 Why? SampleSearch is systematic What is probability of generating B=1? Q R (B=1|A=0)=1 Why? SampleSearch is systematic What is probability of generating B=0? Simple: Q R (B=0|A=0)=0 All samples generated by SampleSearch are solutions  Did you generate samples from Q? -NO! Backtrack-free distribution

Computing Q R  Invoke an oracle or a complete search procedure O(n) times per sample A=0 B=1 C=1 Root ?? Solution

Approximation A R of Q R A=0 B=0B=1 C=1 C=0 Root C=0 Hole Don’t know No solutions here IF Hole THEN A R =Q IF No solutions on the other branch THEN A R =1

Approximation A R of Q R A=0 B=1 C=1 Root A=0 B=0 B=1 C=1 Root C=0  Problem: Can’t guarantee convergence ? ? A=0 B=0 B=1 C=1 Root 0.8 ? 1 A=0 B=1 C=1 Root C=0 0.8 ?

Guarantee convergence in the limit  Store all possible traces A=0 B=1 C=1 Root C=0 0.8 ? A=0 B=0 B=1 C=1 Root 0.8 ? 1 Approximation A R N IF Hole THEN A R N =Q IF No solutions on other branch THEN A R N =1 A=0 B=1 C=1 Root ?

Improving Naive SampleSeach  Handle Non-binary domains See the paper, Proof is complicated.  Better Search Strategy Can use any state-of-the-art CSP/SAT solver e.g. minisat (Sorrenson et al 2006)  All theorems and result hold  Better Importance Function Use output of generalized belief propagation to compute the initial importance function Q (Gogate and Dechter 2005)

Experimental Results  Previous Algorithms Likelihood weighting (LW)  Proposal=Prior IJGP-sampling (IJGP-S) (Gogate and Dechter 2005)  Proposal=Output of generalized belief propagation  Adding SampleSearch SampleSearch with LW (S+LW) SampleSearch with IJGP-sampling (S+IJGP-S)

Linkage BN_69

Linkage BN_73

Linkage BN_76

Conclusions Belief networks with zero probabilities lead to the Rejection problem in importance Sampling. We presented a SampleSearch scheme that works with any importance sampling scheme to circumvent the Rejection Problem. Sampling Distribution of SampleSearch is the backtrack- free distribution Q R –Expensive to compute –Approximation of Q R based on storing all traces that yields an asymptotically unbiased estimator Empirically, when a substantial number of zero probabilities are present, SampleSearch based schemes dominate their pure sampling counter-parts.