Importance Sampling ICS 276 Fall 2007 Rina Dechter

Outline Gibbs Sampling Advances in Gibbs sampling Blocking Cutset sampling (Rao-Blackwellisation) Importance Sampling Advances in Importance Sampling Particle Filtering

Importance Sampling Theory

Given a distribution called the proposal distribution Q (such that P(Z=z,e)>0=> Q(Z=z)>0) w(Z=z) is called as importance weight

Importance Sampling Theory Underlying principle, Approximate Average over a set of numbers by an average over a set of sampled numbers

Importance Sampling (Informally) Express the problem as computing the average over a set of real numbers Sample a subset of real numbers Approximate the true average by sample average. True Average: Average of (0.11, 0.24, 0.55, 0.77, 0.88,0.99)=0.59 Sample Average over 2 samples: Average of (0.24, 0.77) = 0.505

How to generate samples from Q Express Q in product form: Q(Z)=Q(Z 1 )Q(Z 2 |Z 1 )….Q(Z n |Z 1,..Z n-1 ) Sample along the order Z 1,..Z n Example: Q(Z 1 )=(0.2,0.8) Q(Z 2 |Z 1 )=(0.2,0.8,0.1,0.9) Q(Z 3 |Z 1,Z 2 )=Q(Z 3 |Z 1 )=(0.5,0.5,0.3,0.7)

How to sample from Q Generate a random number between 0 and 1 Q(Z 1 )=(0.2,0.8) Q(Z 2 |Z 1 )=(0.2,0.8,0.1,0.9) Q(Z 3 |Z 1,Z 2 )=Q(Z 3 |Z 1 )=(0.5,0.5,0.3,0.7 ) Which value to select for Z 1 ? Domains of each variable is {0,1} 0 1

How to sample from Q? Each Sample Z=z Sample Z 1 =z 1 from Q(Z 1 ) Sample Z 2 =z 2 from Q(Z 2 |Z 1 =z1) Sample Z 3 =z 3 from Q(Z 3 |Z1=z1) Generate N such samples

Likelihood weighting Q= Prior Distribution=CPTs of the Bayesian network

Likelihood weighting example lung Cancer Smoking X-ray Bronchitis Dyspnoea P(D|C,B) P(B|S) P(S) P(X|C,S) P(C|S) P(S, C, B, X, D) = P(S) P(C|S) P(B|S) P(X|C,S) P(D|C,B)

Likelihood weighting example lung Cancer Smoking X-ray Bronchitis Dyspnoea P(D|C,B) P(B|S) P(S) P(X|C,S) P(C|S) Q=Prior Q(S,C,D)=Q(S)*Q(C|S)*Q(D|C,B=0) =P(S)P(C|S)P(D|C,B=0) Sample S=s from P(S) Sample C=c from P(C|S=s) Sample D=d from P(D|C=c,B=0)

The Algorithm

How to solve belief updating?

Difference between estimating P(E=e) and P(X i =x i |E=e) Unbiased Asymptotically Unbiased

Proposal Distribution: Which is better?

Outline Gibbs Sampling Advances in Gibbs sampling Blocking Cutset sampling (Rao-Blackwellisation) Importance Sampling Advances in Importance Sampling Particle Filtering

Research Issues in Importance Sampling Better Proposal Distribution Likelihood weighting Fung and Chang, 1990; Shachter and Peot, 1990 AIS-BN Cheng and Druzdzel, 2000 Iterative Belief Propagation Changhe and Druzdzel, 2003 Iterative Join Graph Propagation and variable ordering Gogate and Dechter, 2005

Research Issues in Importance Sampling (Cheng and Druzdzel 2000) Adaptive Importance Sampling

General case Given k proposal distributions Take N samples out of each distribution Approximate P(e)

Estimating Q'(z)

Cutset importance sampling Divide the Set of variables into two parts Cutset (C) and Remaining Variables (R) (Gogate and Dechter, 2005) and (Bidyuk and Dechter 2006)

Outline Gibbs Sampling Advances in Gibbs sampling Blocking Cutset sampling (Rao-Blackwellisation) Importance Sampling Advances in Importance Sampling Particle Filtering

Dynamic Belief Networks (DBNs) Bayesian Network at time t Bayesian Network at time t+1 Transition arcs XtXt X t+1 YtYt Y t+1 X0X0 X1X1 X2X2 Y0Y0 Y1Y1 Y2Y2 Unrolled DBN for t=0 to t=10 X 10 Y 10

Query Compute P(X 0:t |Y 0:t ) or P(X t |Y 0:t ) Example P(X 0:10 |Y 0:10 ) or P(X 10 |Y 0:10 ) Hard!!! over a long time period Approximate! Sample!

Particle Filtering (PF) = “condensation” = “sequential Monte Carlo” = “survival of the fittest” PF can treat any type of probability distribution, non-linearity, and non- stationarity; PF are powerful sampling based inference/learning algorithms for DBNs.

Particle Filtering On white board