1. Approximate Inference by Sampling
Readings:
K&F: 10.1, 10.2, 10.3
Approximate Inference by Sampling
Graphical Models – 10708
Ajit Singh
Carnegie Mellon University
November 3rd, 2006

2. Approximate inference overview
There are many many many many approximate inference algorithms for PGMs
We will focus on three representative ones:
sampling - today
variational inference - continues next class
loopy belief propagation and generalized belief propagation
– Ajit Singh 2006

3. Goal
Often we want expectations given samples x[1] … x[m] from a distribution P.
– Ajit Singh 2006

4. Forward Sampling Sample nodes in topological order
Generating samples from a joint distribution
Generate topological ordering
Sampling according to the ordering
Sampling from a multinomial (next slide)
– Ajit Singh 2006

5. Forward Sampling P(Y = y) = #(Y = y) / N
P(Y = y | E = e) = #(Y = y, E = e) / #(E = e)
Rejection sampling: throw away samples that do not match the evidence.
Sample efficiency
How often do we expect to see a record with E = e ?
– Ajit Singh 2006

6. Idea What is we just fix the value of evidence nodes ?
What is expected number
of records with (Intelligence = Low) ?
– Ajit Singh 2006

7. Likelihood Weighting In forward sampling, we get D = Easy I = Low
What is xi[m]
What is w[m]
Write the equation for likelihood wrighting
P(y|e) = Sum_m w[m] * Ind(y[m] = y) / Sum_m w[m]
A weighted average of the samples. W[m] correct for the fact
Replaces sampling (in forward sampling) with exact computation --> Lower variance
What if P(E = e) is very low ?
w[m] is small
FINISH - no justification by itself
A special case of importance sampling
– Ajit Singh 2006

8. Importance Sampling What if you cannot easily sample ?
Posterior distribution on a Bayesian network
P(Y = y | E = e) where the evidence itself is a rare event.
Sampling from a Markov network with cycles is always hard
See homework 4
Pick some distribution Q(X) that is easier to sample from.
Assume that if P(x) > 0 then Q(x) > 0
Hopefully D(P||Q) is small
– Ajit Singh 2006

9. Importance Sampling Unnormalized Importance Sampling
Unbiased estimator
How to choose Q when P is a Bayesian network ?
What happens if the proposal distribution is bad ?
– Ajit Singh 2006

10. Mutilated BN Proposal
Generating a proposal distribution for a Bayesian network
Evidenced nodes have no parents.
Each evidence node Zi = zi has distribution P(Zi = zi) = 1
Equivalent to likelihood weighting
This is equivalent to likelihood weighting
– Ajit Singh 2006

11. Forward Sampling Approaches
Forward sampling, rejection sampling, and likelihood weighting are all forward samplers
Requires a topological ordering. This limits us to
Bayesian networks
Tree Markov networks
Unnormalized importance sampling can be done on cyclic Markov networks, but it is expensive
See homework 4
Limitation
Fixing an evidence node only allows it to directly affect its descendents.
– Ajit Singh 2006

12. Scratch space
– Ajit Singh 2006

13. Markov Blanket Approaches
Forward Samplers: Compute weight of Xi given assignment to ancestors in topological ordering
Markov Blanket Samplers: Compute weight of Xi given assignment to its Markov Blanket.
– Ajit Singh 2006

14. Markov Blanket Samplers
Works on any type of graphical model covered in the course thus far.
Draw a spin glass / Potts model
Remind them what the Markov blanket on a grid is
The weight in a MB Sampler is P(Xi | MB(Xi))
– Ajit Singh 2006

15. Gibbs Sampling Let X be the non-evidence variables
Generate an initial assignment (0)
For t = 1..T
(t) = (t-1)
For each Xi in X
ui = Value of variables X - {Xi} in sample (t)
Compute P(Xi | ui)
Sample xi(t) from P(Xi | ui)
Set the value of Xi = xi(t) in (t)
Samples are taken from (0) … (T)
Key is P(Xi | ui)
Talk about burn in later
– Ajit Singh 2006

16. Computing P(Xi | ui)
The major task in designing a Gibbs sampler is deriving P(Xi | ui)
Use conditional independence
Xi  Xj | MB(Xi) for all Xj in X - MB(Xi) - {Xi}
P(X|Y = y) =
P(Y|X = x) =
– Ajit Singh 2006

17. Pairwise Markov Random Field
– Ajit Singh 2006

18. Markov Chain Interpretation
The state space consists of assignments to X.
P(xi | ui) are the transition probability (neighboring states differ only in one variable)
Given the transition matrix you could compute the exact stationary distribution
Typically impossible to store the transition matrix.
Gibbs does not need to store the transition matrix !
– Ajit Singh 2006

19. Scratch space
– Ajit Singh 2006

20. Convergence
Not all samples (0) … (T) are independent. Consider one marginal P(xi|ui).
Burn-in
Thinning
– Ajit Singh 2006

21. MAP by Sampling Generate a few samples from the posterior
For each Xi the MAP is the majority assignment
– Ajit Singh 2006

22. What you need to know Forward sampling approaches
Forward Sampling / Rejection Sampling
Generate samples from P(X) or P(X|e)
Likelihood Weighting / Importance Sampling
Sampling where the evidence is rare
Fixing variables lowers variance of samples when compared to rejection sampling.
Useful on Bayesian networks & tree Markov networks
Markov blanket approaches
Gibbs Sampling
Works on any graphical model where we can sample from P(Xi | rest).
Markov chain interpretation.
Samples are independent when the Markov chain converges.
Convergence heuristics, burn-in, thinning.
– Ajit Singh 2006