1. Lecture #9: Introduction to Markov Chain Monte Carlo, part 3
This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.
CS 679: Text Mining
Lecture #9: Introduction to Markov Chain Monte Carlo, part 3
Slide Credit: Teg Grenager of Stanford University, with adaptations and improvements by Eric Ringger.

2. Announcements Assignment #4 Assignment #5
Prepare and give lecture on your 2 favorite papers
Starting in approx. 1.5 weeks
Assignment #5
Pre-proposal
Answer the 5 Questions!

3. Where are we?
Joint distributions are useful for answering questions (joint or conditional)
Directed graphical models represent joint distributions
Hierarchical bayesian models: make parameters explicit
We want to ask conditional questions on our models
E.g., posterior distributions over latent variables given large collections of data
Simultaneously inferring values of model parameters and answering the conditional questions: “posterior inference”
Posterior inference is a challenging computational problem
Sampling is an efficient mechanism for performing that inference.
Variety of MCMC methods: random walk on a carefully constructed markov chain
Convergence to desirable stationary distribution

4. Agenda Motivation The Monte Carlo Principle Markov Chain Monte Carlo
Metropolis Hastings
Gibbs Sampling
Advanced Topics

5. Metropolis-Hastings
The symmetry requirement of the Metropolis proposal distribution can be hard to satisfy
Metropolis-Hastings is the natural generalization of the Metropolis algorithm, and the most popular MCMC algorithm
Choose a proposal distribution which is not necessarily symmetric
Define a Markov chain with the following process:
Sample a candidate point x* from a proposal distribution q(x*|x(t))
Compute the importance ratio:
With probability min(r,1) transition to x*, otherwise stay in the same state x(t)

6. MH convergence
Theorem: The Metropolis-Hastings algorithm converges to the target distribution p(x).
Proof:
For all 𝑥,𝑦𝑿, WLOG assume 𝑝(𝑥)𝑞(𝑦|𝑥)  𝑝(𝑦)𝑞(𝑥|𝑦)
Thus, it satisfies detailed balance
candidate is always accepted
(i.e., 𝑇(𝑥,𝑦)=𝑝(𝑥)𝑞(𝑦|𝑥))
b/c 𝑝(𝑥)𝑞(𝑦|𝑥)  𝑝(𝑦)𝑞(𝑥|𝑦)
multiply by 1
commute
transition prob.
𝑇(𝑦,𝑥)=𝑞(𝑥|𝑦)𝑝(𝑥)𝑞(𝑦|𝑥)
/(𝑝(𝑦)𝑞(𝑥|𝑦))

7. Gibbs sampling
A special case of Metropolis-Hastings which is applicable to state spaces in which
we have a factored state space
And
access to the full (“complete”) conditionals:
Perfect for Bayesian networks!
Idea: To transition from one state (variable assignment) to another,
Pick a variable,
Sample its value from the conditional distribution
That’s it!
We’ll show in a minute why this is an instance of MH and thus must be sampling from joint or conditional distribution we wanted.

8. Markov blanket
Recall that Bayesian networks encode a factored representation of the joint distribution
Variables are independent of their non-descendents given their parents
Variables are independent of everything else in the network given their Markov blanket!
𝑃 𝐵 𝑣𝑎𝑙𝑢𝑒𝑠 𝑜𝑓 𝑅, 𝑣𝑎𝑙𝑢𝑒𝑠 𝑜𝑓 𝐺
=𝑃 𝐵 𝑣𝑎𝑙𝑢𝑒𝑠 𝑜𝑓 𝐺)
So, to sample each node, we only need to condition on its Markov blanket:
𝑃 𝑥 𝑗 𝑥 =𝑃 𝑥 𝑗 𝑥 −𝑗 =𝑃 𝑥 𝑗 𝑀𝐵 𝑥 𝑗

9. Gibbs sampling More formally, the proposal distribution is
The importance ratio is
So we always accept!
if 𝑥 −𝑗 ∗ = 𝑥 −𝑗 𝑡
otherwise
Dfn of proposal distribution and 𝑥 −𝑗 ∗ = 𝑥 −𝑗 𝑡
Dfn of conditional probability (twice)
Definition of 𝑥 ∗ 𝑎𝑛𝑑 𝑥 𝑡
Cancel common terms

10. Gibbs sampling example
Consider a simple, 2 variable Bayes net
Initialize randomly
Sample variables alternately
A=1
A=0
0.5 F 1 T T F 1 T F 1 A B F T b b a a
B=1
B=0
A=1
0.8
0.2
A=0 T 1 T

11. Gibbs Sampling (in 2-D)
(MacKay, 2002)

12. Practical issues How many iterations? How to know when to stop?
M-H: What’s a good proposal function?
Gibbs: How to derive the complete conditionals?

13. Advanced Topics
Simulated annealing, for global optimization, is a form of MCMC
Mixtures of MCMC transition functions
Monte Carlo EM
stochastic E-step
i.e., sample instead of computing full posterior
Reversible jump MCMC for model selection
Adaptive proposal distributions

14. Next
Document Clustering by Gibbs Sampling on a Mixture of Multinomials
From Dan Walker’s dissertation!