1. Daphne Koller Sampling Methods Metropolis- Hastings Algorithm Probabilistic Graphical Models Inference

2. Daphne Koller Reversible Chains Theorem: If detailed balance holds, and T is regular, then T has a unique stationary distribution  Proof:

3. Daphne Koller Metropolis Hastings Chain At each state x, sample x’ from Q(x  x’) Accept proposal with probability A(x  x’) – If proposal accepted, move to x’ – Otherwise stay at x Proposal distribution Q(x  x’) Acceptance probability: A(x  x’) T(x  x’) = Q(x  x’) A(x  x’) Q(x  x) +  x’  x Q(x  x’) (1-A(x  x’)) if x’  x T(x  x) =

4. Daphne Koller Acceptance Probability

5. Template vertLeftWhite1 Consider a Markov chain over a state space s 1,…,s m and assume we wish to sample from the stationary distribution  ( s i )  q -i for q i, which acceptance probability A(s i → s j ) will give rise to a legal MH chain with this stationary distribution? None of the above – this is not a legal proposal distribution. q j-i q i-j p 2i q j-i

6. Template vertLeftWhite1 In the same setup as the previous problem, let Q(s i,s j )  p^(-d(s i,s j )) but only if |i-j| 1. For j>i, which acceptance probability A(s i → s j ) will give rise to a legal MH chain with this stationary distribution? q j-i None of the above – this is not a legal proposal distribution. q i-j p 2i q j-i

7. Template vertLeftWhite1 Now let d(s i,s j ) be the clockwise distance between s i and s j (so that d(s i,s i+1 )=1 and d(s i +1,s i )=n-1). Let Q(s i,s j )  p^(-d(s i,s j )). For j>i, which acceptance probability A(s i → s j ) will give rise to an MH chain with this stationary distribution? None of the above – this is not a legal proposal distribution. q j-i q i-j p n-2i q j-i

8. Daphne Koller Choice of Q Q must be reversible: – Q(x  x’) > 0  Q(x’  x) > 0 Opposing forces – Q should try to spread out, to improve mixing – But then acceptance probability often low

9. Daphne Koller MCMC for Matching X i =j if i matched to j if every X i has different value otherwise

10. Daphne Koller MH for Matching: Augmenting Path 1) randomly pick one variable X i 2) sample X i, pretending that all values are available 3) pick the variable whose assignment was taken (conflict), and return to step 2 When step 2 creates no conflict, modify assignment to flip augmenting path

11. Daphne Koller Example Results MH proposal 1 MH proposal 2 Gibbs

12. Daphne Koller Summary MH is a general framework for building Markov chains with a particular stationary distribution – Requires a proposal distribution – Acceptance computed via detailed balance Tremendous flexibility in designing proposal distributions that explore the space quickly – But proposal distribution makes a big difference – and finding a good one is not always easy

13. Daphne Koller END END END