Introduction of Markov Chain Monte Carlo Jeongkyun Lee

 Usage  Why MCMC is called MCMC  MCMC methods  Appendix  Reference Contents 2

 Goal : 1) Estimate an unknown target distribution (or posterior) for a complex function, or 2) draw samples from the distribution. 1. Simulation  Draw samples from a probability governed by a system. 2. Integration / computing  Integrate or compute a high dimensional function 3. Optimization / Bayesian inference  Ex. Simulated annealing, MCMC-based particle filter 4. Learning  MLE learning, unsupervised learning Usage 3

Why MCMC is called MCMC 4

5

3. Markov Chain Monte Carlo  Construct a Markov Chain representing a target distribution.  Why MCMC is called MCMC 6 …

MCMC Methods 7 MetropolisMetropolis-Hastings

MCMC Methods 8

9

3. Reversible Jump(or trans-dimensional) MCMC  When the dimension of the state is changed,  Additionally consider a move type. MCMC Methods 10

1. Markov Chain property  Stationary distribution (or detailed balance) Irreducible (all pi > 0) Aperiodic Appendix 11

2. MH sampling as a Markov Chain  The transition probability kernel in the MH algorithm Thus, if the MH kernel satisfies then the stationary distribution from this kernel corresponds to draws from the target distribution. Appendix 12

2. MH sampling as a Markov Chain Appendix 13

    B. Walsh, “Markov Chain Monte Carlo and Gibbs Sampling”, Lecture Notes, MIT, 2004 Reference 14

15 Thank you!