1. Introduction of Markov Chain Monte Carlo Jeongkyun Lee

2.  Usage  Why MCMC is called MCMC  MCMC methods  Appendix  Reference Contents 2

3.  Goal : 1) Estimate an unknown target distribution (or posterior) for a complex function, or 2) draw samples from the distribution. 1. Simulation  Draw samples from a probability governed by a system. 2. Integration / computing  Integrate or compute a high dimensional function 3. Optimization / Bayesian inference  Ex. Simulated annealing, MCMC-based particle filter 4. Learning  MLE learning, unsupervised learning Usage 3

4. Why MCMC is called MCMC 4

5. 5

6. 3. Markov Chain Monte Carlo  Construct a Markov Chain representing a target distribution.  http://www.kev-smith.com/tutorial/flash/markov_chain.swf http://www.kev-smith.com/tutorial/flash/markov_chain.swf Why MCMC is called MCMC 6 …

7. MCMC Methods 7 MetropolisMetropolis-Hastings

8. MCMC Methods 8

9. 9

10. 3. Reversible Jump(or trans-dimensional) MCMC  When the dimension of the state is changed,  Additionally consider a move type. MCMC Methods 10

11. 1. Markov Chain property  Stationary distribution (or detailed balance) Irreducible (all pi > 0) Aperiodic Appendix 11

12. 2. MH sampling as a Markov Chain  The transition probability kernel in the MH algorithm Thus, if the MH kernel satisfies then the stationary distribution from this kernel corresponds to draws from the target distribution. Appendix 12

13. 2. MH sampling as a Markov Chain Appendix 13

14.  http://vcla.stat.ucla.edu/old/MCMC/MCMC_tutorial.htm http://vcla.stat.ucla.edu/old/MCMC/MCMC_tutorial.htm  http://www.kev-smith.com/tutorial/rjmcmc.php http://www.kev-smith.com/tutorial/rjmcmc.php  http://www.cs.bris.ac.uk/~damen/MCMCTutorial.htm http://www.cs.bris.ac.uk/~damen/MCMCTutorial.htm  B. Walsh, “Markov Chain Monte Carlo and Gibbs Sampling”, Lecture Notes, MIT, 2004 Reference 14

15. 15 Thank you!