Daphne Koller Sampling Methods Metropolis- Hastings Algorithm Probabilistic Graphical Models Inference.

Slides:

Advertisements

Similar presentations

02/12/ a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008.

Advertisements

Gibbs sampler - simple properties It’s not hard to show that this MC chain is aperiodic. Often is reversible distribution. If in addition the chain is.

Bayesian Estimation in MARK

Introduction of Markov Chain Monte Carlo Jeongkyun Lee.

Markov Chains Modified by Longin Jan Latecki

1 The Monte Carlo method. 2 (0,0) (1,1) (-1,-1) (-1,1) (1,-1) 1 Z= 1 If  X 2 +Y 2  1 0 o/w (X,Y) is a point chosen uniformly at random in a 2  2 square.

Markov Chains 1.

Markov Chain Monte Carlo Prof. David Page transcribed by Matthew G. Lee.

TCOM 501: Networking Theory & Fundamentals

Introduction to Sampling based inference and MCMC Ata Kaban School of Computer Science The University of Birmingham.

CHAPTER 16 MARKOV CHAIN MONTE CARLO

Lecture 3: Markov processes, master equation

6.896: Probability and Computation Spring 2011 Constantinos (Costis) Daskalakis lecture 2.

Suggested readings Historical notes Markov chains MCMC details

BAYESIAN INFERENCE Sampling techniques

CS774. Markov Random Field : Theory and Application Lecture 16 Kyomin Jung KAIST Nov

1 CE 530 Molecular Simulation Lecture 8 Markov Processes David A. Kofke Department of Chemical Engineering SUNY Buffalo

What if time ran backwards? If X n, 0 ≤ n ≤ N is a Markov chain, what about Y n = X N-n ? If X n follows the stationary distribution, Y n has stationary.

Computational statistics 2009 Random walk. Computational statistics 2009 Random walk with absorbing barrier.

The Gibbs sampler Suppose f is a function from S d to S. We generate a Markov chain by consecutively drawing from (called the full conditionals). The n’th.

The University of Texas at Austin, CS 395T, Spring 2008, Prof. William H. Press IMPRS Summer School 2009, Prof. William H. Press 1 4th IMPRS Astronomy.

Machine Learning CUNY Graduate Center Lecture 7b: Sampling.

Exploring subjective probability distributions using Bayesian statistics Tom Griffiths Department of Psychology Cognitive Science Program University of.

Particle filters (continued…). Recall Particle filters –Track state sequence x i given the measurements ( y 0, y 1, …., y i ) –Non-linear dynamics –Non-linear.

If time is continuous we cannot write down the simultaneous distribution of X(t) for all t. Rather, we pick n, t 1,...,t n and write down probabilities.

1 On the Computation of the Permanent Dana Moshkovitz.

The moment generating function of random variable X is given by Moment generating function.

Approximate Inference 2: Monte Carlo Markov Chain

Entropy Rate of a Markov Chain

Random Walks and Markov Chains Nimantha Thushan Baranasuriya Girisha Durrel De Silva Rahul Singhal Karthik Yadati Ziling Zhou.

Machine Learning Lecture 23: Statistical Estimation with Sampling Iain Murray’s MLSS lecture on videolectures.net:

Markov Decision Processes1 Definitions; Stationary policies; Value improvement algorithm, Policy improvement algorithm, and linear programming for discounted.

6.896: Probability and Computation Spring 2011 Constantinos (Costis) Daskalakis lecture 3.

1 Networks of queues Networks of queues reversibility, output theorem, tandem networks, partial balance, product-form distribution, blocking, insensitivity,

Suppressing Random Walks in Markov Chain Monte Carlo Using Ordered Overrelaxation Radford M. Neal 발표자 : 장 정 호.

Markov-Chain Monte Carlo CSE586 Computer Vision II Spring 2010, Penn State Univ.

Markov Chains and Random Walks. Def: A stochastic process X={X(t),t ∈ T} is a collection of random variables. If T is a countable set, say T={0,1,2, …

Bayesian Reasoning: Tempering & Sampling A/Prof Geraint F. Lewis Rm 560:

Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.

Tracking Multiple Cells By Correspondence Resolution In A Sequential Bayesian Framework Nilanjan Ray Gang Dong Scott T. Acton C.L. Brown Department of.

Numerical Bayesian Techniques. outline How to evaluate Bayes integrals? Numerical integration Monte Carlo integration Importance sampling Metropolis algorithm.

Markov Chain Monte Carlo Prof. David Page transcribed by Matthew G. Lee.

MCMC in practice Start collecting samples after the Markov chain has “mixed”. How do you know if a chain has mixed or not? In general, you can never “proof”

An Introduction to Markov Chain Monte Carlo Teg Grenager July 1, 2004.

Seminar on random walks on graphs Lecture No. 2 Mille Gandelsman,

7. Metropolis Algorithm. Markov Chain and Monte Carlo Markov chain theory describes a particularly simple type of stochastic processes. Given a transition.

Lecture #9: Introduction to Markov Chain Monte Carlo, part 3

A tutorial on Markov Chain Monte Carlo. Problem  g (x) dx I = If{X } form a Markov chain with stationary probability  i  I  g(x ) i  (x ) i  i=1.

CS246 Latent Dirichlet Analysis. LSI  LSI uses SVD to find the best rank-K approximation  The result is difficult to interpret especially with negative.

CS774. Markov Random Field : Theory and Application Lecture 15 Kyomin Jung KAIST Oct

Daphne Koller Markov Networks General Gibbs Distribution Probabilistic Graphical Models Representation.

CS433 Modeling and Simulation Lecture 11 Continuous Markov Chains Dr. Anis Koubâa 01 May 2009 Al-Imam Mohammad Ibn Saud University.

Daphne Koller Overview Conditional Probability Queries Probabilistic Graphical Models Inference.

STAT 534: Statistical Computing

Daphne Koller Bayesian Networks Semantics & Factorization Probabilistic Graphical Models Representation.

Daphne Koller Variable Elimination Variable Elimination Algorithm Probabilistic Graphical Models Inference.

The Monte Carlo Method/ Markov Chains/ Metropolitan Algorithm from sec in “Adaptive Cooperative Systems” -summarized by Jinsan Yang.

Introduction to Sampling based inference and MCMC

MCMC Output & Metropolis-Hastings Algorithm Part I

Advanced Statistical Computing Fall 2016

Jun Liu Department of Statistics Stanford University

Bayesian inference Presented by Amir Hadadi

Remember that our objective is for some density f(y|) for observations where y and  are vectors of data and parameters,  being sampled from a prior.

Haim Kaplan and Uri Zwick

Seminar on Markov Chains and Mixing Times Elad Katz

Markov Chain Monte Carlo

Lecture 15 Sampling.

MCMC for PGMs: The Gibbs Chain

Opinionated Lessons #39 MCMC and Gibbs Sampling in Statistics

Approximate Inference: Particle-Based Methods

Presentation transcript:

Daphne Koller Sampling Methods Metropolis- Hastings Algorithm Probabilistic Graphical Models Inference

Daphne Koller Reversible Chains Theorem: If detailed balance holds, and T is regular, then T has a unique stationary distribution  Proof:

Daphne Koller Metropolis Hastings Chain At each state x, sample x’ from Q(x  x’) Accept proposal with probability A(x  x’) – If proposal accepted, move to x’ – Otherwise stay at x Proposal distribution Q(x  x’) Acceptance probability: A(x  x’) T(x  x’) = Q(x  x’) A(x  x’) Q(x  x) +  x’  x Q(x  x’) (1-A(x  x’)) if x’  x T(x  x) =

Daphne Koller Acceptance Probability

Template vertLeftWhite1 Consider a Markov chain over a state space s 1,…,s m and assume we wish to sample from the stationary distribution  ( s i )  q -i for q i, which acceptance probability A(s i → s j ) will give rise to a legal MH chain with this stationary distribution? None of the above – this is not a legal proposal distribution. q j-i q i-j p 2i q j-i

Template vertLeftWhite1 In the same setup as the previous problem, let Q(s i,s j )  p^(-d(s i,s j )) but only if |i-j| 1. For j>i, which acceptance probability A(s i → s j ) will give rise to a legal MH chain with this stationary distribution? q j-i None of the above – this is not a legal proposal distribution. q i-j p 2i q j-i

Template vertLeftWhite1 Now let d(s i,s j ) be the clockwise distance between s i and s j (so that d(s i,s i+1 )=1 and d(s i +1,s i )=n-1). Let Q(s i,s j )  p^(-d(s i,s j )). For j>i, which acceptance probability A(s i → s j ) will give rise to an MH chain with this stationary distribution? None of the above – this is not a legal proposal distribution. q j-i q i-j p n-2i q j-i

Daphne Koller Choice of Q Q must be reversible: – Q(x  x’) > 0  Q(x’  x) > 0 Opposing forces – Q should try to spread out, to improve mixing – But then acceptance probability often low

Daphne Koller MCMC for Matching X i =j if i matched to j if every X i has different value otherwise

Daphne Koller MH for Matching: Augmenting Path 1) randomly pick one variable X i 2) sample X i, pretending that all values are available 3) pick the variable whose assignment was taken (conflict), and return to step 2 When step 2 creates no conflict, modify assignment to flip augmenting path

Daphne Koller Example Results MH proposal 1 MH proposal 2 Gibbs

Daphne Koller Summary MH is a general framework for building Markov chains with a particular stationary distribution – Requires a proposal distribution – Acceptance computed via detailed balance Tremendous flexibility in designing proposal distributions that explore the space quickly – But proposal distribution makes a big difference – and finding a good one is not always easy

Daphne Koller END END END