Markov chain monte carlo Kevin Stevenson AST 4762/5765 Markov chain monte carlo

What is MCMC? Random sampling algorithm Estimates model parameters and their uncertainty Only samples regions of high probability rather than uniform sampling Faster More efficient Region is called “phase space”

Phase Space Space in which all possible states of a system are represented Each space corresponds to one unique point Every parameter (or DoF) is represented by an axis Eg. 3 position vectors (x, y, z) require a 3-dimensional phase space Eg. Add time to produce a 4-D phase space Can be represented very easily in Python using arrays

Markov Chain A stochastic (or random) process having the Markov property Indeterminate future, evolution is described by probability distributions “Given the present state, future states are independent of the past states” In other words… At a given step, the system has a set of parameters that define its state At the next step, the system might change states or it might remain in the same state according to a certain probability Each prospective step is determined ONLY by its current state (no past memory)

Example: Random Walk Consider a drunk standing under a lamppost trying to get home He takes a step in a random direction (N, E, S, W), each having equal probability Having forgotten his previous step, he again takes a step in a random direction Forms a Markov chain

Random Walk Methods Metropolis-Hastings algorithm Gibbs sampling Vary all parameters simultaneously Accept step with a certain probability Gibbs sampling Special (usually faster) case of M-H Hold all parameters constant, except one Vary parameter to find best fit Choose next parameter and repeat Slice sampling Multi-try Metropolis

Avoiding Random Walk May want stepper to avoid doubling back Methods Faster convergence Harder to implement Methods Successive over-relaxation Variation on Gibbs sampling Hybrid Monte Carlo Introduces momentum

Metropolis-Hastings Algorithm Goal: want to estimate model parameters and their uncertainty M-H algorithm generates a sequence of samples from a probability distribution that is difficult to sample from directly Distribution may not be Gaussian May not know distribution at all How does it generate this set?

Preferential Probability Want to visit a point x with a probability proportional to some given distribution functions, π(x) “Probability distribution” or “target density” Preferentially samples where π(x) is large Probability distribution: Probability of x falling within a particular interval Ergodic Must, in principle, be able to reach every point in the region of interest

Let Me Propose… Proposal distribution/density: Depends on current state, x1 Generates a new proposed sample, x2 Must also be ergodic Can be approximated by a Gaussian centered around x1 May be symmetric:

Target & Proposal Densities P(x) = target density Q(x,xt) = proposal density

Don’t We All Want To Feel Accepted? Acceptance probability: If α ≥ 1: Accept the proposed step Current state becomes x2 If α < 1: Accept step with probability α Reject step with probability 1 – α State remains at x1

Not Too Hot, Not Too Cold Acceptance rate: fraction of accepted steps Want an acceptance rate of 30 – 70% Too high => slow convergence Too low => small sample size Must tune the proposal density, Q, to obtain an acceptable acceptance rate If Q is Gaussian the we tune the standard deviation, σ Think of σ as a step size

What Is π?

Burn-in The equilibrium distribution is rapidly approached from any starting position, x0 Proof: Due to ergodicity, choosing any point as the starting point is equivalent to jumping into the equilibrium distribution chain at that particular point in time Need burn-in to “forget” the starting position Remove AT LEAST the first 2% of the total run length Better yet, look at your data!

After The Fire Remaining set of states represent a sample from the distribution π(x) Compute the mean (or median) and standard deviation of each parameter in your set Plug those parameters into the model DONE!!!