Applications of Extended Ensemble Monte Carlo Yukito IBA The Institute of Statistical Mathematics, Tokyo, Japan

Extended Ensemble MCMC A Generic Name which indicates : Parallel Tempering, Simulated Tempering, Multicanonical Sampling, Wang-Landau, … Umbrella Sampling Valleau and Torrie 1970s

Contents 1. Basic Algorithms Parallel Tempering.vs Multicanonical 2. Exact Calculation with soft Constraints Lattice Protein / Counting Tables 3. Rare Events and Large Deviations Communication Channels Chaotic Dynamical Systems

Basic Algorithms Parallel Tempering Multicanonical Monte Carlo

References in physics Iba (2001) Extended Ensemble Monte Carlo Int. J. Mod. Phys. C12 p.623. A draft version will be found at Landau and Binder (2005) A Guide to Monte Carlo Simulations in Statistical Physics (2nd ed., Cambridge) A number of preprints will be found in Los Alamos Arxiv on the web. # This slide is added after the talk

Slow mixing by multimodal dist.×× ×

Bridging fast mixing high temperature slow mixing low temperature

Path Sampling 1.Facilitate Mixing 2.Calculate Normalizing Constant (“free energy”) In Physics: from 2. to s  1990s “Path Sampling” Gelman and Meng (1998) stress 2. but 1. is also important

Parallel Tempering a.k.a. Replica Exchange MC Metropolis Coupled MCMC Simulate Many “Replica”s in Parallel MCMC in a Product Space Geyer (1991), Kimura and Taki (1991) Hukushima and Nemoto (1996) Iba(1993, in Japanese)

Examples Gibbs Distributions with different temperatures Any Family parameterized by a hyperparameter

Exchange of Replicas K=4

Accept/Reject Exchange Calculate Metropolis Ratio Generate a Uniform Random Number in [0,1) and accept exchange iff

Detailed Balance in Extended Space Combined Distribution

Multicanonical Monte Carlo sufficient statistics Exponential Family Energy not Expectation Berg et al. (1991,1992)

Density of States The number of which satisfy

Multicanonical Sampling

Weight and Marginal Distribution Original (Gibbs) Multicanonical Random

flat marginal distribution Scanning broad range of E

Reweighting Formally, for arbitrary it holds. Practically, is required, else the variance diverges in a large system.

Q. How can we do without knowledge on D(E) Ans ． Estimate D(E) in the preliminary runs k th simulation Simplest Method : Entropic Sampling in

Estimation of Density of States 5 k= k= MCS MCS 3 (Ising Model on a random net)

Estimation of D(E) Histogram Piecewise Linear Fitting, Kernel Density Estimation.. Wang-Landau Flat Histogram Entropic Sampling Original Multicanonical Continuous Cases D(E)dE : Non-trivial Task

Parallel Tempering / Multicanonical parallel tempering combined distribution simulated tempering mixture distribution to approximate

disordered ordered Potts model (2-dim, q=10 states)

Phase Coexistence/ 1 st order transition parameter (Inverse Temperature) changes sufficient statistics (Energy) jumps water and ice coexists

disordered ordered Potts model (2-dim, q=10 states) bridging by multicanoncal construction

Simple Liquids, Potts Models.. Multicanonical seems better than Parallel But, for more difficult cases ? ex. Ising Model with three spin Interaction

Soft Constraints Lattice Protein Counting Tables The results on Lattice Protein are taken from joint works with G Chikenji (Nagoya Univ) and Macoto Kikuchi (Osaka Univ) Some examples are also taken from the other works by Kikuchi and coworkers.

Lattice Protein Model Motivation Simplest Models of Protein Lattice Protein : Prototype of “Protein-like molecules” Ising Model : Prototype of “Magnets”

Lattice Protein (2-dim HP)

FIXED sequence of conformation of chain STOCHASTIC VARIABLE SELF AVOIDING (SELF OVERLAP is not allowed) IMPORTANT! andcorresponds to 2-types of amino acids (H and P)

E(X)= - the number of Energy (HP model) in x the energy of conformation x is defined as

Examples Here we do not count the pairs neighboring on the chain but it is not essential because the difference is const. E=0 E= - 1

MCMC Slow Mixing Even Non-Ergodicity with local moves Chikenji et al. (1999) Phys. Rev. Lett. 83 pp Bastolla et al. (1998) Proteins 32 pp

Multicanonical Multicanonical w.r.t. E only NOT SUFFUCIENT Self-Avoiding condition is essential

Soft Constraint Self-Avoiding condition is essential Soft Constraint is the number of monomers that occupy the site i

Multi Self-Overlap Sampling Multi Self-Overlap Ensemble Bivariate Density of States in the (E,V) plane E V (self-overlap) Samples with are used for the calculation of the averages EXACT !!

V=0large V E Generation of Paths by softening of constraints

Comparison with multicanonical with hard self-avoiding constraint conventional (hard constraint) proposed (soft constraint) switching between three groups of minimum energy states of a sequence

optimization

optimization (polymer pairs) Nakanishi and Kikuchi (2006) J.Phys.Soc.Jpn. 75 pp / q-bio/

double peaks An Advantage of the method is that it can use for the sampling at any temperature as well as optimization 3-dim Yue and Dill (1995) Proc. Nat. Acad. Sci. 92 pp

Another Sequence non monotonic change of the structure Chikenji and Kikuchi (2000) Proc. Nat. Acad. Sci 97 pp

Related Works Self-Avoiding Walk without interaction / Univariate Extension Vorontsov-Velyaminov et al. : J.Phys.Chem.,100, (1996) Lattice Protein but not exact / Soft-Constraint without control Shakhnovich et al. Physical Review Letters (1991) Continuous homopolymer -- Relax “core” Liu and Berne J Chem Phys (1993) See References in Extended Ensemble Monte Carlo, Int J Phys C (2001) but esp. for continuous cases, there seems more in these five years

Counting Tables ４９２ ３５７ ８１６ Pinn et al. (1998) Counting Magic Squares Soft Constraints + Parallel Tempering

Sampling by MCMC Multiple Maxima Parallel Tempering

Normalization Constant calculated by Path sampling (thermodynamic integration)

Latin square (3x3) For column, any given number once and only once For each column, any given number appears once and only once For each raw, any given once and only once For each raw, any given number appears once and only once

Latin square (26x26) # This sample is taken from the web.

Counting Latin Squares MCS x 27 replicas MCS x 49 replicas other 3 trials

Counting Tables Soft Constraints + Extended Ensemble MC “Quick and Dirty” ways of calculating the number of tables that satisfy given constraints. It may not be optimal for a special case, but no case-by-case tricks, no mathematics, and no brain is required.

Rare Events and Large Deviations Communication Channels #1 Chaotic Dynamical Systems #2 # 1 Part of joint works with Koji Hukushima (Tokyo Univ). # 2 Part of joint works with Tatsuo Yanagita (Hokkaido Univ). (The result shown here is mostly due to him )

Applications of MCMC Statistical Physics (1953 ~ ) Statistical Inference ( 1970s, 1980s, 1990~) Solution to any problem on sampling & counting estimation of large deviation generation of rare events

Noisy Communication Channel prior encoded & degraded decodedistance (bit errors) by Viterbi, loopy BP, MCMC

Distribution of Bit Errors Kronecker delta tails of the distribution is not easy to estimate

Introduction of MCMC Sampling noise in channels by the MCMC Given an error-correcting code Some patterns of noise are very harmful difficult to correct Some patterns of noise are safe easy to correct NOT sampling from the posterior

Multicanonical Strategy MCMC sampling of Broad distribution of  Broad distribution of distance and

Multicanonical Sampling MCMC Sampling and with the weight Estimated by the iteration of preliminary runs exactly what we want, but can be..

flat marginal distribution Scanning broad range of bit errors Enable efficient calculation of the tails of the distribution (large deviation)

Example Convolutional Code Binary Symmetric Channel Fix the number of noise (flipped bits) Viterbi decoding

Simplification In this case is independent ofSet Binary Symmetric Channel Fix the number of noise (flipped bits) sum over the possible positions of the noise

Simulation the number of bit errors difficult to calculate by simple sampling

Correlated Channels It will be useful for the study of error- correcting code in a correlated channel. Without assuming models of correlation in the channel we can sample relevant correlation patterns.

Rare events in Dynamical Systems Deterministic Chaos Doll et al. (1994), Kurchan et al. (2005) Sasa, Hayashi, Kawasaki.. (2005 ~) (Mostly) Stochastic Dynamics Chandler Group Frenkel et al. and more … Transition Path Sampling Stagger and Step Method Sweet, Nusse, and Yorke (2001)

Sampling Initial Condition Sampling initial condition of Chaotic dynamical systems  Rare Events

Double Pendulum

control and stop the pendulum one of the three positions Unstable fixed points energy dissipation (friction) is assumed i.e., no time reversal sym.

T is max time Definition of artificial “energy” stop = zero velocity stopping position penalty to long time

Metropolis step Integrate Equation of Motion and Simulate Trajectory Perturb Initial State Evaluate “Energy” Reject or Accept

for given T  Parallel Tempering

An animation by Yanagita is shown in the talk, but might not be seen on the web.

Summary Extended Ensemble + Soft Constraint strategy gives simple solutions to a number of difficult problems The use of MCMC should not be restricted to the standard ones in Physics and Bayesian Statistics. To explore new applications of MCMC extended ensemble MC will play an essential role.

END