1. Applications of Extended Ensemble Monte Carlo Yukito IBA The Institute of Statistical Mathematics, Tokyo, Japan

3. Extended Ensemble MCMC A Generic Name which indicates : Parallel Tempering, Simulated Tempering, Multicanonical Sampling, Wang-Landau, … Umbrella Sampling Valleau and Torrie 1970s

4. Contents 1. Basic Algorithms Parallel Tempering.vs Multicanonical 2. Exact Calculation with soft Constraints Lattice Protein / Counting Tables 3. Rare Events and Large Deviations Communication Channels Chaotic Dynamical Systems

5. Basic Algorithms Parallel Tempering Multicanonical Monte Carlo

6. References in physics Iba (2001) Extended Ensemble Monte Carlo Int. J. Mod. Phys. C12 p.623. A draft version will be found at http://arxiv.org/abs/cond-mat/0012323 Landau and Binder (2005) A Guide to Monte Carlo Simulations in Statistical Physics (2nd ed., Cambridge) A number of preprints will be found in Los Alamos Arxiv on the web. # This slide is added after the talk

7. Slow mixing by multimodal dist.×× ×

8. Bridging fast mixing high temperature slow mixing low temperature

9. Path Sampling 1.Facilitate Mixing 2.Calculate Normalizing Constant (“free energy”) In Physics: from 2. to 1. 1970s  1990s “Path Sampling” Gelman and Meng (1998) stress 2. but 1. is also important

10. Parallel Tempering a.k.a. Replica Exchange MC Metropolis Coupled MCMC Simulate Many “Replica”s in Parallel MCMC in a Product Space Geyer (1991), Kimura and Taki (1991) Hukushima and Nemoto (1996) Iba(1993, in Japanese)

11. Examples Gibbs Distributions with different temperatures Any Family parameterized by a hyperparameter

12. Exchange of Replicas K=4

13. Accept/Reject Exchange Calculate Metropolis Ratio Generate a Uniform Random Number in [0,1) and accept exchange iff

14. Detailed Balance in Extended Space Combined Distribution

15. Multicanonical Monte Carlo sufficient statistics Exponential Family Energy not Expectation Berg et al. (1991,1992)

16. Density of States The number of which satisfy

17. Multicanonical Sampling

18. Weight and Marginal Distribution Original (Gibbs) Multicanonical Random

19. flat marginal distribution Scanning broad range of E

20. Reweighting Formally, for arbitrary it holds. Practically, is required, else the variance diverges in a large system.

21. Q. How can we do without knowledge on D(E) Ans ． Estimate D(E) in the preliminary runs k th simulation Simplest Method : Entropic Sampling in

22. Estimation of Density of States 5 k=1 2 410 k=15 1411 30000 MCS 30000 MCS 3 (Ising Model on a random net)

23. Estimation of D(E) Histogram Piecewise Linear Fitting, Kernel Density Estimation.. Wang-Landau Flat Histogram Entropic Sampling Original Multicanonical Continuous Cases D(E)dE : Non-trivial Task

24. Parallel Tempering / Multicanonical parallel tempering combined distribution simulated tempering mixture distribution to approximate

25. disordered ordered Potts model (2-dim, q=10 states)

26. Phase Coexistence/ 1 st order transition parameter (Inverse Temperature) changes sufficient statistics (Energy) jumps water and ice coexists

27. disordered ordered Potts model (2-dim, q=10 states) bridging by multicanoncal construction

28. Comparison @ Simple Liquids, Potts Models.. Multicanonical seems better than Parallel Tempering @ But, for more difficult cases ? ex. Ising Model with three spin Interaction

30. Soft Constraints Lattice Protein Counting Tables The results on Lattice Protein are taken from joint works with G Chikenji (Nagoya Univ) and Macoto Kikuchi (Osaka Univ) Some examples are also taken from the other works by Kikuchi and coworkers.

31. Lattice Protein Model Motivation Simplest Models of Protein Lattice Protein : Prototype of “Protein-like molecules” Ising Model : Prototype of “Magnets”

32. Lattice Protein (2-dim HP)

33. FIXED sequence of conformation of chain STOCHASTIC VARIABLE SELF AVOIDING (SELF OVERLAP is not allowed) IMPORTANT! andcorresponds to 2-types of amino acids (H and P)

34. E(X)= - the number of Energy (HP model) in x the energy of conformation x is defined as

35. Examples Here we do not count the pairs neighboring on the chain but it is not essential because the difference is const. E=0 E= - 1

36. MCMC Slow Mixing Even Non-Ergodicity with local moves Chikenji et al. (1999) Phys. Rev. Lett. 83 pp.1886-1889 Bastolla et al. (1998) Proteins 32 pp. 52-66

37. Multicanonical Multicanonical w.r.t. E only NOT SUFFUCIENT Self-Avoiding condition is essential

38. Soft Constraint Self-Avoiding condition is essential Soft Constraint is the number of monomers that occupy the site i

39. Multi Self-Overlap Sampling Multi Self-Overlap Ensemble Bivariate Density of States in the (E,V) plane E V (self-overlap) Samples with are used for the calculation of the averages EXACT !!

40. V=0large V E Generation of Paths by softening of constraints

41. Comparison with multicanonical with hard self-avoiding constraint conventional (hard constraint) proposed (soft constraint) switching between three groups of minimum energy states of a sequence

42. optimization

43. optimization (polymer pairs) Nakanishi and Kikuchi (2006) J.Phys.Soc.Jpn. 75 pp.064803 / q-bio/0603024

44. double peaks An Advantage of the method is that it can use for the sampling at any temperature as well as optimization 3-dim Yue and Dill (1995) Proc. Nat. Acad. Sci. 92 pp.146-150

45. Another Sequence non monotonic change of the structure Chikenji and Kikuchi (2000) Proc. Nat. Acad. Sci 97 pp.14273 - 14277

46. Related Works Self-Avoiding Walk without interaction / Univariate Extension Vorontsov-Velyaminov et al. : J.Phys.Chem.,100,1153-1158 (1996) Lattice Protein but not exact / Soft-Constraint without control Shakhnovich et al. Physical Review Letters 67 1665 (1991) Continuous homopolymer -- Relax “core” Liu and Berne J Chem Phys 99 6071 (1993) See References in Extended Ensemble Monte Carlo, Int J Phys C 12 623-656 (2001) but esp. for continuous cases, there seems more in these five years

48. Counting Tables ４９２ ３５７ ８１６ Pinn et al. (1998) Counting Magic Squares Soft Constraints + Parallel Tempering

49. Sampling by MCMC Multiple Maxima Parallel Tempering

50. Normalization Constant calculated by Path sampling (thermodynamic integration)

51. Latin square (3x3) For column, any given number once and only once For each column, any given number appears once and only once For each raw, any given once and only once For each raw, any given number appears once and only once

52. Latin square (26x26) # This sample is taken from the web.

53. Counting Latin Squares 6 10 11 410000 MCS x 27 replicas 510000 MCS x 49 replicas other 3 trials

54. Counting Tables Soft Constraints + Extended Ensemble MC “Quick and Dirty” ways of calculating the number of tables that satisfy given constraints. It may not be optimal for a special case, but no case-by-case tricks, no mathematics, and no brain is required.

56. Rare Events and Large Deviations Communication Channels #1 Chaotic Dynamical Systems #2 # 1 Part of joint works with Koji Hukushima (Tokyo Univ). # 2 Part of joint works with Tatsuo Yanagita (Hokkaido Univ). (The result shown here is mostly due to him )

57. Applications of MCMC Statistical Physics (1953 ~ ) Statistical Inference ( 1970s, 1980s, 1990~) Solution to any problem on sampling & counting estimation of large deviation generation of rare events

58. Noisy Communication Channel prior encoded & degraded decodedistance (bit errors) by Viterbi, loopy BP, MCMC

59. Distribution of Bit Errors Kronecker delta tails of the distribution is not easy to estimate

60. Introduction of MCMC Sampling noise in channels by the MCMC Given an error-correcting code Some patterns of noise are very harmful difficult to correct Some patterns of noise are safe easy to correct NOT sampling from the posterior

61. Multicanonical Strategy MCMC sampling of Broad distribution of  Broad distribution of distance and

62. Multicanonical Sampling MCMC Sampling and with the weight Estimated by the iteration of preliminary runs exactly what we want, but can be..

63. flat marginal distribution Scanning broad range of bit errors Enable efficient calculation of the tails of the distribution (large deviation)

64. Example Convolutional Code Binary Symmetric Channel Fix the number of noise (flipped bits) Viterbi decoding

65. Simplification In this case is independent ofSet Binary Symmetric Channel Fix the number of noise (flipped bits) sum over the possible positions of the noise

66. Simulation the number of bit errors difficult to calculate by simple sampling

67. Correlated Channels It will be useful for the study of error- correcting code in a correlated channel. Without assuming models of correlation in the channel we can sample relevant correlation patterns.

69. Rare events in Dynamical Systems Deterministic Chaos Doll et al. (1994), Kurchan et al. (2005) Sasa, Hayashi, Kawasaki.. (2005 ~) (Mostly) Stochastic Dynamics Chandler Group Frenkel et al. and more … Transition Path Sampling Stagger and Step Method Sweet, Nusse, and Yorke (2001)

70. Sampling Initial Condition Sampling initial condition of Chaotic dynamical systems  Rare Events

71. Double Pendulum

72. control and stop the pendulum one of the three positions Unstable fixed points energy dissipation (friction) is assumed i.e., no time reversal sym.

73. T is max time Definition of artificial “energy” stop = zero velocity stopping position penalty to long time

74. Metropolis step Integrate Equation of Motion and Simulate Trajectory Perturb Initial State Evaluate “Energy” Reject or Accept

75. for given T  Parallel Tempering

76. An animation by Yanagita is shown in the talk, but might not be seen on the web.

77. Summary Extended Ensemble + Soft Constraint strategy gives simple solutions to a number of difficult problems The use of MCMC should not be restricted to the standard ones in Physics and Bayesian Statistics. To explore new applications of MCMC extended ensemble MC will play an essential role.

78. END