@ 15/7/2003 Tokyo Institute of Technology 1 Propagating beliefs in spin- glass models Yoshiyuki Kabashima Dept. of Compt. Intel. & Syst.

Slides:

Advertisements

Similar presentations

Jose-Luis Blanco, Javier González, Juan-Antonio Fernández-Madrigal University of Málaga (Spain) Dpt. of System Engineering and Automation May Pasadena,

Advertisements

Anisotropy and Dzyaloshinsky- Moriya Interaction in V15 Manabu Machida, Seiji Miyashita, and Toshiaki Iitaka IIS, U. Tokyo Dept. of Physics, U. Tokyo RIKEN.

Convergent Message-Passing Algorithms for Inference over General Graphs with Convex Free Energies Tamir Hazan, Amnon Shashua School of Computer Science.

ELEC 303 – Random Signals Lecture 18 – Statistics, Confidence Intervals Dr. Farinaz Koushanfar ECE Dept., Rice University Nov 10, 2009.

Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.

1 Bayesian Image Modeling by Generalized Sparse Markov Random Fields and Loopy Belief Propagation Kazuyuki Tanaka GSIS, Tohoku University, Sendai, Japan.

Relational Learning with Gaussian Processes By Wei Chu, Vikas Sindhwani, Zoubin Ghahramani, S.Sathiya Keerthi (Columbia, Chicago, Cambridge, Yahoo!) Presented.

Variational Methods TCD Interests Simon Wilson. Background We are new to this area of research – so we can’t say very much about it – but we’re enthusiastic!

Frequency-locking and pattern formation in the Kuramoto model with Manhattan delay Karol Trojanowski, Lech Longa Jagiellonian University Institute of Physics,

Clustering (Part II) 10/07/09. Outline Affinity propagation Quality evaluation.

Understanding Belief Propagation and its Applications Dan Yuan June 2004.

Novel approach to nonlinear/non- Gaussian Bayesian state estimation N.J Gordon, D.J. Salmond and A.F.M. Smith Presenter: Tri Tran

Random Field Ising Model on Small-World Networks Seung Woo Son, Hawoong Jeong 1 and Jae Dong Noh 2 1 Dept. Physics, Korea Advanced Institute Science and.

The Role of Specialization in LDPC Codes Jeremy Thorpe Pizza Meeting Talk 2/12/03.

24 November, 2011National Tsin Hua University, Taiwan1 Mathematical Structures of Belief Propagation Algorithms in Probabilistic Information Processing.

Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields Yong-Joong Kim Dept. of Computer Science Yonsei.

Some Surprises in the Theory of Generalized Belief Propagation Jonathan Yedidia Mitsubishi Electric Research Labs (MERL) Collaborators: Bill Freeman (MIT)

Superglasses and the nature of disorder-induced SI transition

Error correction on a tree: Instanton approach Misha Chertkov (LANL) In collaboration with: V. Chernyak (Corning) M. Stepanov (UA, Tucson) B. Vasic (UA,

Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 7th~10th Belief propagation Appendix Kazuyuki Tanaka Graduate School of Information.

1 October, 2007 ALT&DS2007 (Sendai, Japan ） 1 Introduction to Probabilistic Image Processing and Bayesian Networks Kazuyuki Tanaka Graduate School of Information.

Estimating parameters in a statistical model Likelihood and Maximum likelihood estimation Bayesian point estimates Maximum a posteriori point.

Physics 270 – Experimental Physics. Standard Deviation of the Mean (Standard Error) When we report the average value of n measurements, the uncertainty.

Belief Propagation. What is Belief Propagation (BP)? BP is a specific instance of a general class of methods that exist for approximate inference in Bayes.

ECE 8443 – Pattern Recognition LECTURE 07: MAXIMUM LIKELIHOOD AND BAYESIAN ESTIMATION Objectives: Class-Conditional Density The Multivariate Case General.

Mixture Models, Monte Carlo, Bayesian Updating and Dynamic Models Mike West Computing Science and Statistics, Vol. 24, pp , 1993.

Physics Fluctuomatics / Applied Stochastic Process (Tohoku University) 1 Physical Fluctuomatics Applied Stochastic Process 9th Belief propagation Kazuyuki.

28 February, 2003University of Glasgow1 Cluster Variation Method and Probabilistic Image Processing -- Loopy Belief Propagation -- Kazuyuki Tanaka Graduate.

Kevin Cherry Robert Firth Manohar Karki. Accurate detection of moving objects within scenes with dynamic background, in scenarios where the camera is.

自旋玻璃与消息传递算法 Spin Glass and Message-Passing Algorithms 周海军中国科学院理论物理研究所.

Continuous Variables Write message update equation as an expectation: Proposal distribution W t (x t ) for each node Samples define a random discretization.

Survey Propagation. Outline Survey Propagation: an algorithm for satisfiability 1 – Warning Propagation – Belief Propagation – Survey Propagation Survey.

Tokyo Institute of Technology, Japan Yu Nishiyama and Sumio Watanabe Theoretical Analysis of Accuracy of Gaussian Belief Propagation.

14 October, 2010LRI Seminar 2010 (Univ. Paris-Sud)1 Statistical performance analysis by loopy belief propagation in probabilistic image processing Kazuyuki.

Estimating Component Availability by Dempster-Shafer Belief Networks Estimating Component Availability by Dempster-Shafer Belief Networks Lan Guo Lane.

29 December, 2008 National Tsing Hua University, Taiwan 1 Introduction to Probabilistic Image Processing and Bayesian Networks Kazuyuki Tanaka Graduate.

Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 7th~10th Belief propagation Kazuyuki Tanaka Graduate School of Information Sciences,

Workshop on Optimization in Complex Networks, CNLS, LANL (19-22 June 2006) Application of replica method to scale-free networks: Spectral density and spin-glass.

Boosted Particle Filter: Multitarget Detection and Tracking Fayin Li.

Lecture 2: Statistical learning primer for biologists

Wei Sun and KC Chang George Mason University March 2008 Convergence Study of Message Passing In Arbitrary Continuous Bayesian.

Graduate School of Information Sciences, Tohoku University

Slow Dynamics of Magnetic Nanoparticle Systems: Memory effects P. E. Jönsson, M. Sasaki and H. Takayama ISSP, Tokyo University Co-workers: H. Mamiya and.

1 Chapter 8: Model Inference and Averaging Presented by Hui Fang.

The Unscented Particle Filter 2000/09/29 이 시은. Introduction Filtering –estimate the states(parameters or hidden variable) as a set of observations becomes.

Additional NN Models Reinforcement learning (RL) Basic ideas: –Supervised learning: (delta rule, BP) Samples (x, f(x)) to learn function f(.) precise error.

CS Statistical Machine learning Lecture 25 Yuan (Alan) Qi Purdue CS Nov

Markov Networks: Theory and Applications Ying Wu Electrical Engineering and Computer Science Northwestern University Evanston, IL 60208

Long-Range Frustration among Unfrozen Vertices in the Ground-State Configurations of a Spin-Glass System Haijun Zhou 海军周 Institute of Theoretical Physics,

30 November, 2005 CIMCA2005, Vienna 1 Statistical Learning Procedure in Loopy Belief Propagation for Probabilistic Image Processing Kazuyuki Tanaka Graduate.

ICPR2004 (24 July, 2004, Cambridge) 1 Probabilistic image processing based on the Q-Ising model by means of the mean- field method and loopy belief propagation.

10 October, 2007 University of Glasgow 1 EM Algorithm with Markov Chain Monte Carlo Method for Bayesian Image Analysis Kazuyuki Tanaka Graduate School.

ベーテ自由エネルギーに対するCCCPアルゴリズムの拡張

New Results of the Color Diversity Problem Jack Raymond (英偉文) and K. Y

Statistical-Mechanical Approach to Probabilistic Image Processing -- Loopy Belief Propagation and Advanced Mean-Field Method -- Kazuyuki Tanaka and Noriko.

Graduate School of Information Sciences, Tohoku University

Spin Glass Cavity Method in Random K-Satisfiability Problem

Sublinear Computational Time Modeling in Statistical Machine Learning Theory for Markov Random Fields Kazuyuki Tanaka GSIS, Tohoku University, Sendai,

Markov Random Fields with Efficient Approximations

Numerical Study of Partition Function Zeros on Recursive Lattices

Graduate School of Information Sciences, Tohoku University, Japan

Graduate School of Information Sciences Tohoku University, Japan

Generalized Belief Propagation

Graduate School of Information Sciences, Tohoku University

Physical Fluctuomatics 7th~10th Belief propagation

Expectation-Maximization & Belief Propagation

Probabilistic image processing and Bayesian network

Adaptive Cooperative Systems Chapter 3 Coperative Lattice Systems

CS639: Data Management for Data Science

Kazuyuki Tanaka Graduate School of Information Sciences

Presentation transcript:

@ 15/7/2003 Tokyo Institute of Technology 1 Propagating beliefs in spin- glass models Yoshiyuki Kabashima Dept. of Compt. Intel. & Syst. Sci. Tokyo Institute of Technology

Tokyo Institute of Technology2 Background and Motivation Active research on belief propagation (BP) in information sciences (IS) Similarity to methods in physics TMM & Bethe approx. Difference in interest Physics ⇒ obtained solutions IS ⇒ dynamics of the algorithm may cause unexpected developments in both fields

Tokyo Institute of Technology3 Purpose and Results Purpose ： analyze dynamics of BP when employed in spin-glass models Results: Macro. dyn. of BP ⇒ RS solution Micro. stability of BP ⇔ AT condition

Tokyo Institute of Technology4 Outline SG model in Bayesian framework Belief propagation Macro. dyn. and RS solution Micro. stability and AT condition Summary

Tokyo Institute of Technology5 Spin-glass models SG models on a random (Bethe) lattice K-body interaction C-bonds/spin Randomly constructed for other aspects

Tokyo Institute of Technology6 SM and Bayesian Statistics Boltzmann dist. = Bayes formula Magnetization = Posterior average

Tokyo Institute of Technology7 Graph Expression Expression by a bipartite graph … … …

Tokyo Institute of Technology8 Belief Propagation Iterative inference by passing beliefs … … …

Tokyo Institute of Technology9 More Precisely ： Posterior average when is left out. ： Effective field when comes in. ： Estimator

Tokyo Institute of Technology10 Macro. Dyn. vs. RS Solution Distribution (histogram) of beliefs Known result for finite C: Tree approximation (resampling graph/update) Density evolution ⇒ RS solution Vicente, Saad, YK (2000)Richardson & Urbanke (2001)

Tokyo Institute of Technology11 Novel Result for Infinite C Central limit theorem for infinite C Evolution of average and variance Natural iteration of RS SP eqs.

Tokyo Institute of Technology12 Experimental Validation SK model （ N=1000,J=1,T=0.5 ） (AT stable) (AT unstable)

Tokyo Institute of Technology13 Microscopic Instability Possible microscopic instability while BP seems to macroscopically converge Stability analysis of the fixed point

Tokyo Institute of Technology14 Evolution of Perturbation Dist. of Perturb. Perturbation Evolution is attractive ⇔ the fixed point(=RS solution) is stable

Tokyo Institute of Technology15 Pictorial Expression What is performed? … … … … … … … … …

Tokyo Institute of Technology16 Meaning of P. Evolution Link to known results for infinite C Central limit theorem ： P. Evolution → Update of average & variance :Gaussian dist.

Tokyo Institute of Technology17 Meaning of P. Evolution Critical conditions for growth of fluctuation For K=2 (SK model) Average → T f : Para-Ferro transition Variance → T AT : AT condition :Average :Variance

Tokyo Institute of Technology18 Analysis for finite C Is P. evolution equivalent to AT analysis even for finite C? AT analysis for finite C is complicated. But, P. evolution is (numerically) possible. K=2 (Wong-Sherrington model) Paramagnetic solution Average → T f Variance → T AT Known result Klein et al (1979) Mezard & Parisi (1987)

Tokyo Institute of Technology19 Analysis for Finite C Ferromagnetic solution Numerical evaluation of T pevol : New result! N=2000, K=2, C=4, 20000MCS/Spin  : T pevol in Ferro phase

Tokyo Institute of Technology20 Summary Close relationship between BP and the replica analysis Macro. dyn. ⇒ RS solution Micro. stability ⇔ AT condition This correspondence may be useful for AT analysis for SG models of finite connectivity. Application to CDMA multiuser detection (Kabashima, to appear in J. Phys. A, 2003)