Presentation is loading. Please wait.

Presentation is loading. Please wait.

@ 15/7/2003 Tokyo Institute of Technology 1 Propagating beliefs in spin- glass models Yoshiyuki Kabashima Dept. of Compt. Intel. & Syst.

Published byAmos Morton Modified over 9 years ago

Similar presentations

Presentation on theme: "@ 15/7/2003 Tokyo Institute of Technology 1 Propagating beliefs in spin- glass models Yoshiyuki Kabashima Dept. of Compt. Intel. & Syst."— Presentation transcript:

1 Hayashibara@SMAPIP @ 15/7/2003 Tokyo Institute of Technology 1 Propagating beliefs in spin- glass models Yoshiyuki Kabashima Dept. of Compt. Intel. & Syst. Sci. Tokyo Institute of Technology

2 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology2 Background and Motivation Active research on belief propagation (BP) in information sciences (IS) Similarity to methods in physics TMM & Bethe approx. Difference in interest Physics ⇒ obtained solutions IS ⇒ dynamics of the algorithm may cause unexpected developments in both fields

3 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology3 Purpose and Results Purpose ： analyze dynamics of BP when employed in spin-glass models Results: Macro. dyn. of BP ⇒ RS solution Micro. stability of BP ⇔ AT condition

4 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology4 Outline SG model in Bayesian framework Belief propagation Macro. dyn. and RS solution Micro. stability and AT condition Summary

5 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology5 Spin-glass models SG models on a random (Bethe) lattice K-body interaction C-bonds/spin Randomly constructed for other aspects

6 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology6 SM and Bayesian Statistics Boltzmann dist. = Bayes formula Magnetization = Posterior average

7 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology7 Graph Expression Expression by a bipartite graph … … …

8 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology8 Belief Propagation Iterative inference by passing beliefs … … …

9 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology9 More Precisely ： Posterior average when is left out. ： Effective field when comes in. ： Estimator

10 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology10 Macro. Dyn. vs. RS Solution Distribution (histogram) of beliefs Known result for finite C: Tree approximation (resampling graph/update) Density evolution ⇒ RS solution Vicente, Saad, YK (2000)Richardson & Urbanke (2001)

11 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology11 Novel Result for Infinite C Central limit theorem for infinite C Evolution of average and variance Natural iteration of RS SP eqs.

12 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology12 Experimental Validation SK model （ N=1000,J=1,T=0.5 ） (AT stable) (AT unstable)

13 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology13 Microscopic Instability Possible microscopic instability while BP seems to macroscopically converge Stability analysis of the fixed point

14 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology14 Evolution of Perturbation Dist. of Perturb. Perturbation Evolution is attractive ⇔ the fixed point(=RS solution) is stable

15 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology15 Pictorial Expression What is performed? … … … … … … … … …

16 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology16 Meaning of P. Evolution Link to known results for infinite C Central limit theorem ： P. Evolution → Update of average & variance :Gaussian dist.

17 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology17 Meaning of P. Evolution Critical conditions for growth of fluctuation For K=2 (SK model) Average → T f : Para-Ferro transition Variance → T AT : AT condition :Average :Variance

18 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology18 Analysis for finite C Is P. evolution equivalent to AT analysis even for finite C? AT analysis for finite C is complicated. But, P. evolution is (numerically) possible. K=2 (Wong-Sherrington model) Paramagnetic solution Average → T f Variance → T AT Known result Klein et al (1979) Mezard & Parisi (1987)

19 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology19 Analysis for Finite C Ferromagnetic solution Numerical evaluation of T pevol : New result! N=2000, K=2, C=4, 20000MCS/Spin  : T pevol in Ferro phase

20 Hayashibara&SMAPIP @15/7/2003 Tokyo Institute of Technology20 Summary Close relationship between BP and the replica analysis Macro. dyn. ⇒ RS solution Micro. stability ⇔ AT condition This correspondence may be useful for AT analysis for SG models of finite connectivity. Application to CDMA multiuser detection (Kabashima, to appear in J. Phys. A, 2003)

Download ppt "@ 15/7/2003 Tokyo Institute of Technology 1 Propagating beliefs in spin- glass models Yoshiyuki Kabashima Dept. of Compt. Intel. & Syst."

Similar presentations

Jose-Luis Blanco, Javier González, Juan-Antonio Fernández-Madrigal University of Málaga (Spain) Dpt. of System Engineering and Automation May 19-23 Pasadena,

Jose-Luis Blanco, Javier González, Juan-Antonio Fernández-Madrigal University of Málaga (Spain) Dpt. of System Engineering and Automation May Pasadena,
Anisotropy and Dzyaloshinsky- Moriya Interaction in V15 Manabu Machida, Seiji Miyashita, and Toshiaki Iitaka IIS, U. Tokyo Dept. of Physics, U. Tokyo RIKEN.

Anisotropy and Dzyaloshinsky- Moriya Interaction in V15 Manabu Machida, Seiji Miyashita, and Toshiaki Iitaka IIS, U. Tokyo Dept. of Physics, U. Tokyo RIKEN.
Convergent Message-Passing Algorithms for Inference over General Graphs with Convex Free Energies Tamir Hazan, Amnon Shashua School of Computer Science.

Convergent Message-Passing Algorithms for Inference over General Graphs with Convex Free Energies Tamir Hazan, Amnon Shashua School of Computer Science.
ELEC 303 – Random Signals Lecture 18 – Statistics, Confidence Intervals Dr. Farinaz Koushanfar ECE Dept., Rice University Nov 10, 2009.

ELEC 303 – Random Signals Lecture 18 – Statistics, Confidence Intervals Dr. Farinaz Koushanfar ECE Dept., Rice University Nov 10, 2009.
Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.

Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.
1 Bayesian Image Modeling by Generalized Sparse Markov Random Fields and Loopy Belief Propagation Kazuyuki Tanaka GSIS, Tohoku University, Sendai, Japan.

1 Bayesian Image Modeling by Generalized Sparse Markov Random Fields and Loopy Belief Propagation Kazuyuki Tanaka GSIS, Tohoku University, Sendai, Japan.
Relational Learning with Gaussian Processes By Wei Chu, Vikas Sindhwani, Zoubin Ghahramani, S.Sathiya Keerthi (Columbia, Chicago, Cambridge, Yahoo!) Presented.

Relational Learning with Gaussian Processes By Wei Chu, Vikas Sindhwani, Zoubin Ghahramani, S.Sathiya Keerthi (Columbia, Chicago, Cambridge, Yahoo!) Presented.
Variational Methods TCD Interests Simon Wilson. Background We are new to this area of research – so we can’t say very much about it – but we’re enthusiastic!

Variational Methods TCD Interests Simon Wilson. Background We are new to this area of research – so we can’t say very much about it – but we’re enthusiastic!
Frequency-locking and pattern formation in the Kuramoto model with Manhattan delay Karol Trojanowski, Lech Longa Jagiellonian University Institute of Physics,

Frequency-locking and pattern formation in the Kuramoto model with Manhattan delay Karol Trojanowski, Lech Longa Jagiellonian University Institute of Physics,
Clustering (Part II) 10/07/09. Outline Affinity propagation Quality evaluation.

Clustering (Part II) 10/07/09. Outline Affinity propagation Quality evaluation.
Understanding Belief Propagation and its Applications Dan Yuan June 2004.

Understanding Belief Propagation and its Applications Dan Yuan June 2004.
Novel approach to nonlinear/non- Gaussian Bayesian state estimation N.J Gordon, D.J. Salmond and A.F.M. Smith Presenter: Tri Tran 7-2005.

Novel approach to nonlinear/non- Gaussian Bayesian state estimation N.J Gordon, D.J. Salmond and A.F.M. Smith Presenter: Tri Tran
Random Field Ising Model on Small-World Networks Seung Woo Son, Hawoong Jeong 1 and Jae Dong Noh 2 1 Dept. Physics, Korea Advanced Institute Science and.

Random Field Ising Model on Small-World Networks Seung Woo Son, Hawoong Jeong 1 and Jae Dong Noh 2 1 Dept. Physics, Korea Advanced Institute Science and.
The Role of Specialization in LDPC Codes Jeremy Thorpe Pizza Meeting Talk 2/12/03.

The Role of Specialization in LDPC Codes Jeremy Thorpe Pizza Meeting Talk 2/12/03.
24 November, 2011National Tsin Hua University, Taiwan1 Mathematical Structures of Belief Propagation Algorithms in Probabilistic Information Processing.

24 November, 2011National Tsin Hua University, Taiwan1 Mathematical Structures of Belief Propagation Algorithms in Probabilistic Information Processing.
Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei.

Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields Yong-Joong Kim Dept. of Computer Science Yonsei.
Some Surprises in the Theory of Generalized Belief Propagation Jonathan Yedidia Mitsubishi Electric Research Labs (MERL) Collaborators: Bill Freeman (MIT)

Some Surprises in the Theory of Generalized Belief Propagation Jonathan Yedidia Mitsubishi Electric Research Labs (MERL) Collaborators: Bill Freeman (MIT)
Superglasses and the nature of disorder-induced SI transition

Superglasses and the nature of disorder-induced SI transition
Error correction on a tree: Instanton approach Misha Chertkov (LANL) In collaboration with: V. Chernyak (Corning) M. Stepanov (UA, Tucson) B. Vasic (UA,

Error correction on a tree: Instanton approach Misha Chertkov (LANL) In collaboration with: V. Chernyak (Corning) M. Stepanov (UA, Tucson) B. Vasic (UA,
Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 7th~10th Belief propagation Appendix Kazuyuki Tanaka Graduate School of Information.

Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 7th~10th Belief propagation Appendix Kazuyuki Tanaka Graduate School of Information.

Similar presentations

Ads by Google