Dynamics and its stability of Boltzmann-machine learning algorithm for gray scale image restoration J. Inoue (Hokkaido Univ.) and K. Tanaka (Tohoku Univ.)

Slides:

Advertisements

Similar presentations

CHAPTER 8 More About Estimation. 8.1 Bayesian Estimation In this chapter we introduce the concepts related to estimation and begin this by considering.

Advertisements

Biointelligence Laboratory, Seoul National University

3 March, 2003University of Glasgow1 Statistical-Mechanical Approach to Probabilistic Inference --- Cluster Variation Method and Generalized Loopy Belief.

Introduction to Signal Estimation. 94/10/142 Outline 

Efficient Simulation of Physical System Models Using Inlined Implicit Runge-Kutta Algorithms Vicha Treeaporn Department of Electrical & Computer Engineering.

Constrained Optimization Economics 214 Lecture 41.

Maximum likelihood (ML)

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide 2- 1 Homework, Page 245 Find the domain of the function f. Use limits.

24 November, 2011National Tsin Hua University, Taiwan1 Mathematical Structures of Belief Propagation Algorithms in Probabilistic Information Processing.

Asymptotic Techniques

1 物理フラクチュオマティクス論 Physical Fluctuomatics 応用確率過程論 Applied Stochastic Process 第 5 回グラフィカルモデルによる確率的情報処理 5th Probabilistic information processing by means of.

1 Physical Fluctuomatics 5th and 6th Probabilistic information processing by Gaussian graphical model Kazuyuki Tanaka Graduate School of Information Sciences,

3 September, 2009 SSP2009, Cardiff, UK 1 Probabilistic Image Processing by Extended Gauss-Markov Random Fields Kazuyuki Tanaka Kazuyuki Tanaka, Muneki.

ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: Deterministic vs. Random Maximum A Posteriori Maximum Likelihood Minimum.

Calculus Review. How do I know where f is increasing? O It is where f prime is positive. O Find the derivative and set equal to zero. Use test points.

Population Dynamics Application of Eigenvalues & Eigenvectors.

28 February, 2003University of Glasgow1 Cluster Variation Method and Probabilistic Image Processing -- Loopy Belief Propagation -- Kazuyuki Tanaka Graduate.

University of Colorado Boulder ASEN 5070: Statistical Orbit Determination I Fall 2014 Professor Brandon A. Jones Lecture 26: Singular Value Decomposition.

September 2007 IW-SMI2007, Kyoto 1 A Quantum-Statistical-Mechanical Extension of Gaussian Mixture Model Kazuyuki Tanaka Graduate School of Information.

Tokyo Institute of Technology, Japan Yu Nishiyama and Sumio Watanabe Theoretical Analysis of Accuracy of Gaussian Belief Propagation.

14 October, 2010LRI Seminar 2010 (Univ. Paris-Sud)1 Statistical performance analysis by loopy belief propagation in probabilistic image processing Kazuyuki.

Phisical Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 4th Maximum likelihood estimation and EM algorithm Kazuyuki Tanaka Graduate School.

Mathe III Lecture 7 Mathe III Lecture 7. 2 Second Order Differential Equations The simplest possible equation of this type is:

AUTOMATIC CONTROL THEORY II Slovak University of Technology Faculty of Material Science and Technology in Trnava.

Domain of Attraction Remarks on the domain of attraction

The generalization of Bayes for continuous densities is that we have some density f(y|  ) where y and  are vectors of data and parameters with  being.

Week 41 How to find estimators? There are two main methods for finding estimators: 1) Method of moments. 2) The method of Maximum likelihood. Sometimes.

Trajectory Generation Cherevatsky Boris. Mathematical Fact Given n+1 values of a n-degree polynomial : i.e. if we have the values: we can compute the.

M.Sc. in Economics Econometrics Module I Topic 4: Maximum Likelihood Estimation Carol Newman.

Derivative-Enhanced Variable Fidelity Kriging Approach Dept. of Mechanical Engineering, University of Wyoming, USA Wataru YAMAZAKI 23 rd, September, 2010.

Ordinary Differential Equations

2-4 Solving Equations with Variables on Both Sides.

Chaos Control in Nonlinear Dynamical Systems Nikolai A. Magnitskii Institute for Systems Analysis of RAS, Moscow,Russia.

30 November, 2005 CIMCA2005, Vienna 1 Statistical Learning Procedure in Loopy Belief Propagation for Probabilistic Image Processing Kazuyuki Tanaka Graduate.

2( ) 8x + 14y = 4 -12x – 14y = x = x = 4 8x + 14y = 4 8(4) + 14y = y = y = -28 ___ ___ y = -2 The solution is (4, -2)

A Method to Approximate the Bayesian Posterior Distribution in Singular Learning Machines Kenji Nagata, Sumio Watanabe Tokyo Institute of Technology.

29 June, 2006 Kyoto University 1 画像処理における確率伝搬法と EM アルゴリズムの統計的性能評価東北大学大学院情報科学研究科田中和之 Reference 田中和之 : ガウシアングラフィカルモデルにもとづく確率的情報処理におけ.

ICPR2004 (24 July, 2004, Cambridge) 1 Probabilistic image processing based on the Q-Ising model by means of the mean- field method and loopy belief propagation.

10 October, 2007 University of Glasgow 1 EM Algorithm with Markov Chain Monte Carlo Method for Bayesian Image Analysis Kazuyuki Tanaka Graduate School.

マルコフ確率場の統計的機械学習の数理とデータサイエンスへの展開 Statistical Machine Learning in Markov Random Field and Expansion to Data Sciences 田中和之東北大学大学院情報科学研究科 Kazuyuki Tanaka.

Using Number Sense to Solve One-Step Equations

Statistical-Mechanical Approach to Probabilistic Image Processing -- Loopy Belief Propagation and Advanced Mean-Field Method -- Kazuyuki Tanaka and Noriko.

Graduate School of Information Sciences, Tohoku University

Using Number Sense to Solve One-Step Equations

Sublinear Computational Time Modeling in Statistical Machine Learning Theory for Markov Random Fields Kazuyuki Tanaka GSIS, Tohoku University, Sendai,

Autonomous Equations; Phase Lines

Graduate School of Information Sciences, Tohoku University, Japan

Graduate School of Information Sciences, Tohoku University

Graduate School of Information Sciences, Tohoku University

Solve the equation for x. {image}

Modern Control Systems (MCS)

'Linear Hierarchical Models'

Where did we stop? The Bayes decision rule guarantees an optimal classification… … But it requires the knowledge of P(ci|x) (or p(x|ci) and P(ci)) We.

Graduate School of Information Sciences, Tohoku University

What is the function of the graph? {applet}

Differentiate the function: {image} .

Probabilistic image processing and Bayesian network

Probabilistic image processing and Bayesian network

Graduate School of Information Sciences, Tohoku University

X y y = x2 - 3x Solutions of y = x2 - 3x y x –1 5 –2 –3 6 y = x2-3x.

Ch 3. Linear Models for Regression (2/2) Pattern Recognition and Machine Learning, C. M. Bishop, Previously summarized by Yung-Kyun Noh Updated.

Probabilistic image processing and Bayesian network

Variables and Equations

Statistical Model A statistical model for some data is a set of distributions, one of which corresponds to the true unknown distribution that produced.

Statistical Model A statistical model for some data is a set of distributions, one of which corresponds to the true unknown distribution that produced.

Graduate School of Information Sciences, Tohoku University

Kazuyuki Tanaka Graduate School of Information Sciences

San Diego State University

Evaluate the integral {image}

Presentation transcript:

Dynamics and its stability of Boltzmann-machine learning algorithm for gray scale image restoration J. Inoue (Hokkaido Univ.) and K. Tanaka (Tohoku Univ.) The 3 rd International Symposium on Slow Dynamics in Complex Systems in Sendai November 2003

Plan of this talk  Bayesian image restoration and hyper-parameter estimation  Boltzmann-machine learning algorithm for the hyper- parameter estimation  Dynamic behavior of the BML algorithm  Stability of the solution  Concluding remarks

Bayesian image restoration OriginalCorrupted We treat images and the degrading process as spin systems

Definitions of the model by spin systems Original Corrupted : Hyper-parameters (true value)

Bayesian approach and MPM estimation takes its minimum at [Inoue and Carlucci (2001)]

Maximization of the marginal likelihood via Boltzmann-machine learning algorithm takes its maximum aton average We evaluate the data-averaged BML algorithm at the mean-field level : [Inoue and Tanaka (2003)]

Dynamic behavior of the hyper-parameters are integrated numerically

Analysis of the stability Expand the BML equations around and check the sign of eigenvalues of the Hessian A The solution is asymptotically stable

True hyper-parameter dependence of the stability The solution of the BML algorithm is asymptotically stable as long as the solution is identical to the true value of the hyper-parameters (fixed)

Behavior of the BML algorithm around the solution Trajectories in the hyper-parameter space (around the solution)

Concluding remarks  We investigated dynamic behavior and its stability of the BML algorithm for gray scale image restoration  We derived the data-averaged BML equations  The solution is asymptotically stable as long as the solution is identical to the true value of the hyper- parameters  More details of the present study are available at Send