28 February, 2003University of Glasgow1 Cluster Variation Method and Probabilistic Image Processing -- Loopy Belief Propagation -- Kazuyuki Tanaka Graduate School of Information Sciences Tohoku University
28 February, 2003University of Glasgow2 Probabilistic Model and Image Restoration Original Image Degraded Image Transmission Noise
28 February, 2003University of Glasgow3 Image Restoration and Magnetic Material Restored images are determined from a priori information and given data. Ordered states are determined from interactions and external fields. Feature detection from the data and image processing by means of filters. Interpretation and prediction of physical property by means of physical model. Similarity of Mathematical Structure Regular lattice consisting of a lot of nodes. Neighbouring spin interactions and Markov random field
28 February, 2003University of Glasgow4 Massive Information Processing and Probabilistic Information Processing Computational Complexity. Approximation algorithms for massive information processing by means of advanced mean-field methods. Application of the cluster variation method (Bethe/Kikuchi method) to massive information processing Cluster Variation Method is equivalent to a generalized loopy belief propagation for probabilistic inference in the artificial intelligence.
28 February, 2003University of Glasgow5 Important point in the application of cluster variation method to probabilistic image processing Design of iterative algorithms for probabilistic inference based on cluster variation method (Computer Science). Hyperparameter estimation (Statistics). Cooperative phenomena in probabilistic models and probabilistic information processing (Physics).
28 February, 2003University of Glasgow6 Degradation Process and A Priori Probability in Binary Image Restoration Degradation Process (Binary Symmetric Channel) A Priori Probability
28 February, 2003University of Glasgow7 A Priori Probability in Binary Image Restoration
28 February, 2003University of Glasgow8 Bayes Formula and A Posteriori Probability Maximization of Posterior Marginal
28 February, 2003University of Glasgow9 A Posteriori Probability in Binary Image Restoration
28 February, 2003University of Glasgow10 Kullback-Leibler divergence
28 February, 2003University of Glasgow11 Kullback-Leibler Divergence
28 February, 2003University of Glasgow12 Basic Framework of Pair Approximation in Cluster Variation Method
28 February, 2003University of Glasgow13 Propagation Rule of Pair Approximation in Cluster Variation Method Update Rule is reduced to Loopy Belief Propagation
28 February, 2003University of Glasgow14 One-Body Marginal Probability of Pair Approximation in CVM
28 February, 2003University of Glasgow15 Two-Body Marginal Probability of Pair Approximation in CVM
28 February, 2003University of Glasgow16 Message Propagation Rule of Pair Approximation in CVM
28 February, 2003University of Glasgow17 Binary Image Restoration Original images are generated by Monte Carlo simulations in the a priori probability. Original Image Degraded Image (p=0.2) Restored Image
28 February, 2003University of Glasgow18 Binary Image Restoration Original ImageDegraded ImageRestored Image
28 February, 2003University of Glasgow19 Hyperparameter Estimation Maximization of Marginal Likelihood
28 February, 2003University of Glasgow20 Binary Image Restoration Original images are generated by Monte Carlo simulations in the a priori probability. Original Image Degraded Image (p=0.2) Restored Image
28 February, 2003University of Glasgow21 Binary Image Restoration Original Image Degraded Image Mean Field Approximation Pair Approximation in CVM Hyperparameters are determined so as to maximize the marginal likelihood.
28 February, 2003University of Glasgow22 Multi-Valued Image Restoration Degradation Process
28 February, 2003University of Glasgow23 A Priori Probability in Multi-Valued Image Restoration Q-state Ising Model Q-state Potts Model Kronecker Delta
28 February, 2003University of Glasgow24 Multi-Valued Image Restoration (Q=4) Hyperparameters are determined so as to maximize the marginal likelihood. Degraded Image Restored Image Original Image 4-state Ising Model 4-state Potts Model
28 February, 2003University of Glasgow25 Multi-Valued Image Restoration (Q=4) Hyperparameters are determined so as to maximize the marginal likelihood. Degraded Image ( 3p=0.3 ) 4-state Potts Model Original Image 4-state Ising Model
28 February, 2003University of Glasgow26 Summary Probabilistic Image Processing by Bayes Formula Cluster Variation Method and Loopy Belief Propagation Binary Image Restoration Multi-Valued Image Restoration
28 February, 2003University of Glasgow27 Other Practical Applications Edge Detection Segmentation Texture Analysis Image Compression Motion Detection Color Image
28 February, 2003University of Glasgow28 Other Theoretical Works Hyperparameter Estimation by EM algorithm Statistical Performance Estimation and Spin Glass Theory Replica method Inequality of Statistical Quantity Line Field Generalized Loopy Belief Propagation and Cluster Variation Method Information Geometry and Cluster Variation Method