1. Graduate School of Information Sciences, Tohoku University
Physical Fluctuomatics Applied Stochastic Process 10th Probabilistic image processing by means of physical models
Kazuyuki Tanaka
Graduate School of Information Sciences, Tohoku University
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

2. Textbooks
Kazuyuki Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 7.
Kazuyuki Tanaka: Statistical-mechanical approach to image processing (Topical Review), Journal of Physics A: Mathematical and General, vol.35, no.37, pp.R81-R150, 2002.
Muneki Yasuda, Shun Kataoka and Kazuyuki Tanaka: Computer Vision and Image Processing 3 (Edited by Y. Yagi and H. Saito): Chapter 6. Stochastic Image Processing, pp , Advanced Communication Media CO., Ltd., December 2010 (in Japanese).
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

3. Contents Probabilistic Image Processing Loopy Belief Propagation
Statistical Learning Algorithm
More Practical Probabilistic models for Image Processing
Summary
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

4. Image Representation in Computer Vision
Digital image is defined on the set of points arranged on a square lattice.
The elements of such a digital array are called pixels.
We have to treat more than 100,000 pixels even in the digital cameras and the mobile phones.
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

5. Image Representation in Computer Vision
At each point, the intensity of light is represented as an integer number or a real number in the digital image data.
A monochrome digital image is then expressed as a two-dimensional light intensity function and the value is proportional to the brightness of the image at the pixel.
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

6. Noise Reduction by Conventional Filters
The function of a linear filter is to take the sum of the product of the mask coefficients and the intensities of the pixels.
Smoothing Filters
192
202
190
192
202
190
202
219
120
202
173
120
100
218
110
100
218
110
It is expected that probabilistic algorithms for image processing can be constructed from such aspects in the conventional signal processing.
Markov Random Fields
Algorithm
Probabilistic Image Processing
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

7. Fundamental Probabilistic Theory for Image Processing
Marginal Probability of Event B A B C D
Marginalization
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

8. Fundamental Probabilistic Theory for Image Processing: Bayes Formula
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

9. Fundamental Probabilistic Theory for Image Processing: Bayes Formula
Prior Probability
Data-Generating Process
Bayes Rule
Posterior Probability A
Event B is given as the observed data.
Event A corresponds to the original information to estimate.
Thus the Bayes formula can be applied to the estimation of the original information from the given data. B
Bayesian
Network
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

10. Image Restoration by Probabilistic Model
Assumption 1: The degraded image is randomly generated from the original image by according to the degradation process.
Assumption 2: The original image is randomly generated by according to the prior probability.
Noise
Transmission
Original Image
Degraded Image
Bayes Formula
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

11. Image Restoration by Probabilistic Model
The original images and degraded images are represented by f = (f1,f2,…,f|V|) and g = (g1,g2,…,g|V|), respectively.
Original
Image
Degraded
Image
Position Vector of Pixel i i i
fi: Light Intensity of Pixel i
in Original Image
gi: Light Intensity of Pixel i
in Degraded Image
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

12. Probabilistic Modeling of Image Restoration
Assumption 2: The original image is generated according to a prior probability. Prior Probability consists of a product of functions defined on the neighbouring pixels. i j
Random Fields
Product over All the Nearest Neighbour Pairs of Pixels
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

13. Probabilistic Modeling of Image Restoration
Assumption 1: A given degraded image is obtained from the original image by changing the state of each pixel to another state by the same probability, independently of the other pixels. fi gi fi gi or
Random Fields
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

14. Bayesian Image Analysis
Degraded Image
Prior Probability
Original
Image
Degradation Process
Posterior Probability
V：Set of All the pixels
E：Set of all the nearest neighbour pairs of pixels
Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability.
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

15. Estimation of Original Image
We have some choices to estimate the restored image from posterior probability.
In each choice, the computational time is generally exponential order of the number of pixels.
(1)
Maximum A Posteriori (MAP)
estimation
In the image restoration, we usually have to estimate the hyperparameters alpha and p.
In statistics, the maximum likelihood estimation is often employed.
In the standpoint of maximum likelihood estimation, the hyperparameters are determined so as to maximize the marginal likelihood defined by marginalize the joint probability for the original image and degraded image with respect to the original image.
The marginal likelihood is expressed in terms of the partition functions of the a priori probabilistic model and the a posteriori probabilistic model.
We can calculate these partition functions approximately by using the Bethe approximation.
(2)
Maximum posterior
marginal (MPM) estimation
(3)
Thresholded Posterior
Mean (TPM) estimation
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

16. Prior Probability of Image Processing
Digital Images (fi=0,1,…,q-1)
Sampling of Markov Chain Monte Carlo Method
E: Set of all the nearest-neighbour pairs of pixels
Paramagnetic
Ferromagnetic
Fluctuations increase near α=1.76….
V: Set of all the pixels
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

17. Bayesian Image Analysis by Gaussian Graphical Model
Prior
Probability
V:Set of all
the pixels
Patterns are generated by MCMC.
E:Set of all the nearest-neighbour pairs of pixels
Markov Chain Monte Carlo Method
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

18. Degradation Process for Binary Image
Binary Symmetric Channel
V：Set of all the pixels
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

19. Degradation Process for Gaussian Graphical Model
Degradation Process is assumed to be the additive white Gaussian noise.
Histogram of Gaussian Random Numbers
Original Image f
Gaussian Noise n
Degraded Image g
V: Set of all
the pixels
Degraded image is obtained by adding a white Gaussian noise to the original image.
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

20. Bayesian Image Analysis
Degraded Image
Prior Probability
Original
Image
Degradation Process
Posterior Probability
V：Set of All the pixels
E：Set of all the nearest neighbour pairs of pixels
Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability.
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

21. Bayesian Image Analysis
Prior Probability
Original Image
Degradation Process
Degraded Image 画素
Posterior Probability
Computational Complexity
Marginal Posterior Probability of Each Pixel
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

22. Posterior Probability for Image Processing by Bayesian Analysis
Binary Symmetric Channel
Additive White Gaussian Noise
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

23. Bayesian Network and Posterior Probability of Image Processing
Probabilistic Model expressed in terms of Graphical Representations with Cycles
V: Set of all the pixels
E: Set of all the nearest neighbour pairs of pixels
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

24. ∂i: Set of all the nearest neighbour pixels of the pixel i
Markov Random Fields
Markov Random Fields
∂i: Set of all the nearest neighbour pixels of the pixel i
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

25. Contents Probabilistic Image Processing Loopy Belief Propagation
Statistical Learning Algorithm
More Practical Probabilistic models for Image Processing
Summary
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

26. Marginal Probability in Belief Propagation
In order to compute the marginal probability Pr{F2|G=g},
we take summations over all the pixels except the pixel 2.
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

27. Marginal Probability in Belief Propagation2
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

28. Marginal Probability in Belief Propagation2 2
In the belief propagation, the marginal probability Pr{F2|G=g} is approximately expressed in terms of the messages from the neighbouring region of the pixel 2.
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

29. Marginal Probability in Belief Propagation1 2
In order to compute the marginal probability Pr{F1,F2|G=g},
we take summations over all the pixels except the pixels 1 and2.
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

30. Marginal Probability in Belief Propagation1 2 1 2
In the belief propagation, the marginal probability Pr{F1,F2|G=g} is approximately expressed in terms of the messages from the neighbouring region of the pixels 1 and 2.
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

31. Loopy Belief Propagation
Probabilistic Model expressed in terms of Graphical Representations with Cycles
V: Set of all the pixels
E: Set of all the nearest neighbour pairs of pixels 2 1 3 4 5
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

32. Loopy Belief Propagation2 1 3 4 5
Simultaneous fixed point equations to determine the messages
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

33. Belief Propagation Algorithm for Image Processing
Step 1: Solve the simultaneous fixed point equations
for messages by using iterative method j i i
Step 2: Substitute the messages to approximate expressions of marginal probabilities for each pixel and compute estimated image so as to maximize the approximate marginal probability at each pixel.
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

34. Belief Propagation in Probabilistic Image Processing
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

35. Contents Probabilistic Image Processing Loopy Belief Propagation
Statistical Learning Algorithm
More Practical Probabilistic models for Image Processing
Summary
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

36. Statistical Estimation of Hyperparameters
Hyperparameters a, b are determined so as to maximize the marginal likelihood Pr{G=g|a,b} with respect to a, b.
In the image restoration, we usually have to estimate the hyperparameters alpha and p.
In statistics, the maximum likelihood estimation is often employed.
In the standpoint of maximum likelihood estimation, the hyperparameters are determined so as to maximize the marginal likelihood defined by marginalize the joint probability for the original image and degraded image with respect to the original image.
The marginal likelihood is expressed in terms of the partition functions of the a priori probabilistic model and the a posteriori probabilistic model.
We can calculate these partition functions approximately by using the Bethe approximation.
Original Image
Degraded Image
Marginalized with respect to F
Marginal Likelihood
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

37. Maximum Likelihood in Signal Processing
Incomplete Data
Original Image
=Parameter
Degraded Image
=Data
Marginal Likelihood
Hyperparameter
Extremum Condition
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

38. Maximum Likelihood and EM algorithm
Degraded Image
=Data
Incomplete Data
Original Image
=Parameter
Marginal Likelihood
Q -function
Hyperparameter
Expectation Maximization (EM) algorithm provide us one of Extremum Points of Marginal Likelihood.
Extemum Condition
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

39. Maximum Likelihood and EM algorithm
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

40. Bayesian Image Analysis by Exact Solution of Gaussian Graphical Model
Posterior Probability
V:Set of all
the pixels
Average of the posterior probability can be calculated by using the multi-dimensional Gauss integral Formula
E:Set of all the nearest-neghbour pairs of pixels
|V|x|V| matrix
Multi-Dimensional Gaussian Integral Formula
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

41. Bayesian Image Analysis by Exact Solution of Gaussian Graphical Model
Iteration Procedure in Gaussian Graphical Model
In the image restoration, we usually have to estimate the hyperparameters alpha and p.
In statistics, the maximum likelihood estimation is often employed.
In the standpoint of maximum likelihood estimation, the hyperparameters are determined so as to maximize the marginal likelihood defined by marginalize the joint probability for the original image and degraded image with respect to the original image.
The marginal likelihood is expressed in terms of the partition functions of the a priori probabilistic model and the a posteriori probabilistic model.
We can calculate these partition functions approximately by using the Bethe approximation.
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

42. Image Restorations by Exact Solution and Loopy Belief Propagation of Gaussian Graphical Model
Finally, we show only the results for the gray-level image restoration.
For each numerical experiments, the loopy belief propagation ca give us better results than the ones by conventional filters.
Loopy Belief Propagation
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

43. Image Restorations by Gaussian Graphical Model
Original Image
Degraded Image
Belief Propagation
Exact
MSE: 1512
MSE:325
MSE:315
Finally, we show only the results for the gray-level image restoration.
For each numerical experiments, the loopy belief propagation ca give us better results than the ones by conventional filters.
Lowpass Filter
Wiener Filter
Median Filter
MSE: 411
MSE: 545
MSE: 447
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

44. Image Restoration by Gaussian Graphical Model
Original Image
Degraded Image
Belief Propagation
Exact
MSE: 1529
MSE: 260
MSE236
Finally, we show only the results for the gray-level image restoration.
For each numerical experiments, the loopy belief propagation ca give us better results than the ones by conventional filters.
Lowpass Filter
Wiener Filter
Median Filter
MSE: 224
MSE: 372
MSE: 244
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

45. a(t) s(t) Image Restoration by Discrete Gaussian Graphical Model
Original Image
Q=4
a(t)
Degraded Image
s=1
Finally, we show only the results for the gray-level image restoration.
For each numerical experiments, the loopy belief propagation ca give us better results than the ones by conventional filters.
MSE:
SNR: (dB)
Belief Propagation
MSE:
s(t)
SNR: (dB)
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

46. a(t) s(t) Image Restoration by Discrete Gaussian Graphical Model
Original Image
Q=4
a(t)
Degraded Image
s=1
Finally, we show only the results for the gray-level image restoration.
For each numerical experiments, the loopy belief propagation ca give us better results than the ones by conventional filters.
MSE:
SNR: (dB)
Belief Propagation
MSE:
s(t)
SNR: (dB)
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

47. a(t) s(t) Image Restoration by Discrete Gaussian Graphical Model
Original Image
Q=8
a(t)
Degraded Image
s=1
Finally, we show only the results for the gray-level image restoration.
For each numerical experiments, the loopy belief propagation ca give us better results than the ones by conventional filters.
MSE:
SNR: (dB)
Belief Propagation
MSE:
s(t)
SNR: (dB)
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

48. a(t) s(t) Image Restoration by Discrete Gaussian Graphical Model
Original Image
Q=8
a(t)
Degraded Image
s=1
Finally, we show only the results for the gray-level image restoration.
For each numerical experiments, the loopy belief propagation ca give us better results than the ones by conventional filters.
MSE:
SNR: (dB)
Belief Propagation
MSE:
s(t)
SNR: (dB)
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

49. Binary Image Restoration by Discrete Gaussian Graphical Model
Maximization of Marginal Likelihood
MSE
p=0.1
p=0.2
Finally, we show only the results for the gray-level image restoration.
For each numerical experiments, the loopy belief propagation ca give us better results than the ones by conventional filters.
Binary Symmetric Channel
MSE
p=0.1
p=0.2
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

50. Binary Image Restoration by Discrete Gaussian Graphical Model
Binary Symmetric Channel
Degraded Image
Original Image
Restored Image
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

51. Color Image Restorations by Gaussian Graphical Model
Original Image
Degraded Image
Restored Image
Maximization of Marginal Likelihood
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

52. Color Image Restorations by Gaussian Graphical Model
Original Image
Degraded Image
Restored Image
Maximization of Marginal Likelihood
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

53. Contents Probabilistic Image Processing Loopy Belief Propagation
Statistical Learning Algorithm
More Practical Probabilistic models for Image Processing
Summary
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

54. Compound Gauss Markov Random Field Model
Prior Information for Line Fields V(u)
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

55. Compound Gauss Markov Random Field Model
Original Image
Degraded Image
Gaussian Graphical Model (No Edge States)
Compound Gauss Markov Random Field Model with Edge States
Compound Gauss Markov Random Field Model with Quantized Edge States
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

56. Compound Gauss Markov Random Field Model
Original Image
Degraded Image
Gaussian Graphical Model (No Edge States)
Compound Gauss Markov Random Field Model with Edge States
Compound Gauss Markov Random Field Model with Quantized Edge States
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

57. Contents Probabilistic Image Processing Loopy Belief Propagation
Statistical Learning Algorithm
More Practical Probabilistic models for Image Processing
Summary
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

58. Summary Image Processing Bayesian Network for Image Processing
Design of Belief Propagation Algorithm for Image Processing
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

59. Practice 10-1
For random fields F=(F1,F2,…,F|V|)T and G=(G1,G2,…,G|V|)T and state vectors f=(f1,f2,…,f|V|)T and g=(g1,g2,…,g|V|)T, we consider the following posterior probability distribution:
Here ZPosterior is a normalization constant and E is the set of all the nearest-neighbour pairs of nodes. Prove that
and
where ∂i denotes the set of all the nearest neighbour pixels of the pixel i.
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

60. Give some numerical experiments for p=0.1 and 0.2.
Practice 10-2
Make a program that generate a degraded image g=(g1,g2,…,g|V|) from a given binary image f =(f1,f2,…,f|V|) in which each state variable fi takes only 0 and 1 by according to the following conditional probability distribution of the binary symmetric channel:
Give some numerical experiments for p=0.1 and 0.2.
Example of Numerical Experiments for p=0.2
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

61. Give some numerical experiments for s=20 and 40.
Practice 10-3
Make a program that generate a degraded image g=(g1,g2,…,g|V|) from a given image f =(f1,f2,…,f|V|) in which each state variable fi takes integers between 0 and q-1 by according to the following conditional probability distribution of the additive white Gaussian noise:
Give some numerical experiments for s=20 and 40.
Histogram of Gaussian Random Numbers
Original Image f
Gaussian Noise n
Degraded Image g
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

62. Practice 10-4
Make a program for estimating q-valued original images f =(f1,f2,…,f|V|) from a given degraded image g =(g1,g2,…,g|V|) by using a belief propagation algorithm for the following posterior probability distribution for q valued images:
Give some numerical experiments.
K. Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 8 .
M. Yasuda, S. Kataoka and K. Tanaka: Computer Vision and Image Processing 3 (Edited by Y. Yagi and H. Saito): Chapter 6. Stochastic Image Processing, pp , Advanced Communication Media CO., Ltd., December 2010 (in Japanese).
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)

63. Practice 10-5
Make a program for estimating original images f =(f1,f2,…,f|V|) from a given degraded image g =(g1,g2,…,g|V|) by using a belief propagation algorithm for the following posterior probability distribution based on the Gaussian graphical model:
Give some numerical experiments.
Kazuyuki Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 8 and Appendix G.
Physical Fluctuomatics / Applied Stochansic Process (Tohoku University)