1. Biointelligence Laboratory, Seoul National University
Ch 6. Markov Random Fields 6.6 ~ 6.7 Adaptive Cooperative Systems, Martin Beckerman, 1997.
Summarized by
M.-O. Heo
Biointelligence Laboratory, Seoul National University

2. (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Contents
6.6 Simultaneous Autoregressive Models
6.6.1 Simultaneous Autoregressive Random Fields
6.6.2 Image Reconstruction
6.6.3 Fourier Computation and Block Circulant Matrices
6.7 The Method of Geman and Geman
6.7.1 Maximum A posteriori (MAP) Estimation
6.7.2 Clique Potentials
6.7.3 The Posterior Distribution
6.7.4 The Gibbs Sampler
(C) 2009, SNU Biointelligence Lab, 

3. Simultaneous Autoregressive Random Fields
Simultaneous Autoregressive Models
AR representations
Finite support
SAR Random Fields
Defined with noise random variables
Support
Zero-mean independent random variable
Real-valued coefficient
over the lattice
The set of acquaintances of (i,j)
(C) 2009, SNU Biointelligence Lab, 

4. Simultaneous Autoregressive Random Fields
SAR random field is a MRF?
Using joint probability distribution
Find the following relation
(C) 2009, SNU Biointelligence Lab, 

5. Simultaneous Autoregressive Random Fields
Toroidal lattice SAR field
Defined by a pair of equations of the form
For the interior region
For the boundary
(C) 2009, SNU Biointelligence Lab, 

6. (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Image Reconstruction
Problem Formulation
Notations
D{f(x, y)} : point-spread function, linear, space invariant
f(x, y) : ideal image
g(x, y) : observed image
n(x, y) : noise process (independent of the imaging process)
Degredation process (or noise process)
One dimensional process
Two dimensional process
(C) 2009, SNU Biointelligence Lab, 

7. (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Image reconstruction methods 1. Constrained least-squares estimation (LSE)
Constrained least-squares estimation (LSE)
Objective function and the corresponding solution
Selecting an appropriate form of Q
Subject to constraint
Optimal estimate)
Correlation matrix for the image f
Correlation matrix for the noise n
(C) 2009, SNU Biointelligence Lab, 

8. (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Image reconstruction methods 2. linear minimum mean square error (MMSE) estimation
linear minimum mean square error (MMSE) estimation
Define the error e
Minimize the positive quantity
Assuming that f transform into g under a linear operator L,
Assumption of signal-independent noise
Solution by this step)
More assumptions
Image f is stationary
Rf is approximated by the block circulant covariance matrix Qf
White noise process with common variance
Solution
Using SAR and GRF image models
(C) 2009, SNU Biointelligence Lab, 

9. Fourier Computation and Block Circulant Matrices
Degredation with discrete Fourier transform
Diagonal elements Tkk are the eigenvalues of the matrix R
(C) 2009, SNU Biointelligence Lab, 

10. (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Using one dimensional convolution to the degredation model
Discrete Fourier transform with W-1
The result
The discrete convolution theorem
Convolution of a pair of arrays in the spatial domain to element-by-element multiplication in the frequency domain, and vice versa.
Two-dimensional image model
(C) 2009, SNU Biointelligence Lab, 

11. The method of Geman and Geman
MAP estimation for image reconstruction
3 main points
Use of Bayesian formalism to incorporate constraints
Gibbs-Markov equivalence
A second, dual lattice system containing discontinuity-preserving information
(C) 2009, SNU Biointelligence Lab, 

12. Maximum A Posteriori (MAP) Estimation
Posterior distribution to maximize
Degredation process
Additive noise to be described by the multivariate gaussian
Assumption for noise : uncorrelated and stationary with zero mean
Likelihood
Assumption: no blurring, point-spread function takes other effects
(C) 2009, SNU Biointelligence Lab, 

13. (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Model the image as a MRF
Prior distribution
An Energy composed of a sum of local contributions
Partition function Z
The site potentials
Strength parameter like the inverse of the temperature
Clique potentials
Cliques associated with the neighborhood system for the site
(C) 2009, SNU Biointelligence Lab, 

14. (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Clique Potentials
The general property of the lattice systems
Piecewise smoothness
Neighboring pixels tend to have similar grey values
Ex) Four-level model for texture realization
Four type of elements
(a)
(b)
(c)
(d)
Encourages all elements within a given clique to have the same grey level.
Controls the percentage of pixels of each region. And useful for textured image regions.
(C) 2009, SNU Biointelligence Lab, 

15. The Posterior Distribution
The energy
If we use the quadratic difference potentials as clique potentials
Gibbs distribution having the same potential as the posterior distribution
Favors maximally smooth restorations, penalizing large pairwise contrasts in pixel values within a local neighborhood.
Encourages restoration that are not too different from the data

16. (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
The Gibbs Sampler
Finding desired low-energy states using the SA
Gibbs Sampler
The sampling method on the Gibbs distribution
In the exponential in the Posterior Gibbs distribution…
The result of this sampler
Controls the relative hardness of the constraints
Annealing temperature
(C) 2009, SNU Biointelligence Lab, 

17. (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
The Gibbs Sampler
The general form
(C) 2009, SNU Biointelligence Lab,