1 Markov random field: A brief introduction (2) Tzu-Cheng Jen Institute of Electronics, NCTU

2 Outline Markov random field: Review Edge-preserving regularization in image processing

3 Markov random field: Review

4 Prior knowledge In order to explain the concept of the MRF, we first introduce following definition: 1. i: Site (Pixel) 2. f i : The value at site i (Intensity) 3. S: Set of sites (Image) 4. N i : The neighboring site of i (1st order neighborhood of i: f2, f4, f5, f7 ) 5. C i : Clique, the subset of S and the element in this subset must be neighboring f1f1 f2f2 f3f3 f4f4 fifi f5f5 f6f6 f7f7 f8f8 A 3x3 imagined image

5 Markov random field (MRF) View the 2D image f as the collection of the random variables (Random field) A random field is said to be Markov random field if it satisfies following properties Red: Neighboring site f1f1 f2f2 f3f3 f4f4 fifi f5f5 f6f6 f7f7 f8f8

6 Gibbs random field (GRF) and Gibbs distribution A random field is said to be a Gibbs random field if and only if its configuration f obeys Gibbs distribution, that is: f1f1 f2f2 f3f3 f4f4 fifi f5f5 f6f6 f7f7 f8f8 A 3x3 imagined image U(f): Energy function; T: Temperature V i (f): Clique potential

7 Markov-Gibbs equivalence Hammersley-Clifford theorem: A random field F is an MRF if and only if F is a GRF Red: Neighboring site f1f1 f2f2 f3f3 f4f4 fifi f5f5 f6f6 f7f7 f8f8

8 Edge-preserving regularization in image processing

9 MAP formulation for denoising problem Noisy signal ddenoised signal f d = f + N(0, σ)

10 MAP formulation for denoising problem A signal denoising problem could be modeled as the MAP estimation problem, that is, (Prior model) (Observation model)

11 MAP formulation for denoising problem Assume the observation is the true signal plus the independent Gaussian noise, that is Assume the unknown data f is MRF, the prior model is:

12 MAP formulation for denoising problem Substitute above information into the MAP estimator, we could get: Observation model (Similarity measure) Prior model (Reconstruction constrain, Regularization)

13 The solver of the optimization problem: Gradient descent algorithm

14 Simulation results for denoising problem Simulation result Processed profiles are blurred !

15 Discussion for the phenomenon of blur (1) From the potential function point of view: Quadratic function is used as potential function g=x 2 Simulation result 1st derivative Energy

16 Discussion for the phenomenon of blur (2) From the optimization process point of view (gradient descent algorithm): Update equation of gradient descent:

17 Edge-preserving regularization S. Geman and D. Geman, "Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images," IEEE Trans. Pattern Anal. Mach. Intell, 6, , S.Z. Li, “On Discontinuity-adaptive smoothness priors in computer vision,” IEEE Trans. Pattern Anal. Mach. Intell, June, Pierre Charbonnier et al, “Deterministic edge preserving regularization in computed imaging,” IEEE Trans. Image Processing, Feb, S.Z. Li, “Markov random field modeling in computer vision,“ Springer, 1995

18 MRF with pixel process and line process (Geman and Geman) Lattice of pixel site: S P Labeling value: f i p (real value) Lattice of line site: S E Labeling value: f ii’ E (only 0 or 1)

19 MRF with pixel process and line process (Geman and Geman) Based on the concept of line process, we could modify the original restoration problem as follows: Goal: Find realizations f p and f E such that edge-preserving regularization could be achieved ?

20 MRF with pixel process and line process (Geman and Geman) Define the prior: Substitute above information into the MAP estimator, we could get: The above optimization problem is a combination of real and combinatorial problem

21 MRF with pixel process and line process (Geman and Geman) Blake and Zisserman covert previous restoration problem into real minimization problem by introducing truncated quadratic function as potential function Truncated quadratic function Energy

22 MRF with pixel process and line process (Geman and Geman) Simulation results Original imageDegraded imageRestoration result (1000 iterations)

23 MRF with pixel process and line process (Geman and Geman) Simulation results Original image Degraded image Restoration result (1000 iterations) Restoration result (100 iterations)

24 Discontinuity-adaptive regularization (S. Z. Li) Revisit the gradient descent algorithm Adjust it adaptively ! Derivative or compensator Weight or interaction function

25 Discontinuity-adaptive regularization (S. Z. Li) For edge-preserving regularization, interaction function h r should satisfy following property:

26 Discontinuity-adaptive regularization (S. Z. Li) Possible choices for interaction function h r

27 Discontinuity-adaptive regularization (S. Z. Li) Simulation results (1D)

28 Discontinuity-adaptive regularization (S. Z. Li) Simulation results (2D) Original imageEdge-preserving restoration Restoration without preserving edge

29 Discontinuity-adaptive regularization (Pierre Charbonnier et al ) Pierre Charbonnier et al impose following conditions on potential function φfor edge preserving regularization

30 Discontinuity-adaptive regularization (Pierre Charbonnier et al ) Possible choices for potential function φ

31 Other related techniques for edge-preserving regularization P. Perona and J. Malik, “Scale-space and edge detection using anisotropic diffusion,” IEEE Trans. Pattern Anal. Mach. Intell, July, Dropping observation model (w=1) when evaluating f

32 Other related techniques for edge-preserving regularization L.I. Rudin, S. Osher, E. Fatemi (1992): Nonlinear Total Variation Based Noise Removal Algorithms, Physica D, 60(1-4), Replace the quadratic potential function with absolute value function Quadratic function versus Absolute value function Energy