Biointelligence Laboratory, Seoul National University Ch 6. Markov Random Fields 6.10 Coupled Markov Random Fields 6.11 Compound Gauss-Markov Random Fields Adaptive Cooperative Systems, Martin Beckerman, 1997. Summarized by J.-W. Ha Biointelligence Laboratory, Seoul National University http://bi.snu.ac.kr/

(C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ Contents 6.10 Coupled Markov Random Fields 6.10.1 The Line Process 6.10.2 Coupled Markov Random Fields and Integration 6.11 Compound Gauss-Markov Random Fields (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

(C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ The Line Process Coupled Pixel Process + Line Process Line process : discontinuity-preserving, boundary-strengthening The Line Process Second Markov random fields Employ the Gibbs-Markov equivalence to produce a simple in terms of clique potentials. Is defined over a dual lattice and encodes discontinuity information Be visualized as a 2-D array of 2N2 line elements, one horizontal and one vertical line for each pixel in the primary pixel lattice. (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

(C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ The Line Process the first order relations The Suspension of smoothing at surface boundary Line element is ON → second term = 0 Prevents the unwanted phase transitions by suspending Small islands of pixels (6.120) (6.121) (6.122) (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

(C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ The Line Process Off-on system of line elements Six elements : empty, isolated singleton, corner, straight line, tee, crossing Clique potentials Generalized Ising Hamilton A simple way of strengthening boundaries The lowest energies : All ON or All Off (6.123) (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Coupled Markov Random Fields and Integration The goal Image reconstruction : remove noise, blur, distortions and artifacts while preserving boundaries Segmenting : reconstruct and identify the individual surfaces Removing noise and artifacts while preserving and strengthening boundaries. Integration Added task : combine 2D image from many perceptual domains It can be done by coupled MRF (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Coupled Markov Random Fields and Integration Underlying Assumptions 3D physical scenes are composed of surfaces Input : 2D projection of 3D scene Output : image description in terms of surfaces, attendant discontinuities, stable physical properties J.J. Gibson : invariant properties of surfaces were perceived directly from the spatiotemporal patterns of light reaching the observer as contained in the ambient optic array. Barrow et al. replace the concept of direct perception with a computational theory for the reconstruction of surfaces (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Coupled Markov Random Fields and Integration Computational theory of perception Intrinsic images Arrays of elements that are in precise registrations with respect to the original images Each array describes an invariant physical characteristic of the underlying physical surfaces and also contains information regarding the discontinuities associated with that property. Ex : distance(depth), albedo, local surface orientation, illumination Constrains to reconstruct Problems of recovering the properties of 3D surfaces from 2D images Needed to restrict the many possible surface reconstructions to physically meaningful one (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Coupled Markov Random Fields and Integration Coupling Model Simultaneously extract surface and boundary information Consider the integration of information contained in the various intrinsic images One for the pixel process and one for the line process (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Coupled Markov Random Fields and Integration An Example of the integration of data from multiple moduels (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Coupled Markov Random Fields and Integration ICM algorithms for Coupled MRF Initialize the pixel lattice by choosing the value at each site that maximizes the likehood. Initialize all dual lattices using the initial pixel values according to Eq.(6.120) Select a primary dual lattice pair Select a lattice site in the manner of a raster scan Replace the pixel grey values with the mode for the neighborhood. Go back to step 4 and continue the raster scan Initialize the dual lattice using the updated pixel values Select a dual lattice site in the manner of a raster scan If flipping the spin lowers the energy, then flip the spin; otherwise do not flip. Go back to step 8 and continue the raster scan Repeat steps 3 to 10 for the next pair of coupled lattices Iterate five or six times, repeating steps 3 to 10 for the three lattice pairs (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Coupled Markov Random Fields and Integration Compound Gauss-MRF These models extend the method of Geman and Geman to non-compact, continuous-valued range spaces Reconstruction Given the degraded observed image g, reconstruct f Appending a line or structure process (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Coupled Markov Random Fields and Integration A model consisting of a continuous valued pixel process plus a discrete valued line process (C) 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/