1. Edge Preserving Spatially Varying Mixtures for Image Segmentation Giorgos Sfikas, Christophoros Nikou, Nikolaos Galatsanos (CVPR 2008) Presented by Lihan He ECE, Duke University Feb 23, 2009 by

2. Introduction Edge preserving spatially varying GMM Inference using MAP-EM Experimental results Conclusion Outline 2/15

3. Introduction 3/15 Image segmentation GMM: no prior knowledge is exploited  Adjacent pixels most likely belong to the same cluster;  Edge of objectives. SVGMM (spatially variant GMM): Clustering pixels or super pixels such that the same group has common characteristics (same objective, similar texture)  Spatial smoothness is imposed in the neighborhood of each pixel based on the Markov random field;  Without considering the edge of textures

4. Introduction 4/15 In this paper  Hierarchical Bayesian model;  Spatially varying GMM: mixing weights are different for different pixels;  Difference of mixing weights for two neighbored pixels follows a student-t distribution;  Heavy tailed student-t preserves edges of textures;  MAP-EM is used for model inference.

5. St-SVGMM Feature vector for each pixel: SVGMM: Each pixel has its own mixing weights Each pixel x n : weights indicator variables Likelihood: Prior: 5/15

6. St-SVGMM Prior for mixing weight π: d=2 d=1 d: neighborhood adjacency type d=1: horizontal d=2: vertical γ d (n): the set of neighbors of pixel n, with respect to the d th adjacency type K×D different student-t distributions are introduced, with hyperparameters Joint prior for π: 6/15

7. St-SVGMM The student-t distribution can be modeled by introducing the latent variable plays an important role: neighboring pixels n, k belong to the same cluster n, k are at the edge of two clusters n – edge location k (d) – adjacency type (horizontal or vertical) j – cluster index (edges of which cluster) 7/15

8. St-SVGMM Model summary 8/15

9. Inference MAP-EM algorithm for model inference. Complete log-likelihood Model parameters: E-step (update Z, U) 9/15

10. Inference M-step ( update ) = 10/15

11. Results U-variable maps j=1: sky j=2: roof & shadows j=3: building d=1: horizontal d=2: vertical n – edge location k (d) – adjacency type j – cluster index K=3 clusters Brighter regions represent lower values – edges. 11/15

12. Results Comparison on 300 images of the Berkeley image database Statistics on the Rand Index (RI) (measuring the consistency between the ground truth and the segmentation map); higher is better. Statistics on the boundary displacement error (BDE) (measuring error of boundary displacement with respect to the ground truth); lower is better. 12/15

13. Results Segmentation examples K=5 K=15 K=10 original image 13/15

14. Results K=5 K=15 K=10 original image 14/15

15. Conclusion 15/15  Proposed a GMM-based clustering algorithm for image segmentation;  Used smoothness prior to consider the adjacent pixels belonging to the same cluster;  Also captured the image edge structure (no smoothness enforced across segment boundaries);  All required parameters are estimated from the data (no requirement of empirical parameter selection).  Next: automatically estimating the number of components K.