1. EECS 274 Computer Vision Segmentation by Clustering II

2. Segmentation and grouping Watershed Mean shift filtering and segmentation Reading: FP Chapter 14, S Chapter 5

3. Watershed segmentation Visualize an image in 3D: spatial coordinates and gray levels. In such a topographic interpretation, there are 3 types of points Points belonging to a regional minimum (basin) Points at which a drop of water would fall to a single minimum. (  The catchment basin or watershed of that minimum.) Points at which a drop of water would be equally likely to fall to more than one minimum. (  The divide lines or watershed lines.) Watershed lines

4. Watershed segmentation The objective is to find watershed lines (via thresholding) The idea is simple: Suppose that a hole is punched in each regional minimum and that the entire topography is flooded from below by letting water rise through the holes at a uniform rate. When rising water in distinct catchment basins is about the merge, a dam is built to prevent merging. These dam boundaries correspond to the watershed lines.

5. Watershed segmentation

6. Start with all pixels with the lowest possible value –These form the basis for initial watersheds For each intensity level k: –For each group of pixels of intensity k If adjacent to exactly one existing region, add these pixels to that region Else if adjacent to more than one existing regions, mark as boundary Else start a new region –Implemented with breadth-first search Watershed segmentation

7. Watershed algorithm might be used on the gradient image instead of the original image.

8. Watershed segmentation Due to noise and other local irregularities of the gradient, over segmentation might occur.

9. Watershed segmentation A solution is to limit the number of regional minima. Use markers to specify the only allowed regional minima.

10. Watershed segmentation A solution is to limit the number of regional minima. Use markers to specify the only allowed regional minima. (For example, gray-level values might be used as a marker.) Often leads to over-segmentation Useful for interactive segmentation

11. Mean shift Non-parametric model Efficiently finding peaks in the high-dimensional data distribution without computing the complete function explicitly See –K. Fukunaga and L. Hostetler, “The estimation of the gradient of a density function with applications in pattern recognition”, IEEE Trans. Information Theory, 1975 –Y. Cheng, “Mean shift, mode seeking, and clustering”, PAMI 1995

12. Non-parametric density function Smooth the data by convolving it with a fixed kernel of width h Known as kernel density estimation or Parzen window technique Instead of using brute force approach, mean shift uses a variant of multiple restart gradient descent technique is the kernel function or Parzen window

13. 1D mean shift Starting at some initial guess for a local maximum, y k, which can be Random input data pint x i, mean shift computes the gradient of density estimate f(x) at y k and takes uphill step

14. Mean shift algorithm Compute the gradient of the density estimate f(x) at y k Rewrite the gradient function Mean shift is the difference between the weighted mean of neighbors x i around x and the current value of x

15. Modes The mode y k at iteration k is replaced by locally weighted mean Comaniciu and Meer show that this algorithm converges to a local maximum of f(x) under some reasonably weak condition on kernel k(r) Two kind of kernels: The corresponding derivative kernel g(r) are a unit ball and another is Gaussian, respectively Epanechnikov kernel Gaussian kernel

16. Finding modes Simplest way –start a separate mean shift mode estimate y at every input point x i –and iterate for a fixed number of steps until the magnitude is below a threshold Faster way –randomly subsample the input points x i –and find the nearest mode sample evolution path to the remaining points

17. Mean shift segmentation Better results can be obtained with a combination of kernels, i.e., joint domain of color, r, and location, s Clustering of {x i } is achieved by placing a seed y i at each x i in n-d feature space (e.g., n=5) Regions containing less than M pixels are discarded

18. Mean shift filtering and segmentation

19. Mean Shift Segmentation http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html

20. Mean shift algorithm Mean Shift Algorithm 1.Choose a search window size. 2.Choose the initial location of the search window. 3.Compute the mean location (centroid of the data) in the search window. 4.Center the search window at the mean location computed in Step 3. 5.Repeat Steps 3 and 4 until convergence. The mean shift algorithm seeks the “mode” or point of highest density of a data distribution:

21. Mean Shift Segmentation Algorithm 1.Convert the image into tokens (via color, gradients, texture measures etc). 2.Choose initial search window locations uniformly in the data. 3.Compute the mean shift window location for each initial position. 4.Merge windows that end up on the same “peak” or mode. 5.The data these merged windows traversed are clustered together. *Image From: Dorin Comaniciu and Peter Meer, Distribution Free Decomposition of Multivariate Data, Pattern Analysis & Applications (1999)2:22–30 Mean shift segmentation

22. http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html