1. CS 376b Introduction to Computer Vision 04 / 04 / 2008 Instructor: Michael Eckmann

2. Michael Eckmann - Skidmore College - CS 376b - Spring 2008 Today’s Topics Comments/Questions Segmentation algorithms –clustering K-means histogram based algorithm –region growing single linkage centroid linkage

3. Segmentation Iterative K-means clustering (from Shapiro and Stockman)‏ D(x i, m k (ic)) = || x i – m k || 2

4. Segmentation Overall desire is to minimize the least squares error measure (from page 282 on the board). Can think of each mean as a centroid in n-dimensions It's a greedy algorithm Results clearly depend on chosen K Results depend on initially chosen means –standard solution (according to page below) is try various different initial means and then pick your favorite result Let's look here for some additional comments on K- means clustering http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/kmeans.html

5. Segmentation from Shapiro and Stockman figure 10.4 example with K=6, clustering characteristic is RGB color

6. Michael Eckmann - Skidmore College - CS 376b - Spring 2008 Segmentation Some other clustering algorithms are based on using histograms to cluster Recall the Otsu method for thresholding –this is a simple case of clustering into 2 clusters based on greyvalue

7. Michael Eckmann - Skidmore College - CS 376b - Spring 2008 Segmentation Ohlander, Price and Reddy define a recursive histogram based clustering algorithm. –an initial image-sized binary mask with all pixels on is created –do something similar to Otsu on the histogram of the image pixels that correspond to the on-pixels in the mask –then, perform connected components in each of the clusters –create an image-sized mask for each connected component ---with on-pixels for the pixels in the connected component and off-pixels elsewhere –for each mask repeat the above steps until one cluster per mask

8. Segmentation figure 10.8 from Shapiro and Stockman

9. Michael Eckmann - Skidmore College - CS 376b - Spring 2008 Segmentation Ohta, Kanade and Sakai suggested histograms not on each R G and B channel but instead on (R+G+B)/3 (R-B)/2 (2G-R-B)/4 The Ohlander et.al. and this improvement work well for segmenting color images with clear single color regions (e.g. scenes with man made objects) but do not work all that well for natural scenes. We'll talk about an algorithm by Shi and Malik that can segment based on color, texture etc. probably get to this one tomorrow.

10. Michael Eckmann - Skidmore College - CS 376b - Spring 2008 Segmentation Region growing –begin at one position in the image –and grow each region until the pixel being compared is different enough to not add to the region Single linkage region growing –consider each pixel as a node in a graph –edges connect neighboring pixels that are similar –image segments are the sets of pixels all belonging to same connected component –example on board

11. Michael Eckmann - Skidmore College - CS 376b - Spring 2008 Segmentation Haralick and Shapiro (1985)‏ –considered a region to be a set of connected pixels with same mean and variance –mean and scatter of a region is defined (on board)‏ –when a new pixel is added into a region, the new mean and scatter are given by (equations on board)‏ why is this a good thing?

12. Michael Eckmann - Skidmore College - CS 376b - Spring 2008 Segmentation Haralick and Shapiro (1985) region growing algorithm –a T statistic is also defined which is used to determine if a test pixel is part of the same distribution of a region (should be added into the region) up to some confidence level –if T is too high, y (the test pixel) is likely to not be part of the same population of pixels in the region –if T is too high for y and all neighboring regions of y, then y starts as the first pixel in a new region –besides being close to the mean of a region's distribution (the T test above), could also require that a neighboring pixel have a close enough value to y (which is the main criteria in single-linkage region growers)‏

13. Michael Eckmann - Skidmore College - CS 376b - Spring 2008 Segmentation Haralick and Shapiro (1985) region growing algorithm –T is considered for some fraction called alpha (e.g. 1/20)‏ this alpha is the probability that a T with N-1 d.o.f. exceeds T N-1 (alpha)‏ –T N-1 (alpha) for a given N-1 and alpha, is a defined quantity if our computed T does exceed this T N-1 (alpha) then it is determined that the pixel is significantly different than the population of the region if the pixel really did belong to the same population as the region, then the probability that the test we did was incorrect is alpha (e.g. 1/20)

14. Michael Eckmann - Skidmore College - CS 376b - Spring 2008 Segmentation Haralick and Shapiro (1985) region growing algorithm –the d.o.f. N-1 T N-1 (alpha) is higher for smaller N-1 and lower for larger N-1 the d.o.f. is determined from the number of pixels in the region --- the more pixels already determined as part of the population, the larger the d.o.f. (and vice-versa)‏ what does this mean for us? –the larger number of pixels a region has, the closer a pixel's value has to be to that mean to join the region (and the smaller a region is...)‏

15. Michael Eckmann - Skidmore College - CS 376b - Spring 2008 Segmentation Shi and Malik –developed a new graph partitioning method –Let G=(V,E) where V is the set of points in the measurement space and E is the set of weighted edges where the weight represents the similarity between the two vertices –Goal partition the graph into disjoint subsets of V so that similarity within sets is high and similarity between sets is low –To partition a graph into two disjoint graphs G a =(V a,E a ) and G b =(V b,E b ) whereV = V a union V b we remove any edges from E that connect v's in V a with v's in V b

16. Michael Eckmann - Skidmore College - CS 376b - Spring 2008 Segmentation Shi and Malik –To partition a graph into two disjoint graphs G a =(V a,E a ) and G b =(V b,E b ) whereV = V a union V b we remove any edges from E that connect v's in V a with v's in V b –the degree of dissimilarity between the two disjoint sets of V can be computed as the sum of the weights of the edges removed --- this sum of the weights of the edges removed is called a cut –So, for segmentation, we can look for a minimum cut in the graph and continue to do so until regions are uniform enough –problem with minimum cut --- favors small sets of isolated vertices --- not useful for segmenting into large uniform (in some measurement) regions

17. Michael Eckmann - Skidmore College - CS 376b - Spring 2008 Segmentation Shi and Malik –problem with minimum cut --- favors small sets of isolated vertices --- not useful for segmenting into large uniform (in some measurement) regions –proposed normalized cut (Ncut)