Segmentation Kyongil Yoon

Segmentation Obtain a compact representation of what is helpful (in the image) No comprehensive theory of segmentation Human vision: Grouping and Gestalt –Proximity, similarity, common fate, common region, parallelism, closure, symmetry, continuity, familiar configuration

Segmentation by clustering Partitioning vs. grouping Applications –Background subtraction –Shot boundary detection Image segmentation by clustering pixels –Using simple clustering Agglomerative clustering (clustering by merging) Divisive clustering (clustering by splitting) –K-means –Using graph-theoretic clustering Affinity measure Normalized cut –cut(A,B)/assoc(A,V) + cut(A,B)/assoc(B,V)

K-Means Choose k data points to act as cluster centers Until the cluster centers are unchanged Allocate each data point to cluster whose center is nearest Now ensure that every cluster has at least one data point; possible techniques for doing this include supplying empty clusters with a point chosen at random from points far from their cluster center. Replace the cluster centers with the mean of the elements in their clusters. end

Graph Eigenvectors Construct an affinity matrix Compute the eigenvalues and eigenvectors of the affinity matrix Until there are sufficient clusters Take the eigenvector corresponding to the largest unprocessed eigenvalue; zero all components corresponding to elements that have already been clustered, and threshold the remaining components to determine which element belongs to this cluster, choosing a threshold by clustering the components, or using a threshold fixed in advance. If all elements have been accounted for, there are sufficient clusters end

Segmentation by fitting a model To assert that pixels belong together to conform to some model –Large scale explicit model Hough transform –Three problems: What is the line? Which points belong to which line? How many lines? –Point space line space x*cos(t) + y*sin(t) + r = 0, (t, r) line space Half-infinite cylinder –Quantization errors, difficulties with noise Fitting lines, fitting curves –Least square –Total least square

Segmentation by fitting a model(2) Two big problems –Robustness: what if one data point is FAR, and all others fill well? –Missing data: which point is noise and which point is not? Robustness –Outliers: Improve the model either by giving the noise “heavier tails” or allowing an explicit outlier model –M-estimators Assuming that somewhere in the collection of process close to our model is the real process, and it just happens to be the one that makes the estimator produce the worst possible estimates

Segmentation by fitting a model(3) RANSAC (RAMdom SAmple Consensus) –Searching for a random sample that leads to a fit on which many of the data points agree –Determine n : the smallest # of points required k : the # of iterations required t : the threshold used to identify a point that fits well d : the # of nearby points required Until k iterations have occurred Pick n sample points Fit to that set of n points For each data point outside the sample Test distance; if the distance < t, it is close If there are d or more points close, this is a good fit. Refit the line using all these points End Use the best fit from this collection, using the fitting error as a criterion –Need to choose 3 parameters # of samples required (n) Telling whether a point is close (t) # of points that must agree (d)

Segmentation E. Borenstein and S. Ullman. Class-specific, top-down segmentation, In Proc. 7th Europ. Conf. Comput. Vision, May 2002 J. Freixenet, X. Munoz, D. Raba, J. Marti, and X. Cufi, Yet another survey on image segmentation: region and boundary information integration, University of Girona, Institute of Information and Applications, ECCV 2002, LNCS 2352, pp , 2002 Harwood, D., Chang, S., Davis, L.S., Interpreting aerial photographs by segmentation and search, IUW(87), pp D. C. Alexander and B. F. Buxton. Statistical modeling of colour data, International Journal of Computer Vision, 44(2): , September Friedman, N. and Russell, S Image segmentation in video sequences: A probabilistic approach, In Proceedings 13. Conf. on Uncertainty in Articial Intelligence Ahmed Elgammal, David Harwood, Larry Davis, Non-parametric model for background subtraction, 6th European Conference on Computer Vision. Dublin, Ireland, June/July 2000 T. H. Chalidabhongse, K. Kim, D. Harwood and L. Davis, A perturbation method for evaluating background subtraction algorithms, Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, Nice, France, Oct , 2003 (in conjunction with ICCV'03) J. Shi and J. Malik. Normalized cuts and image segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'97), pages , 1997 J. Shi and J. Malik, "Motion segmentation and tracking using normalized cuts", in International Conference on Computer Vision, January 1998, Bombay, India E. Sharon, A. Brandt and R. Basri, Fast Multiscale Image Segmentation, Proceedings IEEE Conference on Computer Vision and Pattern Recognition, pp , 2000 C. Stauffer and W.E.L. Grimson. Adaptive background mixture models for real-time tracking. In CVPR99, pages II: , 1999 Weiss, Y., Segmentation using eigenvectors: A unifying view, Proc. 7th Int. Conf. Computer Vision, 1999,