1. Clustering and Segmentation
Supplement materials
Yu Hen Hu

2. What is “Clustering”?
What can we learn from these “unlabeled” data samples?
Structures: Some samples are closer to each other than other samples
The closeness between samples are determined using a “similarity measure”
The number of samples per unit volume is related to the concept of “density” or “distribution”
© 2003 by Yu Hen Hu

3. Clustering Problem Statement
Given a set of vectors {xk; 1  k  K}, find a set of M clustering centers {w(i); 1  i  c} such that each xk is assigned to a cluster, say, w(i*), according to a distance (distortion, similarity) measure d(xk, w(i)) such that the average distortion
is minimized.
I(xk,i) = 1 if x is assigned to cluster i with cluster center w(I); and = 0 otherwise -- indicator function.
© 2003 by Yu Hen Hu

4. k-means Clustering Algorithm
Initialization: Initial cluster center w(i); 1  i  c, D(–1)= 0, I(xk,i) = 0, 1  i  c, 1  k  K;
Repeat
(A) Assign cluster membership (Expectation step)
Evaluate d(xk, w(i));  i  c, 1  k  K
I(xk,i) = 1 if d(xk, w(i)) < d(xk, w(j)), j  i;
= 0; otherwise  k  K
(B) Evaluate distortion D:
(C) Update code words according to new assignment (Maximization)
(D) Check for convergence
if 1–D(Iter–1)/D(Iter) < e , then convergent = TRUE,
© 2003 by Yu Hen Hu

5. A Numerical Example x = {-1, -2,0,2,3,4}, W={2.1, 2.3}
Assign membership
2.1: {-1, -2, 0, 2}
2.3: {3, 4}
Distortion
D = (-1-2.1)2 + (-2-2.1)2 + (0-2.1)2 + (2-2.1)2 + (3-2.3)2 + (4-2.3)2
3. Update W to minimize distortion
W1 = ( )/4 = -.25
W2 = (3+4)/2 = 3.5
4. Reassign membership
-.25: {-1, -2, 0}
3.5: {2, 3, 4}
Update W:
w1 = (-1-2+0)/3 = -1
w2 = (2+3+4)/3 = 3.
Converged.
© 2003 by Yu Hen Hu

6. Kmeans Algorithm Demonstration
Clusterdemo.m
© 2003 by Yu Hen Hu

7. Image Segmentation Using Threshold
© 2003 by Yu Hen Hu

8. Image Segmentation Using Local Threshold
© 2003 by Yu Hen Hu

9. Color Image Segmentation
Perceptual distance in color space needs to be defined. That is, two close colors should be the ones that are perceptually similar.
© 2003 by Yu Hen Hu

10. Segmentation by Region Growing
Select seed points
Determine similarity criteria between two regions.
For each seed point or seed region, compute the similarity between it and points adjacent to its boundary. If the similarity exceeds a preset threshold, the neighboring points are added to the current region.
Iterate until stop criteria are satisfied.
A bottom-up approach
Clustering with connectivity constraint.
Clustering criteria pre-determined.
With local similarity measures, the results can be sensitive to the selection of seed points and stop criteria.
Region merge and region split may be required during post-processing phase.
© 2003 by Yu Hen Hu

11. Segmentation by Region Growing
© 2003 by Yu Hen Hu

12. Segmentation by Region Split and Merge
Split into a quad-tree representation
A region is split if a condition (predicate) is not satisfied.
May lead to over-fragmentation. Must also consider region merging.
Two adjacent regions should be merge if a condition (predicate) is satisfied. This can be accomplished by growing each region toward its neighbors.
© 2003 by Yu Hen Hu

13. Region Split-Merge Example
© 2003 by Yu Hen Hu

14. Segmentation by Morphological Watersheds
Image as a 3D plots: gray value (z-dim) vs. spatial coordinates (x-y plane).
Two types of points:
1. Watersheds: if a drop of water is landed on that point, it will fall into the same basin of water.
2. Divide lines (watershed lines): points equally likely to fall into more than one watersheds.
Region growing by flooding.
© 2003 by Yu Hen Hu

15. Segmentation by Morphological Watersheds
© 2003 by Yu Hen Hu

16. Segmentation by Morphological Watersheds
© 2003 by Yu Hen Hu