1. Spectral Clustering Scatter plot of a 2D data set K-means ClusteringSpectral Clustering U. von Luxburg. A tutorial on spectral clustering. Technical report, Max Planck Institute for Biological Cybernetics, Germany, 2006.

2. Conventional K-means Clustering 1) k initial "means" (in this case k=3) 2) associating every observation with the nearest mean. 3) The centroid of each of the k clusters becomes the new means. 4) Steps 2 and 3 are repeated until convergence has been reached. How to determine the k?

3. Spectral Graph Connected Groups Similarity Graph

4. Vertex Set Similarity Graph Weighted Adjacency Matrix Similarity Graph

5.  ε-neighborhood Graph  k-nearest neighbor Graphs  Fully connected graph ε K-nearest neighbor Gaussian Similarity Function ε-neighborhood

6. Spectral Graph Connected Groups Similarity Graph

7. Graph Laplacian w11w12w13w14w15 w21w22w23w24w25 w31w32w33w34w35 w41w42w43w44w45 w51w52w53w54w55 12345 1 2 3 4 5 d10000 0d2000 00d300 000d40 0000d5 12345 1 2 3 4 5 D: degree matrix W: adjacency matrix L: Laplacian matrix

8. Example 2 3 1 4 01100 10100 11000 00001 00010 W: adjacency matrix 5 20000 02000 00200 00010 00001 D: degree matrix L: Laplacian matrix 2 00 2 00 200 0001 000 1 Similarity Graph

9. Property of Graph Laplacian 2 3 1 4 5 Similarity Graph 1.L is symmetric and positive semi-definite. 2.The smallest eigenvalue of L is 0, the corresponding eigenvector is the constant one vector 1. 3.L has n non-negative, real-valued eigenvalues 0= λ 1 ≦ λ 2 ≦... ≦ λ n. L: Laplacian matrix 2 00 2 00 200 0001 000 1

10. Eigenvalue and Eigenvector of Graph Laplacian Connected Component  Constant Eigenvector

11. Example 2 3 1 4 5 Similarity Graph L: Laplacian matrix 2 00 2 00 200 0001 000 1 Two Connected Components  Double Zero Eigenvalue Eigenvectors: f1= [1 1 1 0 0]’ f2= [0 0 0 1 1]’

12. Example 2 3 1 4 5 Similarity Graph 01100 10100 11000 00001 00010 W: adjacency matrix v1 v2 v3 v4 v5 v1v2v3v4v5u1u2 10 10 10 01 01 First Two Eigenvectors For all block diagonal matrices, the spectrum of L is given by the union of the spectra of Li

13. Spectral Clustering 2 3 1 4 5 Similarity Graph First k Eigenvectors  New Clustering Space 10 10 10 01 01 y1 y2 y3 y4 y5 u1u2 Use k-means clustering in the new space

14. Spectral Clustering Scatter plot of a 2D data set K-means ClusteringSpectral Clustering