Spectral Clustering Scatter plot of a 2D data set K-means ClusteringSpectral Clustering U. von Luxburg. A tutorial on spectral clustering. Technical report, Max Planck Institute for Biological Cybernetics, Germany, 2006.

Conventional K-means Clustering 1) k initial "means" (in this case k=3) 2) associating every observation with the nearest mean. 3) The centroid of each of the k clusters becomes the new means. 4) Steps 2 and 3 are repeated until convergence has been reached. How to determine the k?

Spectral Graph Connected Groups Similarity Graph

Vertex Set Similarity Graph Weighted Adjacency Matrix Similarity Graph

 ε-neighborhood Graph  k-nearest neighbor Graphs  Fully connected graph ε K-nearest neighbor Gaussian Similarity Function ε-neighborhood

Spectral Graph Connected Groups Similarity Graph

Graph Laplacian w11w12w13w14w15 w21w22w23w24w25 w31w32w33w34w35 w41w42w43w44w45 w51w52w53w54w d d d d d D: degree matrix W: adjacency matrix L: Laplacian matrix

Example W: adjacency matrix D: degree matrix L: Laplacian matrix Similarity Graph

Property of Graph Laplacian Similarity Graph 1.L is symmetric and positive semi-definite. 2.The smallest eigenvalue of L is 0, the corresponding eigenvector is the constant one vector 1. 3.L has n non-negative, real-valued eigenvalues 0= λ 1 ≦ λ 2 ≦... ≦ λ n. L: Laplacian matrix

Eigenvalue and Eigenvector of Graph Laplacian Connected Component  Constant Eigenvector

Example Similarity Graph L: Laplacian matrix Two Connected Components  Double Zero Eigenvalue Eigenvectors: f1= [ ]’ f2= [ ]’

Example Similarity Graph W: adjacency matrix v1 v2 v3 v4 v5 v1v2v3v4v5u1u First Two Eigenvectors For all block diagonal matrices, the spectrum of L is given by the union of the spectra of Li

Spectral Clustering Similarity Graph First k Eigenvectors  New Clustering Space y1 y2 y3 y4 y5 u1u2 Use k-means clustering in the new space

Spectral Clustering Scatter plot of a 2D data set K-means ClusteringSpectral Clustering