A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan, and Brian Kulis.

Slides:

Advertisements

Similar presentations

Partitional Algorithms to Detect Complex Clusters

Advertisements

Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.

Fast SDP Relaxations of Graph Cut Clustering, Transduction, and Other Combinatorial Problems (JMLR 2006) Tijl De Bie and Nello Cristianini Presented by.

The Stability of a Good Clustering Marina Meila University of Washington

Carolina Galleguillos, Brian McFee, Serge Belongie, Gert Lanckriet Computer Science and Engineering Department Electrical and Computer Engineering Department.

Distance Metric Learning with Spectral Clustering By Sheil Kumar.

Normalized Cuts and Image Segmentation

Online Social Networks and Media. Graph partitioning The general problem – Input: a graph G=(V,E) edge (u,v) denotes similarity between u and v weighted.

Clustering II CMPUT 466/551 Nilanjan Ray. Mean-shift Clustering Will show slides from:

Semi-Definite Algorithm for Max-CUT Ran Berenfeld May 10,2005.

10/11/2001Random walks and spectral segmentation1 CSE 291 Fall 2001 Marina Meila and Jianbo Shi: Learning Segmentation by Random Walks/A Random Walks View.

Markov Chains Ali Jalali. Basic Definitions Assume s as states and s as happened states. For a 3 state Markov model, we construct a transition matrix.

One-Shot Multi-Set Non-rigid Feature-Spatial Matching

Determinants Bases, linear Indep., etc Gram-Schmidt Eigenvalue and Eigenvectors Misc

Lecture 21: Spectral Clustering

Normalized Cuts and Image Segmentation Jianbo Shi and Jitendra Malik, Presented by: Alireza Tavakkoli.

A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts

Normalized Cuts Demo Original Implementation from: Jianbo Shi Jitendra Malik Presented by: Joseph Djugash.

A Closed Form Solution to Natural Image Matting

Gaussian Information Bottleneck Gal Chechik Amir Globerson, Naftali Tishby, Yair Weiss.

Clustering In Large Graphs And Matrices Petros Drineas, Alan Frieze, Ravi Kannan, Santosh Vempala, V. Vinay Presented by Eric Anderson.

Lecture#6: segmentation Anat Levin Introduction to Computer Vision Class Fall 2009 Department of Computer Science and App math, Weizmann Institute of Science.

Lecture 18 Eigenvalue Problems II Shang-Hua Teng.

Orthogonal Matrices and Spectral Representation In Section 4.3 we saw that n  n matrix A was similar to a diagonal matrix if and only if it had n linearly.

Introduction The central problems of Linear Algebra are to study the properties of matrices and to investigate the solutions of systems of linear equations.

Dimensionality reduction Usman Roshan CS 675. Supervised dim reduction: Linear discriminant analysis Fisher linear discriminant: –Maximize ratio of difference.

Graph-based consensus clustering for class discovery from gene expression data Zhiwen Yum, Hau-San Wong and Hongqiang Wang Bioinformatics, 2007.

Image Segmentation Rob Atlas Nick Bridle Evan Radkoff.

8.1 Vector spaces A set of vector is said to form a linear vector space V Chapter 8 Matrices and vector spaces.

+ Review of Linear Algebra Optimization 1/14/10 Recitation Sivaraman Balakrishnan.

Presenter ： Kuang-Jui Hsu Date ： 2011/5/3(Tues.).

Segmentation using eigenvectors Papers: “Normalized Cuts and Image Segmentation”. Jianbo Shi and Jitendra Malik, IEEE, 2000 “Segmentation using eigenvectors:

Image Segmentation February 27, Implicit Scheme is considerably better with topological change. Transition from Active Contours: –contour v(t) 

Machine Learning Problems Unsupervised Learning – Clustering – Density estimation – Dimensionality Reduction Supervised Learning – Classification – Regression.

Low-Rank Kernel Learning with Bregman Matrix Divergences Brian Kulis, Matyas A. Sustik and Inderjit S. Dhillon Journal of Machine Learning Research 10.

Spectral Analysis based on the Adjacency Matrix of Network Data Leting Wu Fall 2009.

Andreas Papadopoulos - [DEXA 2015] Clustering Attributed Multi-graphs with Information Ranking 26th International.

Learning Spectral Clustering, With Application to Speech Separation F. R. Bach and M. I. Jordan, JMLR 2006.

K ERNEL - BASED W EIGHTED M ULTI - VIEW C LUSTERING Grigorios Tzortzis and Aristidis Likas Department of Computer Science, University of Ioannina, Greece.

Gaussian Mixture Models and Expectation-Maximization Algorithm.

A Convergent Solution to Tensor Subspace Learning.

About Me Swaroop Butala  MSCS – graduating in Dec 09  Specialization: Systems and Databases  Interests:  Learning new technologies  Application of.

1.7 Linear Independence. in R n is said to be linearly independent if has only the trivial solution. in R n is said to be linearly dependent if there.

 In the previews parts we have seen some kind of segmentation method.  In this lecture we will see graph cut, which is a another segmentation method.

Presented by Alon Levin

A Tutorial on Spectral Clustering Ulrike von Luxburg Max Planck Institute for Biological Cybernetics Statistics and Computing, Dec. 2007, Vol. 17, No.

K -means clustering via Principal Component Analysis (Chris Ding and Xiaofeng He, ICML 2004) 03 March 2011 Kwak, Namju 1.

Clustering Usman Roshan CS 675. Clustering Suppose we want to cluster n vectors in R d into two groups. Define C 1 and C 2 as the two groups. Our objective.

Chapter 61 Chapter 7 Review of Matrix Methods Including: Eigen Vectors, Eigen Values, Principle Components, Singular Value Decomposition.

Normalized Cuts and Image Segmentation Patrick Denis COSC 6121 York University Jianbo Shi and Jitendra Malik.

Document Clustering with Prior Knowledge Xiang Ji et al. Document Clustering with Prior Knowledge. SIGIR 2006 Presenter: Suhan Yu.

MAT 4725 Numerical Analysis Section 8.2 Orthogonal Polynomials and Least Squares Approximations (Part II)

CS246 Linear Algebra Review. A Brief Review of Linear Algebra Vector and a list of numbers Addition Scalar multiplication Dot product Dot product as a.

Clustering Clustering definition: Partition a given set of objects into M groups (clusters) such that the objects of each group are ‘similar’ and ‘different’

Introduction The central problems of Linear Algebra are to study the properties of matrices and to investigate the solutions of systems of linear equations.

Introduction The central problems of Linear Algebra are to study the properties of matrices and to investigate the solutions of systems of linear equations.

Review of Linear Algebra

Clustering Usman Roshan.

Matrices and vector spaces

CS6220: Data Mining Techniques

Degree and Eigenvector Centrality

Graph Based Multi-Modality Learning

Derivative of scalar forms

Unitialized, Globally Optimal, Graph-Based Rectilinear Shape Segmentation - The Opposing Metrics Method Computer Science Department – Carnegie Mellon University,

Spectral Clustering Eric Xing Lecture 8, August 13, 2010

3.3 Network-Centric Community Detection

Linear Algebra Lecture 32.

Performance Optimization

Clustering Usman Roshan CS 675.

Marios Mattheakis and Pavlos Protopapas

Presentation transcript:

A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan, and Brian Kulis

K means and Kernel K means

Weighted Kernel k means Matrix Form

Spectral Methods

Represented with Matrix L for Ncut Ratio assoc Ratio cut Norm assoc

Weighted Graph Cut Weighted association Weighted cut

Conclusion Spectral Methods are special case of Kernel K means

Solve the uniformed problem A standard result in linear algebra states that if we relax the trace maximizations such that Y is an arbitrary orthonormal matrix, then the optimal Y is of the form V k Q, where V k consists of the leading k eigenvectors of W 1/2 KW 1/2 and Q is an arbitrary k × k orthogonal matrix. As these eigenvectors are not indicator vectors, we must then perform postprocessing on the eigenvectors to obtain a discrete clustering of the point

From Eigen Vector to Cluster Indicator Normalized U with L 2 norm equal to 1 2 1

The Other Way Using k means to solve the graph cut problem: (random start points+ EM, local optimal). To make sure k mean converge, the kernel matrix must be positive definite. This is not true for arbitrary kernel matrix

The effect of the regularization a i is in a i is not in

Experiment results

Results (ratio association)

Results (normalized association)

Image Segmentation

Thank you. Any Question?