Lecture 21 Clustering (2).

Slides:

Advertisements

Similar presentations

Advertisements

Document Clustering Carl Staelin. Lecture 7Information Retrieval and Digital LibrariesPage 2 Motivation It is hard to rapidly understand a big bucket.

Aaker, Kumar, Day Seventh Edition Instructor’s Presentation Slides

SEEM Tutorial 4 – Clustering. 2 What is Cluster Analysis?  Finding groups of objects such that the objects in a group will be similar (or.

Hierarchical Clustering

1 CSE 980: Data Mining Lecture 16: Hierarchical Clustering.

Hierarchical Clustering, DBSCAN The EM Algorithm

Component Analysis (Review)

Data Mining Cluster Analysis: Basic Concepts and Algorithms

Linear Discriminant Analysis

Image Indexing and Retrieval using Moment Invariants Imran Ahmad School of Computer Science University of Windsor – Canada.

IT 433 Data Warehousing and Data Mining Hierarchical Clustering Assist.Prof.Songül Albayrak Yıldız Technical University Computer Engineering Department.

6-1 ©2006 Raj Jain Clustering Techniques  Goal: Partition into groups so the members of a group are as similar as possible and different.

Today Unsupervised Learning Clustering K-means. EE3J2 Data Mining Lecture 18 K-means and Agglomerative Algorithms Ali Al-Shahib.

Cluster Analysis.  What is Cluster Analysis?  Types of Data in Cluster Analysis  A Categorization of Major Clustering Methods  Partitioning Methods.

Prénom Nom Document Analysis: Data Analysis and Clustering Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.

Intro. ANN & Fuzzy Systems Lecture 21 Clustering (2)

Clustering Ram Akella Lecture 6 February 23, & 280I University of California Berkeley Silicon Valley Center/SC.

Chapter 3: Cluster Analysis  3.1 Basic Concepts of Clustering  3.2 Partitioning Methods  3.3 Hierarchical Methods The Principle Agglomerative.

METU Informatics Institute Min 720 Pattern Classification with Bio-Medical Applications PART 2: Statistical Pattern Classification: Optimal Classification.

CSE554AlignmentSlide 1 CSE 554 Lecture 5: Alignment Fall 2011.

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory 1 CLUSTERS Prof. George Papadourakis,

ECE 8443 – Pattern Recognition LECTURE 10: HETEROSCEDASTIC LINEAR DISCRIMINANT ANALYSIS AND INDEPENDENT COMPONENT ANALYSIS Objectives: Generalization of.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

Clustering Algorithms Presented by Michael Smaili CS 157B Spring

1Ellen L. Walker Category Recognition Associating information extracted from images with categories (classes) of objects Requires prior knowledge about.

A split-and-merge framework for 2D shape summarization D. Gerogiannis, C. Nikou and A. Likas Department of Computer Science, University of Ioannina, Greece.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

CURE: EFFICIENT CLUSTERING ALGORITHM FOR LARGE DATASETS VULAVALA VAMSHI PRIYA.

Hierarchical Clustering Produces a set of nested clusters organized as a hierarchical tree Can be visualized as a dendrogram – A tree like diagram that.

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 32: HIERARCHICAL CLUSTERING Objectives: Unsupervised.

© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Data Mining: Cluster Analysis This lecture node is modified based on Lecture Notes for Chapter.

Intro. ANN & Fuzzy Systems Lecture 38 Mixture of Experts Neural Network.

Example Apply hierarchical clustering with d min to below data where c=3. Nearest neighbor clustering d min d max will form elongated clusters!

Clustering Usman Roshan CS 675. Clustering Suppose we want to cluster n vectors in R d into two groups. Define C 1 and C 2 as the two groups. Our objective.

Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach, Kumar Introduction.

Part 3: Estimation of Parameters. Estimation of Parameters Most of the time, we have random samples but not the densities given. If the parametric form.

Clustering (1) Clustering Similarity measure Hierarchical clustering

Data Mining: Basic Cluster Analysis

PREDICT 422: Practical Machine Learning

Clustering CSC 600: Data Mining Class 21.

Machine Learning Clustering: K-means Supervised Learning

Clustering Usman Roshan.

LECTURE 10: DISCRIMINANT ANALYSIS

Clustering and Segmentation

CH 5: Multivariate Methods

Data Clustering Michael J. Watts

Computer Vision Lecture 12: Image Segmentation II

Clustering (3) Center-based algorithms Fuzzy k-means

Lecture 25 Radial Basis Network (II)

REMOTE SENSING Multispectral Image Classification

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Data Mining – Chapter 4 Cluster Analysis Part 2

Feature space tansformation methods

Generally Discriminant Analysis

Fixed, Random and Mixed effects

LECTURE 09: DISCRIMINANT ANALYSIS

Text Categorization Berlin Chen 2003 Reference:

Lecture 8. Learning (V): Perceptron Learning

SEEM4630 Tutorial 3 – Clustering.

EM Algorithm and its Applications

LECTURE 22: HIERARCHICAL CLUSTERING

Hierarchical Clustering

Data Mining Cluster Analysis: Basic Concepts and Algorithms

Presentation transcript:

Lecture 21 Clustering (2)

Outline Similarity (Distance) Measures Distortion Criteria Scattering Criterion Hierarchical Clustering and other clustering methods (C) 2001 by Yu Hen Hu

Distance Measure Distance Measure – What does it mean “Similar"? Norm: Mahalanobis distance: d(x,y) = |x – y|TSxy1|x – y| Angle: d(x,y) = xTy/(|x|•|y|) Binary and symbolic features (x, y contains 0, 1 only): Tanimoto coefficient: (C) 2001 by Yu Hen Hu

Clustering Criteria Is the current clustering assignment good enough? Most popular one is the mean-square error distortion measure Other distortion measures can also be used: (C) 2001 by Yu Hen Hu

Scatter Matrics Scatter matrices are defined in the context of analysis of variance in statistics. They are used in linear discriminant analysis. However, they can also be used to gauge the fitness of a particular clustering assignment. Mean vector for i-th cluster: Total mean vector Scatter matrix for i-th cluster: Within-cluster scatter matrix Between-cluster scatter matrix (C) 2001 by Yu Hen Hu

Scattering Criteria Total scatter matrix: Note that the total scatter matrix is independent of the assignment I(xk,i). But … SW and SB both depend on I(xk,i)! Desired clustering property SW small SB large How to gauge Sw is small or SB is large? There are several ways. Tr. Sw (trace of SW): Let be the eigenvalue decomposition of SW, then (C) 2001 by Yu Hen Hu

Cluster Separating Measure (CSM) Similar to scattering criteria. csm = (mi-mj)/(i+j) The larger its value, the more separable the two clusters. Assume underlying data distribution is Gaussian. (C) 2001 by Yu Hen Hu

Hierarchical Clustering Merge Method: Initially, each xk is a cluster. During each iteration, nearest pair of distinct clusters are merged until the number of clusters is reduced to 1. How to measure distance between two clusters: dmin(C(i), C(j)) = min. d(x,y); x  C(i), y  C(j)  leads to minimum spanning tree dmax(C(i), C(j)) = max. d(x,y); x  C(i), y  C(j) davg(C(i), C(j)) = dmean(C(i), C(j)) = mi– mj (C) 2001 by Yu Hen Hu

Hierarchical Clustering (II) Split method: Initially, only one cluster. Iteratively, a cluster is splited into two or more clusters, until the total number of clusters reaches a predefined goal. The scattering criterion can be used to decide how to split a given cluster into two or more clusters. Another way is to perform a m-way clustering, using, say, k-means algorithm to split a cluster into m smaller clusters. (C) 2001 by Yu Hen Hu