Dan Phelleg, Andrew Moore Carnegie Mellon University

Slides:

Advertisements

Similar presentations

Advertisements

6/10/20141 Top-Down Clustering Method Based On TV-Tree Zbigniew W. Ras.

Christian Lauterbach COMP 770, 2/16/2009. Overview  Acceleration structures  Spatial hierarchies  Object hierarchies  Interactive Ray Tracing techniques.

Hierarchical Clustering. Produces a set of nested clusters organized as a hierarchical tree Can be visualized as a dendrogram – A tree-like diagram that.

Augmenting Data Structures Advanced Algorithms & Data Structures Lecture Theme 07 – Part I Prof. Dr. Th. Ottmann Summer Semester 2006.

Machine Learning and Data Mining Clustering

Spatial Indexing I Point Access Methods. PAMs Point Access Methods Multidimensional Hashing: Grid File Exponential growth of the directory Hierarchical.

Speech Recognition Training Continuous Density HMMs Lecture Based on:

10/6/2015Nikos Hourdakis, MSc Thesis1 Design and Evaluation of Clustering Approaches for Large Document Collections, The “BIC-Means” Method Nikolaos Hourdakis.

Clustering… in General In vector space, clusters are vectors found within  of a cluster vector, with different techniques for determining the cluster.

Daniel Blackburn Load Balancing in Distributed N-Body Simulations.

Scalable Data Mining The Auton Lab, Carnegie Mellon University Brigham Anderson, Andrew Moore, Dan Pelleg, Alex Gray, Bob Nichols, Andy.

Spatial Indexing I Point Access Methods.

Switch to Top-down Top-down or move-to-nearest Partition documents into ‘k’ clusters Two variants “Hard” (0/1) assignment of documents to clusters “soft”

Techniques and Data Structures for Efficient Multimedia Similarity Search.

Fast Temporal State-Splitting for HMM Model Selection and Learning Sajid Siddiqi Geoffrey Gordon Andrew Moore.

Advanced Multimedia Text Clustering Tamara Berg. Reminder - Classification Given some labeled training documents Determine the best label for a test (query)

What is Cluster Analysis?

K-means Clustering. What is clustering? Why would we want to cluster? How would you determine clusters? How can you do this efficiently?

Multiple Object Class Detection with a Generative Model K. Mikolajczyk, B. Leibe and B. Schiele Carolina Galleguillos.

Spatial Indexing I Point Access Methods. Spatial Indexing Point Access Methods (PAMs) vs Spatial Access Methods (SAMs) PAM: index only point data Hierarchical.

Clustering Ram Akella Lecture 6 February 23, & 280I University of California Berkeley Silicon Valley Center/SC.

Spatial Indexing I Point Access Methods. Spatial Indexing Point Access Methods (PAMs) vs Spatial Access Methods (SAMs) PAM: index only point data Hierarchical.

Review Rong Jin. Comparison of Different Classification Models  The goal of all classifiers Predicating class label y for an input x Estimate p(y|x)

Birch: An efficient data clustering method for very large databases

Chapter 3: Cluster Analysis  3.1 Basic Concepts of Clustering  3.2 Partitioning Methods  3.3 Hierarchical Methods The Principle Agglomerative.

1/16 Final project: Web Page Classification By: Xiaodong Wang Yanhua Wang Haitang Wang University of Cincinnati.

Evaluating Performance for Data Mining Techniques

Module 04: Algorithms Topic 07: Instance-Based Learning

Bayesian Networks 4 th, December 2009 Presented by Kwak, Nam-ju The slides are based on, 2nd ed., written by Ian H. Witten & Eibe Frank. Images and Materials.

Unsupervised Learning. CS583, Bing Liu, UIC 2 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate.

DATA MINING CLUSTERING K-Means.

Bayesian Hierarchical Clustering Paper by K. Heller and Z. Ghahramani ICML 2005 Presented by HAO-WEI, YEH.

The X-Tree An Index Structure for High Dimensional Data Stefan Berchtold, Daniel A Keim, Hans Peter Kriegel Institute of Computer Science Munich, Germany.

Combining Regression Trees and Radial Basis Function Networks paper by: M. Orr, J. Hallam, K. Takezawa, A. Murray, S. Ninomiya, M. Oide, T. Leonard presentation.

Clustering Methods K- means. K-means Algorithm Assume that K=3 and initially the points are assigned to clusters as follows. C 1 ={x 1,x 2,x 3 }, C 2.

Clustering Algorithms k-means Hierarchic Agglomerative Clustering (HAC) …. BIRCH Association Rule Hypergraph Partitioning (ARHP) Categorical clustering.

1 Motivation Web query is usually two or three words long. –Prone to ambiguity –Example “keyboard” –Input device of computer –Musical instruments How can.

Data Mining Practical Machine Learning Tools and Techniques Chapter 4: Algorithms: The Basic Methods Section 4.7: Instance-Based Learning Rodney Nielsen.

Randomized Algorithms for Bayesian Hierarchical Clustering

Chapter 11 Statistical Techniques. Data Warehouse and Data Mining Chapter 11 2 Chapter Objectives  Understand when linear regression is an appropriate.

Marketing Research Aaker, Kumar, Day and Leone Tenth Edition Instructor’s Presentation Slides 1.

BIRCH: An Efficient Data Clustering Method for Very Large Databases Tian Zhang, Raghu Ramakrishnan, Miron Livny University of Wisconsin-Maciison Presented.

Data Mining Practical Machine Learning Tools and Techniques Chapter 4: Algorithms: The Basic Methods Section 4.8: Clustering Rodney Nielsen Many of these.

CS 8751 ML & KDDData Clustering1 Clustering Unsupervised learning Generating “classes” Distance/similarity measures Agglomerative methods Divisive methods.

Data Mining Practical Machine Learning Tools and Techniques By I. H. Witten, E. Frank and M. A. Hall 6.8: Clustering Rodney Nielsen Many / most of these.

Compiled By: Raj Gaurang Tiwari Assistant Professor SRMGPC, Lucknow Unsupervised Learning.

1 Machine Learning Lecture 9: Clustering Moshe Koppel Slides adapted from Raymond J. Mooney.

AQWA Adaptive Query-Workload-Aware Partitioning of Big Spatial Data Dimosthenis Stefanidis Stelios Nikolaou.

Data Mining By Farzana Forhad CS 157B. Agenda Decision Tree and ID3 Rough Set Theory Clustering.

Clustering (1) Chapter 7. Outline Introduction Clustering Strategies The Curse of Dimensionality Hierarchical k-means.

Example Apply hierarchical clustering with d min to below data where c=3. Nearest neighbor clustering d min d max will form elongated clusters!

Hierarchical clustering approaches for high-throughput data Colin Dewey BMI/CS 576 Fall 2015.

Introduction to Data Mining Clustering & Classification Reference: Tan et al: Introduction to data mining. Some slides are adopted from Tan et al.

Linear Models & Clustering Presented by Kwak, Nam-ju 1.

Estimating the Number of Clusters (k) Clustering error cannot be used as a criterion for deciding on the number of clusters. Selection Approaches: Use.

Estimating the Number of Clusters (k) Selection Approaches: Use a Criterion to select among the solutions for several values of k (kmeans or GMMs are used)

Rodney Nielsen Many of these slides were adapted from: I. H. Witten, E. Frank and M. A. Hall Data Science Algorithms: The Basic Methods Clustering WFH:

Data Science Practical Machine Learning Tools and Techniques 6.8: Clustering Rodney Nielsen Many / most of these slides were adapted from: I. H. Witten,

Data Science Algorithms: The Basic Methods

10701 / Machine Learning.

The BIRCH Algorithm Davitkov Miroslav, 2011/3116

Birch presented by : Bahare hajihashemi Atefeh Rahimi

Text Categorization Berlin Chen 2003 Reference:

Clustering The process of grouping samples so that the samples are similar within each group.

Data Mining CSCI 307, Spring 2019 Lecture 24

Machine Learning and Data Mining Clustering

Data Mining CSCI 307, Spring 2019 Lecture 23

Presentation transcript:

X-means: Extending K-means with Efficient Estimation of the Number of Clusters Dan Phelleg, Andrew Moore Carnegie Mellon University Published: ICML 2000 Presentation by: Payam Refaeilzadeh

Problems with K-means Need to know K Searching for K is expensive Even K-means with fixed-K scales poorly Need to calculate the distance from each point to each centroid to find new cluster assignments

Remedies Forward search for the appropriate value of k in a given range Recursively split each cluster and use BIC score to decide if we should keep each split Use kd-trees to accelerate individual rounds of K-means

Splitting Use local BIC score to decide on keeping a split Use global BIC score to decide which K to output at the end

BIC (Bayesian Information Criterion) Adjusted Log-likelihood of the model. The likelihood that the data is “explained by” the clusters according to the spherical-Gaussian assumption of k-means

Kd-trees Points to be clustered are put into a binary hierarchical structure Each node represents a subset of points and stores The minimal hyper-rectangle enclosing all points in the subset The vector-sum of all the points in the subset The number of points in the subset

Using kd-trees For each centroid store a counter containing the vector sum of all the points belonging to it and the number of points Update the above by scanning the kd-tree only once Start with the root node and all centroids As you walk down the tree centroids start to get black-listed (when the points in that node could not possibly belong to a centroid) When only one centroid remains, the counter for that centroid can be updated using the statistics stored in the node At the end of the scan we have enough info to recalculate the centroid coordinates

Results