Unsupervised learning: Clustering Ata Kaban The University of Birmingham

Slides:

Advertisements

Similar presentations

SEEM Tutorial 4 – Clustering. 2 What is Cluster Analysis?  Finding groups of objects such that the objects in a group will be similar (or.

Advertisements

Albert Gatt Corpora and Statistical Methods Lecture 13.

AEB 37 / AE 802 Marketing Research Methods Week 7

2004/05/03 Clustering 1 Clustering (Part One) Ku-Yaw Chang Assistant Professor, Department of Computer Science and Information.

Clustering… in General In vector space, clusters are vectors found within  of a cluster vector, with different techniques for determining the cluster.

Today Unsupervised Learning Clustering K-means. EE3J2 Data Mining Lecture 18 K-means and Agglomerative Algorithms Ali Al-Shahib.

Dimension reduction : PCA and Clustering by Agnieszka S. Juncker

Dimension reduction : PCA and Clustering Slides by Agnieszka Juncker and Chris Workman.

Slide 1 EE3J2 Data Mining Lecture 16 Unsupervised Learning Ali Al-Shahib.

Clustering. 2 Outline  Introduction  K-means clustering  Hierarchical clustering: COBWEB.

Dimension reduction : PCA and Clustering Christopher Workman Center for Biological Sequence Analysis DTU.

Clustering. 2 Outline  Introduction  K-means clustering  Hierarchical clustering: COBWEB.

Introduction to Hierarchical Clustering Analysis Pengyu Hong 09/16/2005.

Adapted by Doug Downey from Machine Learning EECS 349, Bryan Pardo Machine Learning Clustering.

Ranking by Odds Ratio A Probability Model Approach let be a Boolean random variable: document d is relevant to query q otherwise Consider document d as.

Microarray analysis 2 Golan Yona. 2) Analysis of co-expression Search for similarly expressed genes experiment1 experiment2 experiment3 ……….. Gene i:

Dimension reduction : PCA and Clustering by Agnieszka S. Juncker Part of the slides is adapted from Chris Workman.

Tutorial 8 Clustering 1. General Methods –Unsupervised Clustering Hierarchical clustering K-means clustering Expression data –GEO –UCSC –ArrayExpress.

Ulf Schmitz, Pattern recognition - Clustering1 Bioinformatics Pattern recognition - Clustering Ulf Schmitz

Clustering. What is clustering? Grouping similar objects together and keeping dissimilar objects apart. In Information Retrieval, the cluster hypothesis.

Slide 1 EE3J2 Data Mining EE3J2 Data Mining Lecture 11: Clustering Martin Russell.

Computer Vision - A Modern Approach Set: Segmentation Slides by D.A. Forsyth Segmentation and Grouping Motivation: not information is evidence Obtain a.

Evaluating Performance for Data Mining Techniques

COMP53311 Clustering Prepared by Raymond Wong Some parts of this notes are borrowed from LW Chan ’ s notes Presented by Raymond Wong

START OF DAY 8 Reading: Chap. 14. Midterm Go over questions General issues only Specific issues: visit with me Regrading may make your grade go up OR.

CLUSTERING. Overview Definition of Clustering Existing clustering methods Clustering examples.

Clustering I. 2 The Task Input: Collection of instances –No special class label attribute! Output: Clusters (Groups) of instances where members of a cluster.

Chapter 11 Statistical Techniques. Data Warehouse and Data Mining Chapter 11 2 Chapter Objectives  Understand when linear regression is an appropriate.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

Prepared by: Mahmoud Rafeek Al-Farra

By Timofey Shulepov Clustering Algorithms. Clustering - main features  Clustering – a data mining technique  Def.: Classification of objects into sets.

K-Means Algorithm Each cluster is represented by the mean value of the objects in the cluster Input: set of objects (n), no of clusters (k) Output:

CURE: EFFICIENT CLUSTERING ALGORITHM FOR LARGE DATASETS VULAVALA VAMSHI PRIYA.

CHAPTER 1: Introduction. 2 Why “Learn”? Machine learning is programming computers to optimize a performance criterion using example data or past experience.

Clustering Instructor: Max Welling ICS 178 Machine Learning & Data Mining.

V. Clustering 인공지능 연구실 이승희 Text: Text mining Page:82-93.

Hierarchical Clustering Produces a set of nested clusters organized as a hierarchical tree Can be visualized as a dendrogram – A tree like diagram that.

Machine Learning Queens College Lecture 7: Clustering.

Slide 1 EE3J2 Data Mining Lecture 18 K-means and Agglomerative Algorithms.

Compiled By: Raj Gaurang Tiwari Assistant Professor SRMGPC, Lucknow Unsupervised Learning.

Flat clustering approaches

Definition Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from (or unrelated to)

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 32: HIERARCHICAL CLUSTERING Objectives: Unsupervised.

© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Data Mining: Cluster Analysis This lecture node is modified based on Lecture Notes for Chapter.

1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 28 Nov 9, 2005 Nanjing University of Science & Technology.

Clustering Algorithms Sunida Ratanothayanon. What is Clustering?

Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach, Kumar Introduction.

Unsupervised Learning: Clustering

Unsupervised Learning: Clustering

Clustering CSC 600: Data Mining Class 21.

Chapter 15 – Cluster Analysis

Machine Learning Clustering: K-means Supervised Learning

Hierarchical Clustering

Canadian Bioinformatics Workshops

Data Clustering Michael J. Watts

John Nicholas Owen Sarah Smith

Hierarchical and Ensemble Clustering

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

The BIRCH Algorithm Davitkov Miroslav, 2011/3116

Hierarchical and Ensemble Clustering

CSCI N317 Computation for Scientific Applications Unit Weka

Dimension reduction : PCA and Clustering

Data Mining – Chapter 4 Cluster Analysis Part 2

Clustering Wei Wang.

Text Categorization Berlin Chen 2003 Reference:

Clustering Techniques

SEEM4630 Tutorial 3 – Clustering.

Presentation transcript:

Unsupervised learning: Clustering Ata Kaban The University of Birmingham

The Clustering Problem Unsupervi sed Learning Data (input)‘Interesting structure’ (output) -Should contain essential traits -discard unessential details -provide a compact summary the data -interpretable for humans -… Objective function that expresses our notion of interestingness for this data

Here is some data…

Formalising Data points x n n=1,2,… N Assume K clusters Binary indicator variables z kn associated with each data point and cluster: 1 if x n is in cluster k and 0 otherwise Define a measure of cluster compactness as the total distance from the cluster mean:

Cluster quality objective (the smaller the better): Two sets of parameters - the cluster mean values m k and the cluster allocation indicator variables z kn Minimise the above objective over each set of variables while holding one set fixed  This is exactly what the K-means algorithm is doing! (can you prove it?)

–Pseudo-code of K-means algorithm: Begin initialize  1,  2, …,  K (randomly selected) do classify n samples according to nearest  i recompute  i until no change in  i return  1,  2, …,  K End

Other forms of clustering Many times, clusters are not disjoint, but a cluster may have subclusters, in turn having sub- subclusters.  Hierarchical clustering

Given any two samples x and x’, they will be grouped together at some level, and if they are grouped a level k, they remain grouped for all higher levels Hierarchical clustering  tree representation called dendrogram

The similarity values may help to determine if the grouping are natural or forced, but if they are evenly distributed no information can be gained Another representation is based on set, e.g., on the Venn diagrams

Hierarchical clustering can be divided in agglomerative and divisive. Agglomerative (bottom up, clumping): start with n singleton cluster and form the sequence by merging clusters Divisive (top down, splitting): start with all of the samples in one cluster and form the sequence by successively splitting clusters

Agglomerative hierarchical clustering The procedure terminates when the specified number of cluster has been obtained, and returns the cluster as sets of points, rather than the mean or a representative vector for each cluster

Application to image segmentation

Application to clustering face images Cluster centres = face prototypes

The problem of the number of clusters Typically, the number of clusters is known. When it’s not, that is a hard problem called model selection. There are several ways of proceed. A common approach is to repeat the clustering with K=1, K=2, K=3, etc.

What did we learn today? Data clustering K-means algorithm in detail How K-means can get stuck and how to take care of that The outline of Hierarchical clustering methods