Presentation is loading. Please wait.

Presentation is loading. Please wait.

Machine Learning Clustering: K-means Supervised Learning

Published byShanon Green Modified over 6 years ago

Similar presentations

Presentation on theme: "Machine Learning Clustering: K-means Supervised Learning"— Presentation transcript:

1 Machine Learning Clustering: K-means Supervised Learning
Classification and Regression K-Nearest Neighbor Classification Fisher’s Criteria & Linear Discriminant Analysis Perceptron: Linearly Separable Multilayer Perceptron & EBP & Deep Learning, RBF Network Support Vector Machine Ensemble Learning: Voting, Boosting(Adaboost) Unsupervised Learning Dimensionality Reduction: Principle Component Analysis Independent Component Analysis Clustering: K-means Semi-supervised Learning & Reinforcement Learning

2 Classification vs. Clustering
Classification (known categories) Clustering (creation of new categories)

3 Clustering Clustering: the process of grouping a set of objects into classes of similar objects high intra-class similarity low inter-class similarity It is the commonest form of unsupervised learning Unsupervised learning: learning from unlabelled data, as opposed to supervised data where a classification of examples is given A common and important task that finds many applications in science and engineering Group genes that perform the same function Group individuals that have similar political view Categorize documents of similar topics Identify similar objects from photos

4 Clustering examples People Images Language Species

5 Issues for clustering “Groupness": What is a natural grouping among these objects? “Similarity/Distance": What makes objects related? “Representation": How do we represent objects? Vectors? Do we normalise? “Parameters": How many clusters? Fixed a priori? Data-driven? “Algorithms": Partitioning the data? Hierarchical algorithm? Formal foundation and convergence

6 Groupness: What is a natural grouping among these objects?

7 How do we define similarity?

8 What properties should a distance measure have?
Symmetry Self-similarity Separation Triangular inequality

9 Distance measures

10 An example

11 Hamming distance Manhattan distance is called Hamming distance when all features are binary E.g. Gene expression levels under 17 conditions (1-High, 0-Low)

12 Similarity measures: Correlation coefficient

13 Edit distance: a generic technique for measuring similarity
To measure the similarity between two objects, transform one of the objects into the other, and measure how much effort it took. The measure of effort becomes the distance measure.

14 Clustering algorithms

15 Hierarchical Clustering algorithms

16 Hierarchical Clustering
Bottom-up agglomerative clustering Starts with each object in a separate cluster Then repeatedly joins the closest pair of clusters Until there is only one cluster Top-down divisive clustering Starts with all data in a single cluster Considers every possible way to divide the cluster into two and chooses the best division Recurse on both sides

17 Distance between clusters
We can define the distance between two clusters as Single-link nearest neighbor: their closest members Complete-link farthest neighbor: their farthest members Centroid: the centrois (centers of gravity) of the two clusters Average: the average of all cross-cluster pairs

18 Single-link

19 Complete-link

20 Dendograms

21 Partitioning Algorithms
Partitioning method: construct a partition of n objects into a set of K clusters Given: a set of objects and the number K Find: a partition of K clusters that optimizes the chosen partitioning criterion Globally optimal: exhaustively enumerate all partitions Effective heuristic methods: K-means algorithm

22 K-means Clustering

23 K-means Clustering

24 K-means Clustering Algorithm

25 K-means Algorithm Decide on a value for k
Initialize the k cluster centers randomly Decide the class memberships of the N objects by assigning them to the nearest cluster centroids (aka the center of gravity or mean) Re-estimate the k cluster centers, by assuming the memberships found above are correct If none of the N objects changed membership in the last iteration, exit. Otherwise go back to 3.

26 K-means

27 K-means

28 K-means

29 K-means

30 K-means: seed selection

31 K-means: number of clusters

32 K-means: criteria for cluster quality

33 K-means: purity

Download ppt "Machine Learning Clustering: K-means Supervised Learning"

Similar presentations

COMPUTER AIDED DIAGNOSIS: CLASSIFICATION Prof. Yasser Mostafa Kadah – www.k-space.org.

COMPUTER AIDED DIAGNOSIS: CLASSIFICATION Prof. Yasser Mostafa Kadah –
Unsupervised Learning Clustering K-Means. Recall: Key Components of Intelligent Agents Representation Language: Graph, Bayes Nets, Linear functions Inference.

Unsupervised Learning Clustering K-Means. Recall: Key Components of Intelligent Agents Representation Language: Graph, Bayes Nets, Linear functions Inference.
Albert Gatt Corpora and Statistical Methods Lecture 13.

Albert Gatt Corpora and Statistical Methods Lecture 13.
Clustering Paolo Ferragina Dipartimento di Informatica Università di Pisa This is a mix of slides taken from several presentations, plus my touch !

Clustering Paolo Ferragina Dipartimento di Informatica Università di Pisa This is a mix of slides taken from several presentations, plus my touch !
Introduction to Bioinformatics

Introduction to Bioinformatics
1 Text Clustering. 2 Clustering Partition unlabeled examples into disjoint subsets of clusters, such that: –Examples within a cluster are very similar.

1 Text Clustering. 2 Clustering Partition unlabeled examples into disjoint subsets of clusters, such that: –Examples within a cluster are very similar.
Semi-Supervised Clustering Jieping Ye Department of Computer Science and Engineering Arizona State University

Semi-Supervised Clustering Jieping Ye Department of Computer Science and Engineering Arizona State University
Ranking by Odds Ratio A Probability Model Approach let be a Boolean random variable: document d is relevant to query q otherwise Consider document d as.

Ranking by Odds Ratio A Probability Model Approach let be a Boolean random variable: document d is relevant to query q otherwise Consider document d as.
Revision (Part II) Ke Chen COMP24111 Machine Learning Revision slides are going to summarise all you have learnt from Part II, which should be helpful.

Revision (Part II) Ke Chen COMP24111 Machine Learning Revision slides are going to summarise all you have learnt from Part II, which should be helpful.
INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, 2004 Lecture Slides for.

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.
Clustering Unsupervised learning Generating “classes”

Clustering Unsupervised learning Generating “classes”
CSC 4510 – Machine Learning Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website: www.csc.villanova.edu/~map/4510/

CSC 4510 – Machine Learning Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website:
Data mining and machine learning A brief introduction.

Data mining and machine learning A brief introduction.
Text Clustering.

Text Clustering.
Clustering Supervised vs. Unsupervised Learning Examples of clustering in Web IR Characteristics of clustering Clustering algorithms Cluster Labeling 1.

Clustering Supervised vs. Unsupervised Learning Examples of clustering in Web IR Characteristics of clustering Clustering algorithms Cluster Labeling 1.
Basic Machine Learning: Clustering CS 315 – Web Search and Data Mining 1.

Basic Machine Learning: Clustering CS 315 – Web Search and Data Mining 1.
1 Motivation Web query is usually two or three words long. –Prone to ambiguity –Example “keyboard” –Input device of computer –Musical instruments How can.

1 Motivation Web query is usually two or three words long. –Prone to ambiguity –Example “keyboard” –Input device of computer –Musical instruments How can.
Chapter 11 Statistical Techniques. Data Warehouse and Data Mining Chapter 11 2 Chapter Objectives  Understand when linear regression is an appropriate.

Chapter 11 Statistical Techniques. Data Warehouse and Data Mining Chapter 11 2 Chapter Objectives  Understand when linear regression is an appropriate.
Data Science and Big Data Analytics Chap 4: Advanced Analytical Theory and Methods: Clustering Charles Tappert Seidenberg School of CSIS, Pace University.

Data Science and Big Data Analytics Chap 4: Advanced Analytical Theory and Methods: Clustering Charles Tappert Seidenberg School of CSIS, Pace University.
1Ellen L. Walker Category Recognition Associating information extracted from images with categories (classes) of objects Requires prior knowledge about.

1Ellen L. Walker Category Recognition Associating information extracted from images with categories (classes) of objects Requires prior knowledge about.

Similar presentations

Ads by Google