DATA CLUSTERING WITH KERNAL K-MEANS++ PROJECT OBJECTIVES o PROJECT GOAL  Experimentally demonstrate the application of Kernel K-Means to non-linearly.

Slides:

Advertisements

Similar presentations

Clustering k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein.

Advertisements

Unsupervised Learning Clustering K-Means. Recall: Key Components of Intelligent Agents Representation Language: Graph, Bayes Nets, Linear functions Inference.

Christoph F. Eick Questions and Topics Review Dec. 10, Compare AGNES /Hierarchical clustering with K-means; what are the main differences? 2. K-means.

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

Computer vision: models, learning and inference Chapter 8 Regression.

Supervised Learning Recap

UNSUPERVISED ANALYSIS GOAL A: FIND GROUPS OF GENES THAT HAVE CORRELATED EXPRESSION PROFILES. THESE GENES ARE BELIEVED TO BELONG TO THE SAME BIOLOGICAL.

Clustering and Dimensionality Reduction Brendan and Yifang April

Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui

Lecture 17: Supervised Learning Recap Machine Learning April 6, 2010.

© University of Minnesota Data Mining for the Discovery of Ocean Climate Indices 1 CSci 8980: Data Mining (Fall 2002) Vipin Kumar Army High Performance.

1 Abstract This paper presents a novel modification to the classical Competitive Learning (CL) by adding a dynamic branching mechanism to neural networks.

Speaker Adaptation for Vowel Classification

Reduced Support Vector Machine

Prénom Nom Document Analysis: Data Analysis and Clustering Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.

Dimension reduction : PCA and Clustering by Agnieszka S. Juncker

Lecture 4 Unsupervised Learning Clustering & Dimensionality Reduction

FACE RECOGNITION, EXPERIMENTS WITH RANDOM PROJECTION

Clustering In Large Graphs And Matrices Petros Drineas, Alan Frieze, Ravi Kannan, Santosh Vempala, V. Vinay Presented by Eric Anderson.

Unsupervised Learning

Bioinformatics Challenge  Learning in very high dimensions with very few samples  Acute leukemia dataset: 7129 # of gene vs. 72 samples  Colon cancer.

Support Vector Machines

Dimension reduction : PCA and Clustering by Agnieszka S. Juncker Part of the slides is adapted from Chris Workman.

Clustering with Bregman Divergences Arindam Banerjee, Srujana Merugu, Inderjit S. Dhillon, Joydeep Ghosh Presented by Rohit Gupta CSci 8980: Machine Learning.

Clustering & Dimensionality Reduction 273A Intro Machine Learning.

Difficulties with Nonlinear SVM for Large Problems  The nonlinear kernel is fully dense  Computational complexity depends on  Separating surface depends.

“A Comparison of Document Clustering Techniques” Michael Steinbach, George Karypis and Vipin Kumar (Technical Report, CSE, UMN, 2000) Mahashweta Das

8/10/ RBF NetworksM.W. Mak Radial Basis Function Networks 1. Introduction 2. Finding RBF Parameters 3. Decision Surface of RBF Networks 4. Comparison.

Dimensionality reduction Usman Roshan CS 675. Supervised dim reduction: Linear discriminant analysis Fisher linear discriminant: –Maximize ratio of difference.

Methods in Medical Image Analysis Statistics of Pattern Recognition: Classification and Clustering Some content provided by Milos Hauskrecht, University.

Presented By Wanchen Lu 2/25/2013

Efficient Model Selection for Support Vector Machines

Unsupervised Learning and Clustering k-means clustering Sum-of-Squared Errors Competitive Learning SOM Pre-processing and Post-processing techniques.

COMMON EVALUATION FINAL PROJECT Vira Oleksyuk ECE 8110: Introduction to machine Learning and Pattern Recognition.

Particle Filters for Shape Correspondence Presenter: Jingting Zeng.

A Regression Approach to Music Emotion Recognition Yi-Hsuan Yang, Yu-Ching Lin, Ya-Fan Su, and Homer H. Chen, Fellow, IEEE IEEE TRANSACTIONS ON AUDIO,

Today Ensemble Methods. Recap of the course. Classifier Fusion

Chapter 9 DTW and VQ Algorithm  9.1 Basic idea of DTW  9.2 DTW algorithm  9.3 Basic idea of VQ  9.4 LBG algorithm  9.5 Improvement of VQ.

UNSUPERVISED LEARNING David Kauchak CS 451 – Fall 2013.

Presenter : Lin, Shu-Han Authors : Jeen-Shing Wang, Jen-Chieh Chiang

Clustering I. 2 The Task Input: Collection of instances –No special class label attribute! Output: Clusters (Groups) of instances where members of a cluster.

RSVM: Reduced Support Vector Machines Y.-J. Lee & O. L. Mangasarian First SIAM International Conference on Data Mining Chicago, April 6, 2001 University.

Project 11: Determining the Intrinsic Dimensionality of a Distribution Okke Formsma, Nicolas Roussis and Per Løwenborg.

Computational Approaches for Biomarker Discovery SubbaLakshmiswetha Patchamatla.

1 Classification and Feature Selection Algorithms for Multi-class CGH data Jun Liu, Sanjay Ranka, Tamer Kahveci

Clustering Instructor: Max Welling ICS 178 Machine Learning & Data Mining.

Abdul Wahid, Xiaoying Gao, Peter Andreae

Machine Learning Queens College Lecture 7: Clustering.

Flat clustering approaches

A new initialization method for Fuzzy C-Means using Fuzzy Subtractive Clustering Thanh Le, Tom Altman University of Colorado Denver July 19, 2011.

Final Exam Review CS479/679 Pattern Recognition Dr. George Bebis 1.

Vector Quantization CAP5015 Fall 2005.

Data Mining Course 2007 Eric Postma Clustering. Overview Three approaches to clustering 1.Minimization of reconstruction error PCA, nlPCA, k-means clustering.

Iterative K-Means Algorithm Based on Fisher Discriminant UNIVERSITY OF JOENSUU DEPARTMENT OF COMPUTER SCIENCE JOENSUU, FINLAND Mantao Xu to be presented.

1 Learning Bias & Clustering Louis Oliphant CS based on slides by Burr H. Settles.

Given a set of data points as input Randomly assign each point to one of the k clusters Repeat until convergence – Calculate model of each of the k clusters.

Debrup Chakraborty Non Parametric Methods Pattern Recognition and Machine Learning.

Next, this study employed SVM to classify the emotion label for each EEG segment. The basic idea is to project input data onto a higher dimensional feature.

SUPERVISED AND UNSUPERVISED LEARNING Presentation by Ege Saygıner CENG 784.

1 Kernel Machines A relatively new learning methodology (1992) derived from statistical learning theory. Became famous when it gave accuracy comparable.

Gaussian Mixture Model classification of Multi-Color Fluorescence In Situ Hybridization (M-FISH) Images Amin Fazel 2006 Department of Computer Science.

Machine Learning Lecture 4: Unsupervised Learning (clustering) 1.

Accelerating K-Means Clustering with Parallel Implementations and GPU Computing Janki Bhimani Miriam Leeser Ningfang Mi

Machine Learning Basics

LINEAR AND NON-LINEAR CLASSIFICATION USING SVM and KERNELS

AIM: Clustering the Data together

KMeans Clustering on Hadoop Fall 2013 Elke A. Rundensteiner

Hairong Qi, Gonzalez Family Professor

Presentation transcript:

DATA CLUSTERING WITH KERNAL K-MEANS++ PROJECT OBJECTIVES o PROJECT GOAL  Experimentally demonstrate the application of Kernel K-Means to non-linearly clusterable data sets o ACADEMIC IMPORTANCE  Expand the application of the Kernel K-Means clustering algorithm to non-traditional uses Matt Strautmann, Dept. of Electrical and Computer Engineering BACKGROUND oWHAT IS K-MEANS CLUSTERING?  K-Means clustering aims to divide the dataset into clusters (“groups”) in which each data point belongs to the cluster with the nearest mean vector. oWHAT IS KERNAL K-MEANS?  Sum-of-squares algorithm  Two step process: data point assignment and update o WHAT IS THE PLUS PLUS INITIALIZATION SCHEME?  The first mean vector is a randomly selected data point  Each subsequent mean vector is created by evaluating randomly selected data points against a vector weighting probability APPROACH Evaluate standard K-Means (Soft++) against 4 datasets to form benchmark Hybridize Soft K-Means++ with Kernel K-Means to form Kernel K-Means++ Test Kernel K-Means++ on small size, small dimension Gaussian, large dimension Gaussian, and large size datasets Dr. Donald C. Wunsch II, Dept. of Electrical and Computer Engineering PROJECT DATASETS DISCUSSION Kernel K-Means++ was found to cluster the test datasets in a superior manner over Soft K-Means++ Kernel data-mapping was seen to solve the overlapping data sets by: Mapping the data before clustering to a higher- dimensional feature space using a nonlinear function Partitioning the points with linear separators in the new space Soft K-Means++ could not successfully cluster the Lung Cancer Dataset; results were for one cluster out of three successfully clustered Soft K-Means++ clustered the two dimension, two cluster Gaussian dataset with only one error out of the one thousand data points SOFT K-MEANS++ VS. KERNEL K-MEANS++ CONCLUDING REMARKS The initialization was seen to be the most important factor in the algorithm converging The “PLUS PLUS” cluster mean initialization was seen to improve the results Kernel assignment works better than the maximum responsibility calculation of Soft K-Means Kernel K-Means++ can handle small or large dimension datasets well; the increase of dimensionally seemed to be advantageous for the Lung Cancer Dataset (56 dimensions) over the lower clustering accuracy of the Iris Plant Dataset (4 dimensions) Kernel K-Means++ produced superior results to Soft K- Means++ when clustering the Lung Cancer Dataset and demonstrated recognition of all three clusters RESULTS COMPARISON Kernel K-Means++ clustering accuracy superior in all cases except the two dimensional, two cluster dataset. The clustering accuracy of the datasets increased by the following amounts: Iris Plant: 104% Lung Cancer: 38% 2D2k: -2.5% 8D5K: 30% FUTURE WORK Further improvement of the mean vector initialization is believed possible over the “PLUS PLUS” initialization Other options for the mean-squared error calculation for data point evaluation are possible The time analysis of the algorithm must be calculate The author would like to acknowledge the expertise of Dr. Rui Xu in advising this project. Acknowledgements 1.) Initial Mean Orientations 2.) Voronoi Diagram Generated by the Means (data points associated with nearest cluster mean) 3.) Cluster Centroid Becomes New Cluster Mean 4.) Step 2 and 3 Repeated until Convergence Iris Plant Dataset 2 Dimension, 2 Cluster Dataset (Gaussian 2D2K) 2 Dimension, 2 Cluster Dataset (Gaussian 2D2K) lans.ece.utexas.edu eleves.ens.fr Soft K-Means++ Clustering Accuracy Average (over ten runs) Standard Deviation of Accuracy Calculation (over ten runs) Variance of Accuracy Calculation (over ten runs) Iris Plant Dataset28.00%8.218%2.867% Lung Cancer Dataset43.75%-- 2D2K Gaussian Dataset99.00%-- 8D5K Gaussian Dataset58.50%2.082%0.043% Kernel K-Means++ Clustering Accuracy Average (over ten runs) Standard Deviation of Accuracy Calculation (over ten runs) Variance of Accuracy Calculation (over ten runs) Iris Plant Dataset57.00%5.009%2.238% Lung Cancer Dataset62.00%6.878%0.473% 2D2K Gaussian Dataset96.50%1.677%0.028% 8D5K Gaussian Dataset76.31%10.366%1.075%