1. Sheeraza Bandi Sheeraza Bandi CS-635 Advanced Machine Learning Zahid Irfan 2001-03-0037 12 February 2004 Picture © Greg Martin, http://www.artofgregmartin.com

2. Introduction Clustering Principle Clustering Principle Similar things together and non-similar things in different (First Law of Geography !!) Similar things together and non-similar things in different (First Law of Geography !!) Applications Applications Divide and Rule Divide and Rule Economics, Image Processing, Pattern Recognition Economics, Image Processing, Pattern Recognition Document classification Document classification

3. K-Mean Clustering Basic Principle Basic Principle Make k-cluster centers Make k-cluster centers Compare item to the cluster Compare item to the cluster Put in the nearest cluster center Put in the nearest cluster center Reevaluate center metric Reevaluate center metric Repeat until the items are no more able to shift cluster centers Repeat until the items are no more able to shift cluster centers

4. Basic Leader Follower Algorithm Basic Idea Basic Idea Get a new item Get a new item Compare to clusters if minimum match < threshold add to cluster else make new cluster Compare to clusters if minimum match < threshold add to cluster else make new cluster Iterate for all the data Iterate for all the data

5. Self Organizing Maps Inspired by human brain maps Inspired by human brain maps Neural Networks simulating human brain Neural Networks simulating human brain Knowledge Acquisition Knowledge Acquisition Learning Process Learning Process Knowledge Storage Knowledge Storage inter-neuron synaptic weights inter-neuron synaptic weights Applications Applications Market Forecast ~ Simulation of Retina Market Forecast ~ Simulation of Retina

6. Self Organizing Maps Dimensionality Reduction Dimensionality Reduction 1-D or 2-D Maps of high dimensional data 1-D or 2-D Maps of high dimensional data Winner Takes All (WTA) Winner Takes All (WTA) And teaches others Updating Neighbors And teaches others Updating Neighbors Neighborhood Functions Neighborhood Functions k-Nearest Neighbor Function k-Nearest Neighbor Function Mexican Hat (under-development !!) Mexican Hat (under-development !!)

7. Performance Analysis K-Mean Clustering K-Mean Clustering Depends a lot on a priori knowledge Depends a lot on a priori knowledge Very Stable and Predictive Very Stable and Predictive Basic Leader Follower Principles Basic Leader Follower Principles Give and Take Give and Take Faster but unstable Faster but unstable Too much dependent on threshold Too much dependent on threshold Smaller the threshold higher the clusters & vice versa Smaller the threshold higher the clusters & vice versa

8. Performance Analysis …(continued) Self Organizing Map Self Organizing Map Stability and Convergence Assured Stability and Convergence Assured Principle of self-ordering Principle of self-ordering Losses !! Losses !! Too slow and way too many iterations for convergence !! Too slow and way too many iterations for convergence !! Computationally intensive Computationally intensive

9. Conclusions Nothing works well !! Nothing works well !! Gain one thing loose one.. Gain one thing loose one.. Ensemble clustering ?? Ensemble clustering ?? Use SOM and Basic Leader Follower to identify clusters and then use k-mean clustering to refine. Use SOM and Basic Leader Follower to identify clusters and then use k-mean clustering to refine.

10. Demonstration of Sheeraza Bandi Followed by Q&A