K-Means Algorithm Each cluster is represented by the mean value of the objects in the cluster Input: set of objects (n), no of clusters (k) Output: set of k clusters Algo Randomly select k samples & mark them a initial cluster Repeat Assign/ reassign in sample to any given cluster to which it is most similar depending upon the mean of the cluster Update the cluster’s mean until No Change.

K-Means (graph) Step1: Form k centroids, randomly Step2: Calculate distance between centroids and each object Use Euclidean’s law do determine min distance: d(A,B) = (x 2 -x 1 ) 2 + (y 2 -y 1 ) 2 Step3: Assign objects based on min distance to k clusters Step4: Calculate centroid of each cluster using C =(x 1 +x 2 +…x n, y 1 +y 2 +…y n ) n n Go to step 2. Repeat until no change in centroids.

K-Mediod (PAM) Also called Partitioning Around Mediods. Step1:choose k mediods Step2:assign all points to closest mediod Step3:form distance matrix for each cluster and choose the next best mediod. i.e., the point closest to all other points in cluster go to step2. Repeat until no change in any mediods

What are Hierarchical Methods? Groups data objects into a tree of clusters Classified as Agglomerative (Bottom-up) Divisive (Top-Bottom) Once a merge or split decision is made it cannot be backtracked

Types of hierarchical clustering Agglomerative (Bottom-up) AGNES Places each object into a cluster and merges atomic clusters into larger clusters They differ in the definition of intercluster similarity Divisive: (Top-Bottom) DIANA All objects are initially in one cluster Subdivides the cluster into smaller and smaller pieces, until each object forms a cluster of its own or satisfies some termination condition In both of the above methods the termination condition is the number of clusters

Dendogram Level 0 Level 1 Level 2 Level 3 Level 4

Measures of Distance Minimum distance – Nearest Neighbor- single linkage –minimum spanning tree Maximum distance – Farthest neighbor clustering algorithm – complete linkage Mean distance - avoids outlier sensitivity problem Average distance : can handle categorical as well as numeric data

Euclidean Distance

Agglomerative Algorithm Step1:Make each object as a cluster Step2:Calculate the Euclidean distance from every point to every other point. i.e., construct a Distance Matrix Step3:Identify two clusters with shortest distance. Merge them Go to Step 2 Repeat until all objects are in one cluster

Agglomerative Algorithm Approaches Single Link: Quite simple Not very efficient Suffers from chain effect Complete Link More compact than those found using the single link technique Average Link

Simple Example ItemEACBD E A C B D 33630

Another Example Find single link technique to find clusters in the given database. XY

Plot given data

Identify two nearest clusters

Repeat process until all objects in same cluster

Average link Average distance matrix

Construct a distance matrix

Divisive Clustering All items are initially placed in one cluster The clusters are repeatedly split in two until all items are in their own cluster A B C D E

Difficulties in Hierarchical Clustering Difficulties regarding the selection of merge or split points This decision is critical because the further merge or split decisions are based on the newly formed clusters Method does not scale well So hierarchical methods are integrated with other clustering techniques to form multiple-phase clustering

Types of hierarchical clustering techniques BIRCH-Balanced Iterative Reducing and Clustering using hierarchies ROCK: Robust clustering with links, explores the concept of links CHAMELEON: hierarchical clustering algorithm using dynamic modeling

Outlier Analysis Outliers are data objects, which are different from or inconsistent with the remaining set of data Outliers can be caused because of Measurement or execution error Result of inherent data variability Can be used in fraud detection Outlier detection and analysis is referred to as outlier mining.

Applications of outlier mining Fraud detection Customized marketing for identifying the spending behavior of customers with extremely low or high incomes. Medical analysis for finding unusual responses to various medical treatments.

What is outlier mining? Given a set of n data points or objects and k, the expected number of outliers find the top k objects that are dissimilar, exceptional or inconsistent with respect to remaining data There are two subproblems Define what data can be considered as inconsistent in a given data set Method to mine the outliers

Methods of outlier detection Statistical approach distance-based approach Density-based local outlier approach Deviation-based approach

Statistical Distribution Identifies outliers with respect to a discordancy test Discordancy test examines a working hypothesis and an alternative hypothesis It verifies whether an object oi, is significantly large in relation to the distribution F. This helps in accepting the working hypothesis or rejecting it (alternative distribution) Inherent alternative distribution Mixture alternative distribution Slippage alternative distribution

Procedures for detecting outliers Block procedures: All suspect objects are treated as outliers or all of then are accepted as consistent Consecutive procedures: object that is least likely to be an outlier is tested first. If it is found to be an outlier then all of the more extreme values are also considered as outliers. Else the next most extreme object is tested and so on

Questions in Clustering