1. Unit 5 : Cluster Analysis May 26, 2016 Data Mining: Concepts and Techniques1

2. May 26, 2016Data Mining: Concepts and Techniques2 Chapter 7. Cluster Analysis 1. What is Cluster Analysis? 2. Types of Data in Cluster Analysis 3. A Categorization of Major Clustering Methods 4. Partitioning Methods 5. Hierarchical Methods 6. Density-Based Methods 7. Grid-Based Methods 8. Model-Based Methods 9. Clustering High-Dimensional Data 10. Constraint-Based Clustering 11. Outlier Analysis 12. Summary

3. May 26, 2016Data Mining: Concepts and Techniques3 What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Finding similarities between data according to the characteristics found in the data and grouping similar data objects into clusters Unsupervised learning: no predefined classes Typical applications As a stand-alone tool to get insight into data distribution As a preprocessing step for other algorithms

4. Clustering is also called data segmentation in some applications because clustering partitions large data sets into groups according to their similarity. Clustering can also be used for outlier detection. May 26, 2016Data Mining: Concepts and Techniques4

5. May 26, 2016Data Mining: Concepts and Techniques5 Clustering: Rich Applications and Multidisciplinary Efforts Pattern Recognition Spatial Data Analysis Create thematic maps in GIS by clustering feature spaces Detect spatial clusters or for other spatial mining tasks Image Processing Economic Science (especially market research) WWW Document classification Cluster Weblog data to discover groups of similar access patterns

6. May 26, 2016Data Mining: Concepts and Techniques6 Examples of Clustering Applications Marketing: Help marketers discover distinct groups in their customer bases, and then use this knowledge to develop targeted marketing programs Land use: Identification of areas of similar land use in an earth observation database Insurance: Identifying groups of motor insurance policy holders with a high average claim cost City-planning: Identifying groups of houses according to their house type, value, and geographical location Earth-quake studies: Observed earth quake epicenters should be clustered along continent faults

7. 7 Quality: What Is Good Clustering? A good clustering method will produce high quality clusters with high intra-class similarity low inter-class similarity The quality of a clustering result depends on both the similarity measure used by the method and its implementation The quality of a clustering method is also measured by its ability to discover some or all of the hidden patterns The quality of the cluster also depends on the definition and representation of cluster chosen.

8. May 26, 2016Data Mining: Concepts and Techniques8 Measure the Quality of Clustering Dissimilarity/Similarity metric: Similarity is expressed in terms of a distance function, typically metric: d(i, j) There is a separate “quality” function that measures the “goodness” of a cluster. The definitions of distance functions are usually very different for interval-scaled, boolean, categorical, ordinal ratio, and vector variables. Weights should be associated with different variables based on applications and data semantics. It is hard to define “similar enough” or “good enough” the answer is typically highly subjective.

9. May 26, 2016Data Mining: Concepts and Techniques9 Requirements of Clustering in Data Mining Scalability: can work well on both the small data sets containing fewer than 200 data objects; as well as on a large database which may contain millions of objects; Ability to deal with different types of attributes: e.g binary, categorial(nominal), ordinal data or mixture of these data types. Ability to handle dynamic data Discovery of clusters with arbitrary shape : many clustering algorithms determines cluster based on euclidean or manhattan distance measures. That tends to find spherical shape. It is important to develop algorithms that can detect clusters of arbitarary shape.

10. Minimal requirements for domain knowledge to determine input parameters: cluster analysis sometimes require parameters from users. The clustering results can be quite sensitive to input parameters and are often hard to determine. This burdens users, as well as makes the quality of clustering difficult to control. Able to deal with noise and outliers Incremental clustering and insensitivity to the order of input records. High dimensionality Incorporation of user-specified constraints Interpretability and usability 10

11. May 26, 2016Data Mining: Concepts and Techniques11 Chapter 7. Cluster Analysis 1. What is Cluster Analysis? 2. Types of Data in Cluster Analysis 3. A Categorization of Major Clustering Methods 4. Partitioning Methods 5. Hierarchical Methods 6. Density-Based Methods

12. 12 Data Structures Clustering algorithm typically operate on either of the following two data structure. a) Data matrix ( object by variable structure) Represents n objects (e. persons) and variables (also called measurements or attributes) (e.g age,height) Structure is in relational form or n by p (n objects * p variables) (two modes)

13. b) Dissimilarity matrix ( object by object structure) Stores collection of proximities that are available for all pairs of n objects. Represented by an n-by-n table. (one mode) d(i,j) is the measured difference or dissimilarity between objects I and j. d(i,j) is non negative close to 0 when objects are highly similar and becomes large the more they differ d(i,j)=d(j,i) d( i,i)=0 13

14. Many clustering algorithms operate on a dissimilarity matrix. If the data are presented in the form of a data matrix, it can first be transformed into a dissimilarity matrix before applying clustering algorithms. May 26, 2016Data Mining: Concepts and Techniques14

15. May 26, 2016Data Mining: Concepts and Techniques15 Type of data in clustering analysis Interval-scaled variables Binary variables Nominal, ordinal, and ratio variables Variables of mixed types

16. 16 Interval-valued variables ISV are continuous measurements of a roughly linear scale. e.g weight, height, lattitude and longitude coordinates. Standardize data Calculate the mean absolute deviation: where Calculate the standardized measurement (z-score) Using mean absolute deviation is more robust than using standard deviation

17. 17 Similarity and Dissimilarity Between Objects Distances are normally used to measure the similarity or dissimilarity between two data objects Some popular ones include: Minkowski distance: where i = (x i1, x i2, …, x ip ) and j = (x j1, x j2, …, x jp ) are two p- dimensional data objects, and q is a positive integer If q = 1, d is Manhattan distance

18. May 26, 2016Data Mining: Concepts and Techniques18 Similarity and Dissimilarity Between Objects (Cont.) If q = 2, d is Euclidean distance(most popular distance measure): Properties d(i,j)  0 d(i,i) = 0 d(i,j) = d(j,i) d(i,j)  d(i,k) + d(k,j) Also, one can use weighted distance, parametric Pearson product moment correlation, or other disimilarity measures

19. May 26, 2016Data Mining: Concepts and Techniques19

20. Binary variables A binary variable has only two states: 0 or 1, where 0 means that the variable is absent, and 1 means that it is present. “So, how can we compute the dissimilarity between two binary variables?” One approach involves computing a dissimilarity matrix from the given binary data. If all binary variables are thought of as having the same weight, we have the 2-by-2 contingency table of, where q is the number of variables that equal 1 for both objects i and j, r is the number of variables that equal 1 for object i but that are 0 for object j, s is the number of variables that equal 0 for object i but equal 1 for object j, and t is the number of variables that equal 0 for both objects i and j. The total number of variables is p, where p = q+r+s+t. 20

21. A binary variable is symmetric if both of its states are equally valuable and carry the same weight; that is, there is no preference on which outcome should be coded as 0 or 1. dissimilarity (or distance) measure, to assess the dissimilarity between symmetric objects i and j. A binary variable is asymmetric if the outcomes of the states are not equally important. Dissimilarity measure for asymmetric binary variables: 21

22. May 26, 2016Data Mining: Concepts and Techniques22

23. 23 Dissimilarity between Binary Variables Example gender is a symmetric attribute the remaining attributes are asymmetric binary let the values Y(yes) and P (positive) be set to 1, and the value N(no or negative) be set to 0. distance between objects(patients) based on asymmetric variables. Interpretation: mary and jim are unlikely to have same disease beacause they have highest dissimilarity. Jack and mary are the most likely to same disease.

24. May 26, 2016Data Mining: Concepts and Techniques24 Nominal Variables or categorial A generalization of the binary variable in that it can take more than 2 states, e.g., red, yellow, blue, green Method 1: Simple matching m: # of matches, p: total # of variables

25. May 26, 2016Data Mining: Concepts and Techniques25

26. May 26, 2016Data Mining: Concepts and Techniques26 Ordinal Variables An ordinal variable can be discrete or continuous Order is important, e.g., rank Can be treated like interval-scaled replace x if by their rank map the range of each variable onto [0, 1] by replacing i-th object in the f-th variable by compute the dissimilarity using methods for interval- scaled variables

27. May 26, 2016Data Mining: Concepts and Techniques27 Ratio-Scaled Variables Ratio-scaled variable: a positive measurement on a nonlinear scale, approximately at exponential scale, such as Ae Bt or Ae -Bt Methods: treat them like interval-scaled variables—not a good choice! (why?—the scale can be distorted) apply logarithmic transformation y if = log(x if ) treat them as continuous ordinal data treat their rank as interval-scaled

28. May 26, 2016Data Mining: Concepts and Techniques28 Variables of Mixed Types A database may contain all the six types of variables symmetric binary, asymmetric binary, nominal, ordinal, interval and ratio One may use a weighted formula to combine their effects f is binary or nominal: d ij (f) = 0 if x if = x jf, or d ij (f) = 1 otherwise f is interval-based: use the normalized distance f is ordinal or ratio-scaled compute ranks r if and and treat z if as interval-scaled

29. May 26, 2016Data Mining: Concepts and Techniques29 Vector Objects Vector objects: keywords in documents, gene features in micro-arrays, etc. Broad applications: information retrieval, biologic taxonomy, etc. Cosine measure A variant: Tanimoto coefficient

30. May 26, 2016Data Mining: Concepts and Techniques30 Chapter 7. Cluster Analysis 1. What is Cluster Analysis? 2. Types of Data in Cluster Analysis 3. A Categorization of Major Clustering Methods 4. Partitioning Methods 5. Hierarchical Methods 6. Density-Based Methods 7. Outlier Analysis

31. May 26, 2016Data Mining: Concepts and Techniques31 Major Clustering Approaches (I) Partitioning approach: Construct various partitions and then evaluate them by some criterion, e.g., minimizing the sum of square errors Typical methods: k-means, k-medoids, CLARANS Hierarchical approach: Create a hierarchical decomposition of the set of data (or objects) using some criterion Typical methods: Diana, Agnes, BIRCH, ROCK, CAMELEON Density-based approach: Based on connectivity and density functions Typical methods: DBSACN, OPTICS, DenClue

32. May 26, 2016Data Mining: Concepts and Techniques32 Major Clustering Approaches (II) Grid-based approach: based on a multiple-level granularity structure Typical methods: STING, WaveCluster, CLIQUE Model-based: A model is hypothesized for each of the clusters and tries to find the best fit of that model to each other Typical methods: EM, SOM, COBWEB Frequent pattern-based: Based on the analysis of frequent patterns Typical methods: pCluster User-guided or constraint-based: Clustering by considering user-specified or application-specific constraints Typical methods: COD (obstacles), constrained clustering

33. May 26, 2016Data Mining: Concepts and Techniques33 Typical Alternatives to Calculate the Distance between Clusters Single link: smallest distance between an element in one cluster and an element in the other, i.e., dis(K i, K j ) = min(t ip, t jq ) Complete link: largest distance between an element in one cluster and an element in the other, i.e., dis(K i, K j ) = max(t ip, t jq ) Average: avg distance between an element in one cluster and an element in the other, i.e., dis(K i, K j ) = avg(t ip, t jq ) Centroid: distance between the centroids of two clusters, i.e., dis(K i, K j ) = dis(C i, C j ) Medoid: distance between the medoids of two clusters, i.e., dis(K i, K j ) = dis(M i, M j ) Medoid: one chosen, centrally located object in the cluster

34. May 26, 2016Data Mining: Concepts and Techniques34 Centroid, Radius and Diameter of a Cluster (for numerical data sets) Centroid: the “middle” of a cluster Radius: square root of average distance from any point of the cluster to its centroid Diameter: square root of average mean squared distance between all pairs of points in the cluster

35. May 26, 2016Data Mining: Concepts and Techniques35 Chapter 7. Cluster Analysis 1. What is Cluster Analysis? 2. Types of Data in Cluster Analysis 3. A Categorization of Major Clustering Methods 4. Partitioning Methods 5. Hierarchical Methods 6. Density-Based Methods 7. Grid-Based Methods 8. Model-Based Methods 9. Clustering High-Dimensional Data 10. Constraint-Based Clustering 11. Outlier Analysis 12. Summary

36. Partitioning Algorithms: Basic Concept commonly used partitioning methods are k-means, k- medoids, and their variations The k-means algorithm takes the input parameter, k, and partitions a set of n objects into k clusters so that the resulting intracluster similarity is high but the intercluster similarity is low. Cluster similarity is measured in regard to the mean value of the objects in a cluster, which can be viewed as the cluster’s centroid or center of gravity. May 26, 2016Data Mining: Concepts and Techniques36

37. May 26, 2016Data Mining: Concepts and Techniques37

38. May 26, 2016Data Mining: Concepts and Techniques38

39. May 26, 2016Data Mining: Concepts and Techniques39

40. May 26, 2016Data Mining: Concepts and Techniques40 The K-Means Clustering Method Example 0 1 2 3 4 5 6 7 8 9 10 0123456789 0 1 2 3 4 5 6 7 8 9 0123456789 K=2 Arbitrarily choose K object as initial cluster center Assign each objects to most similar center Update the cluster means reassign

41. Strength Determine K partitions that minimize the squared error function Works well when clusters are compact clouds Relatively scalable and efficient: computational complexity of algorithms is O(nkt) where n- total number of objects, k- total number of clusters, t- number of iterations. Terminates at local optimal. Weakness Applicable only when mean is defined, then what about categorical data? Need to specify k, the number of clusters, in advance Unable to handle noisy data and outliers Not suitable to discover clusters with non-convex shapes May 26, 2016Data Mining: Concepts and Techniques41

42. Variations of the k-means methods K-modes method:- extends the k-means paradigm to cluster categorical data by replacing, the means of clusters with modes. K mean and k mode method can be integrated to cluster data with mixed numeric and categorical values resulting in k prototype method. EM(expectation maximization) :- it assigns each cluster according to a weight representing probability of membership. There are no strict boundaries between clusters. May 26, 2016Data Mining: Concepts and Techniques42

43. May 26, 2016Data Mining: Concepts and Techniques43 What Is the Problem of the K-Means Method? The k-means algorithm is sensitive to outliers ! Since an object with an extremely large value may substantially distort the distribution of the data. K-Medoids: Instead of taking the mean value of the object in a cluster as a reference point, medoids can be used, which is the most centrally located object in a cluster. 0 1 2 3 4 5 6 7 8 9 10 0123456789 0 1 2 3 4 5 6 7 8 9 0123456789

44. May 26, 2016Data Mining: Concepts and Techniques44 The K-Medoids Clustering Method Find representative objects, called medoids, in clusters PAM (Partitioning Around Medoids, 1987) starts from an initial set of medoids and iteratively replaces one of the medoids by one of the non-medoids if it improves the total distance of the resulting clustering PAM works effectively for small data sets, but does not scale well for large data sets CLARA (Kaufmann & Rousseeuw, 1990) CLARANS (Ng & Han, 1994): Randomized sampling Focusing + spatial data structure (Ester et al., 1995)

45. 2) K-mediods methods Instead of mean, pick actual object to represent the cluster, using one object per cluster. Each remaining object is clustered with the representative object to which it is most similar. Partitioning method is then performed based on the principle of minimizing the sum of the dissimilarities between each object and its corresponding reference point. Absolute error criterion is defined as where E is the sum of the absolute error for all objects in the data set; p – point in space representing given objects in cluster Cj Oj representative object of Cj 45

46. In general, algorithm iterates until each representative object is actually the mediod or most centrally located object,of its cluster. Initially representative object is chosen arbitrarily. Iterative process of replacing representative objects by non representative objects continues as long as the quality of resulting clustering is improved. Quality is estimated by cost function that measures the average dissimilarity between an object and the representative object of its cluster. To determine whether a non representative object Orandom is a good replacement for current representative object Oj, the following 4 cases are examined for each non representative objects p May 26, 2016Data Mining: Concepts and Techniques46

47. May 26, 2016Data Mining: Concepts and Techniques47

48. P is object Oj replaced by Orandom, now Orandom is new representative of cluster Oi, Oj representative of different clusters. May 26, 2016Data Mining: Concepts and Techniques48

49. May 26, 2016Data Mining: Concepts and Techniques49

50. Each time a reassignment occurs, a difference in E (absolute error) is contributed to cost function. Study PAM May 26, 2016Data Mining: Concepts and Techniques50

51. May 26, 2016Data Mining: Concepts and Techniques51 What Is the Problem with PAM? Pam is more robust than k-means in the presence of noise and outliers because a medoid is less influenced by outliers or other extreme values than a mean Pam works efficiently for small data sets but does not scale well for large data sets. O(k(n-k) 2 ) for each iteration where n is # of data,k is # of clusters  Sampling based method, CLARA(Clustering LARge Applications)

52. May 26, 2016Data Mining: Concepts and Techniques52 CLARA (Clustering Large Applications) (1990) CLARA (Kaufmann and Rousseeuw in 1990) Built in statistical analysis packages, such as S+ It draws multiple samples of the data set, applies PAM on each sample, and gives the best clustering as the output Strength: deals with larger data sets than PAM Weakness: Efficiency depends on the sample size A good clustering based on samples will not necessarily represent a good clustering of the whole data set if the sample is biased

53. May 26, 2016Data Mining: Concepts and Techniques53 CLARANS (“Randomized” CLARA) (1994) CLARANS (A Clustering Algorithm based on Randomized Search) (Ng and Han’94) CLARANS draws sample of neighbors dynamically The clustering process can be presented as searching a graph where every node is a potential solution, that is, a set of k medoids If the local optimum is found, CLARANS starts with new randomly selected node in search for a new local optimum It is more efficient and scalable than both PAM and CLARA Focusing techniques and spatial access structures may further improve its performance (Ester et al.’95)

54. May 26, 2016Data Mining: Concepts and Techniques54 Chapter 7. Cluster Analysis 1. What is Cluster Analysis? 2. Types of Data in Cluster Analysis 3. A Categorization of Major Clustering Methods 4. Partitioning Methods 5. Hierarchical Methods 6. Density-Based Methods 7. Grid-Based Methods 8. Model-Based Methods 9. Clustering High-Dimensional Data 10. Constraint-Based Clustering 11. Outlier Analysis 12. Summary

55. Hierarchical clustering method work by grouping data objects into a tree of clusters. Methods: agglomerative and divisive May 26, 2016Data Mining: Concepts and Techniques55

56. May 26, 2016Data Mining: Concepts and Techniques56 Hierarchical Clustering Use distance matrix as clustering criteria. This method does not require the number of clusters k as an input, but needs a termination condition Step 0 Step 1Step 2Step 3Step 4 b d c e a a b d e c d e a b c d e Step 4 Step 3Step 2Step 1Step 0 Agglomerative nesting (AGNES) Divisive analysis (DIANA)

57. Nearest neighbor clustering algorithm Farthest neighbor clustering algorithm May 26, 2016Data Mining: Concepts and Techniques57

58. May 26, 2016Data Mining: Concepts and Techniques58 AGNES (Agglomerative Nesting) Introduced in Kaufmann and Rousseeuw (1990) Implemented in statistical analysis packages, e.g., Splus Use the Single-Link method and the dissimilarity matrix. Merge nodes that have the least dissimilarity Go on in a non-descending fashion Eventually all nodes belong to the same cluster

59. May 26, 2016Data Mining: Concepts and Techniques59 Dendrogram: Shows How the Clusters are Merged Decompose data objects into a several levels of nested partitioning (tree of clusters), called a dendrogram. A clustering of the data objects is obtained by cutting the dendrogram at the desired level, then each connected component forms a cluster.

60. May 26, 2016Data Mining: Concepts and Techniques60 DIANA (Divisive Analysis) Introduced in Kaufmann and Rousseeuw (1990) Implemented in statistical analysis packages, e.g., Splus Inverse order of AGNES Eventually each node forms a cluster on its own

61. May 26, 2016Data Mining: Concepts and Techniques61 Recent Hierarchical Clustering Methods Major weakness of agglomerative clustering methods do not scale well: time complexity of at least O(n 2 ), where n is the number of total objects can never undo what was done previously Integration of hierarchical with distance-based clustering BIRCH (1996): uses CF-tree and incrementally adjusts the quality of sub-clusters ROCK (1999): clustering categorical data by neighbor and link analysis CHAMELEON (1999): hierarchical clustering using dynamic modeling

62. BIRCH Birch: Balanced Iterative Reducing and Clustering using Hierarchies Is an integrated hierarchical clustering method Based on 2 concepts : clustering feature(CF) and clustering feature tree( CF Tree) Concepts are used to summarize cluster representations. CF is a triplet summarizing function about clusters of objects. Given n D-dimensional objects in a cluster {xi} then CF of cluster is CF = ( n, LS, SS) Where n – number of points in cluster LS – linear sum of n points i.e SS – square sum of the data points i.e 62

63. May 26, 2016Data Mining: Concepts and Techniques63 Clustering Feature Vector in BIRCH Clustering Feature: CF = (N, LS, SS) N: Number of data points LS:  N i=1 =X i SS:  N i=1 =X i 2 CF = (5, (16,30),(54,190)) (3,4) (2,6) (4,5) (4,7) (3,8)

64. 64

65. May 26, 2016Data Mining: Concepts and Techniques65 CF-Tree in BIRCH A CF tree is a height-balanced tree that stores the clustering features for a hierarchical clustering A nonleaf node in a tree has descendants or “children” The nonleaf nodes store sums of the CFs of their children A CF tree has two parameters Branching factor,B: specify the maximum number of children per non leaf node Threshold,T: max diameter of sub-clusters stored at the leaf nodes These parameters influence the size of resulting tree.

66. May 26, 2016Data Mining: Concepts and Techniques66 The CF Tree Structure CF 1 child 1 CF 3 child 3 CF 2 child 2 CF 6 child 6 CF 1 child 1 CF 3 child 3 CF 2 child 2 CF 5 child 5 CF 1 CF 2 CF 6 prevnext CF 1 CF 2 CF 4 prevnext B = 7 L = 6 Root Non-leaf node Leaf node

67. Phases of BIRCH Incrementally construct a CF (Clustering Feature) tree, a hierarchical data structure for multiphase clustering Phase 1: scan DB to build an initial in-memory CF tree (a multi- level compression of the data that tries to preserve the inherent clustering structure of the data) Phase 2: use an arbitrary clustering algorithm to cluster the leaf nodes of the CF-tree May 26, 2016Data Mining: Concepts and Techniques67

68. Strength: Handles large data sets Good clustering quality Single scan of the dataset – I/O is minimized Scales linearly – finds a good clustering with a single scan and improves the quality with a few additional scans Computational complexity – O(n) where n is number of objects to be clustered Weakness: Handles only numeric data Sensitive to the order of the data record Fails to detect clusters of arbitrary shapes 68

69. May 26, 2016Data Mining: Concepts and Techniques69 Clustering Categorical Data: The ROCK Algorithm ROCK: RObust Clustering using linKs Major ideas Use links to measure similarity(number of common neighbors between two objects) Categorical attributes. Not distance-based Computational complexity: Algorithm: sampling-based clustering Draw random sample Cluster with links Label data in disk

70. ROCK takes global approach to clustering by considering the neighborhoods of individual pairs of points. If two similar points also have similar neighborhoods, then the two points likely belong to the same cluster and so can be merged. May 26, 2016Data Mining: Concepts and Techniques70

71. May 26, 2016Data Mining: Concepts and Techniques71 Similarity Measure in ROCK Traditional measures for categorical data may not work well, e.g., Jaccard coefficient Example: Two groups (clusters) of transactions C 1. : {a, b, c}, {a, b, d}, {a, b, e}, {a, c, d}, {a, c, e}, {a, d, e}, {b, c, d}, {b, c, e}, {b, d, e}, {c, d, e} C 2. : {a, b, f}, {a, b, g}, {a, f, g}, {b, f, g} Jaccard co-efficient may lead to wrong clustering result C 1 : 0.2 ({a, b, c}, {b, d, e}} to 0.5 ({a, b, c}, {a, b, d}) C 1 & C 2 : could be as high as 0.5 ({a, b, c}, {a, b, f}) Jaccard co-efficient-based similarity function: Ex. Let T 1 = {a, b, c}, T 2 = {c, d, e}

72. May 26, 2016Data Mining: Concepts and Techniques72 Link Measure in ROCK Links: # of common neighbors. Suppose market basket analysis database contains transaction containing a,b,…..g items. Consider two clusters of transactions C1 and C2. C 1 : {a, b, c}, {a, b, d}, {a, b, e}, {a, c, d}, {a, c, e}, {a, d, e}, {b, c, d}, {b, c, e}, {b, d, e}, {c, d, e} C 2 : {a, b, f}, {a, b, g}, {a, f, g}, {b, f, g} Let T 1 = {a, b, c}, T 2 = {c, d, e}, T 3 = {a, b, f} link(T 1, T 2 ) = 4, since they have 4 common neighbors {a, c, d}, {a, c, e}, {b, c, d}, {b, c, e} link(T 1, T 3 ) = 3, since they have 3 common neighbors {a, b, d}, {a, b, e}, {a, b, g} Thus link is a better measure than Jaccard coefficient

73. May 26, 2016Data Mining: Concepts and Techniques73 Chapter 7. Cluster Analysis 1. What is Cluster Analysis? 2. Types of Data in Cluster Analysis 3. A Categorization of Major Clustering Methods 4. Partitioning Methods 5. Hierarchical Methods 6. Density-Based Methods

74. May 26, 2016Data Mining: Concepts and Techniques74 Density-Based Clustering Methods Clustering based on density (local cluster criterion), such as density- connected points regard clusters as dense regions of objects in the data space that are separated by regions of low density (representing noise). Major features: Discover clusters of arbitrary shape Handle noise One scan Need density parameters as termination condition

75. May 26, 2016Data Mining: Concepts and Techniques75 Density-Based Clustering: Basic Concepts Two parameters: Eps: Maximum radius of the neighbourhood MinPts: Minimum number of points in an Eps- neighbourhood of that point N Eps (p):{q belongs to D | dist(p,q) <= Eps} Directly density-reachable: A point p is directly density- reachable from a point q w.r.t. Eps, MinPts if p belongs to N Eps (q) core point condition: |N Eps (q)| >= MinPts p q MinPts = 5 Eps = 1 cm

76. May 26, 2016Data Mining: Concepts and Techniques76 Density-Reachable and Density-Connected Density-reachable: A point p is density-reachable from a point q w.r.t. Eps, MinPts if there is a chain of points p 1, …, p n, p 1 = q, p n = p such that p i+1 is directly density-reachable from p i Density-connected A point p is density-connected to a point q w.r.t. Eps, MinPts if there is a point o such that both, p and q are density-reachable from o w.r.t. Eps and MinPts p q p1p1 pq o

77. From han & kamber book May 26, 2016Data Mining: Concepts and Techniques77

78. May 26, 2016Data Mining: Concepts and Techniques78 DBSCAN: Density Based Spatial Clustering of Applications with Noise Relies on a density-based notion of cluster: A cluster is defined as a maximal set of density-connected points Discovers clusters of arbitrary shape in spatial databases with noise Core Border Outlier Eps = 1cm MinPts = 5

79. e.g. density reachability and density connectivity Consider for a given e represented by the radius of the circles, and, say, let MinPts = 3. Based on the above definitions: Of the labeled points,m, p, o, and r are core objects because each is in an e-neighborhood containing at least three points. 79

80. q is directly density-reachable from m. m is directly density- reachable from p and vice versa. q is (indirectly) density-reachable from p because q is directly density-reachable from m and m is directly density- reachable from p. However, p is not density-reachable from q because q is not a core object. Similarly, r and s are density-reachable from o, and o is density-reachable from r because r is core object.. but not from s as s is not core object. o, r, and s are all density-connected. 80

81. How does DBSCAN find cluster? DBSCAN searches for clusters by checking the e- neighborhood of each point in the database. If the e-neighborhood of a point p contains more than Minpts, a new cluster with p as a core object is created. DBSCAN then iteratively collects density reachable objects from these core objects, which may involve the merge of a few density reachable clusters. The process terminates when no new point can be added to any cluster. May 26, 201681

82. May 26, 2016Data Mining: Concepts and Techniques82 DBSCAN: The Algorithm Arbitrary select a point p Retrieve all points density-reachable from p w.r.t. Eps and MinPts. If p is a core point, a cluster is formed. If p is a border point, no points are density-reachable from p and DBSCAN visits the next point of the database. Continue the process until all of the points have been processed.

83. May 26, 2016Data Mining: Concepts and Techniques83 OPTICS: A Cluster-Ordering Method (1999) OPTICS: Ordering Points To Identify the Clustering Structure Ankerst, Breunig, Kriegel, and Sander (SIGMOD ’ 99) Produces a special order of the database wrt its density-based clustering structure This cluster-ordering contains info equiv to the density- based clusterings corresponding to a broad range of parameter settings Good for both automatic and interactive cluster analysis, including finding intrinsic clustering structure Can be represented graphically or using visualization techniques

84. May 26, 2016Data Mining: Concepts and Techniques84 OPTICS: Some Extension from DBSCAN Index-based: k = number of dimensions N = 20 p = 75% M = N(1-p) = 5 Complexity: O(kN 2 ) Core Distance Reachability Distance D p2 MinPts = 5  = 3 cm Max (core-distance (o), d (o, p)) r(p1, o) = 2.8cm. r(p2,o) = 4cm o o p1