1. Clustering Large Datasets in Arbitrary Metric Spaceby
Muralikrishna Achari
COSC6341 Information Retreival Project Presentation

2. COSC6341 Information Retreival Project Presentation
Contents
Introduction to Clustering
Problems in Traditional Clustering
Clustering Large Datasets
BIRCH*
BUBBLE
BUBBLE-FM
Scalability
Conclusion
COSC6341 Information Retreival Project Presentation

3. Traditional Clustering
Unsupervised Learning.
A process of grouping similar object into groups.
Distance between object is used as a common metric to assess similarity
COSC6341 Information Retreival Project Presentation

4. Types of Clustering Algorithms
Hierarchical clustering
Minimal Spanning Tree Method, BIRCH, BUBBLE
Partition based clustering
K-means, CLARANS
COSC6341 Information Retreival Project Presentation

5. Hierarchical Clustering
A crude division of instances into groups at the top level, and each of these groups is refined further – perhaps all the way down to the individual instances.
COSC6341 Information Retreival Project Presentation

6. Partition based clustering
A desired number of clusters are assumed at the start and instances are allocated among clusters so that a particular clustering criterion is optimized (e.g. minimization of the variability within clusters).
COSC6341 Information Retreival Project Presentation

7. COSC6341 Information Retreival Project Presentation
Applications
Marketing: finding groups of customers with similar behavior.
Landscapes : Characterizing different regions.
Biology: classification of plants and animals given their features.
Earthquake studies: clustering observed earthquake epicenters to identify dangerous zones.
WWW: document classification; clustering weblog data to discover groups of similar access patterns.
COSC6341 Information Retreival Project Presentation

8. Problem with Traditional Clustering
Dealing with large number of dimensions and large number of data items can be problematic because of time complexity.
COSC6341 Information Retreival Project Presentation

9. Requirements for a Good Clustering Algorithm
Scalability.
Dealing with different types of attributes.
Discovering clusters with arbitrary shape.
Minimal requirements for domain knowledge to determine input parameters.
Ability to deal with noise and outliers.
COSC6341 Information Retreival Project Presentation

10. Clustering Large Datasets
COSC6341 Information Retreival Project Presentation

11. Clustering Large Datasets
CLARANS
Assumes all object fit in main memory,  sensitive to input order.
Uses R* to improve efficiency.
BIRCH
Minimizes memory usage and scans data only once from disk.
Uses cluster representatives instead of actual data points.
1st algorithm proposed in the database area that addresses outliers.
DBSCAN
Uses distance based notion to clusters to discover clusters of arbitrary shapes.
Sensitive to the input parameters and incurs substantial I/O cost.
COSC6341 Information Retreival Project Presentation

12. COSC6341 Information Retreival Project Presentation
Drawbacks
Both BIRCH and CLARANS works well for clusters with Spherical or Convex sphape and uniform size and are unsuitable when clusters have different sizes and are non-spherical.
All the three algorithms relies on vector operations which are only defined in coordinate space and are unsuitable to datasets in distance space.
COSC6341 Information Retreival Project Presentation
BRICH

13. COSC6341 Information Retreival Project Presentation
Proposed Approach
2 algorithms for clustering large datasets based on BIRCH* framework.
BUBBLE
BUBBLE-FM
COSC6341 Information Retreival Project Presentation

14. COSC6341 Information Retreival Project Presentation
BIRCH*
Balanced Iterative Reducing and Clustering using Hierarchies
BIRCH* is generalized framework for incremental clustering algorithms.
BIRCH* components can be instantiated to generate concrete clustering algorithms.
COSC6341 Information Retreival Project Presentation

15. COSC6341 Information Retreival Project Presentation
BIRCH* Components
Cluster Feature (CF*)
A Summarized representation of the cluster
Cluster Tree (CF*-tree)
A Height balanced tree for CF*’s
COSC6341 Information Retreival Project Presentation

16. COSC6341 Information Retreival Project Presentation
Clustering Feature
CFs are summarized representations of clusters.
Requirements:-
Incrementally maintainable when a new object is inserted.
Contain Sufficient information to compute distance between clusters and objects.
COSC6341 Information Retreival Project Presentation

17. COSC6341 Information Retreival Project Presentation
CF*-Tree
A height-balanced tree.
Two parameters
1. Branching Factor, B
2. Threshold, T
Non-leaf node has B entries; ( [CFi, childi], i = 1..B)
Where CFi is the CF of the sub clusters represented by this child
Childi is a pointer to its ith child.
COSC6341 Information Retreival Project Presentation

18. COSC6341 Information Retreival Project Presentation
CF*-tree
Leaf node
Satisfies threshold T, which controls its tightness and quality.
Diameter or radius < T
Tree size is a function of T
T increase tree size decreases.
COSC6341 Information Retreival Project Presentation

19. COSC6341 Information Retreival Project Presentation
CF Tree
COSC6341 Information Retreival Project Presentation

20. Functionality of CF*-tree
Direct a new object, O, to the cluster closest to it.
Non-leaf node: exits to guide new objects to appropriate leaf clusters.
Leaf node: absorbs the new object.
COSC6341 Information Retreival Project Presentation

21. COSC6341 Information Retreival Project Presentation
BIRCH*: Mechanism
Starts with an initial T.
Scans the data and inserts the objects into the tree.
During scan, existing clusters are updated and new clusters are formed.
If runs out of memory, M, increases T and builds a smaller CF*-tree.
After inserting the old leaf entries, resumes from the point at which it was interrupted.
COSC6341 Information Retreival Project Presentation

22. COSC6341 Information Retreival Project Presentation
CF*-tree insertion
CF* tree insertion mechanism is same as that of B+ trees.
Each new object, O:
Reaches the leaf node, L.
Inserted into a closest clusters C, if threshold, T, is not violated else forms a new Cluster
If there is not enough space in L, then split into two leaf nodes and distribute entries between the two nodes.
Like B+ tree, node splits might propagate till the root.
The path from the root to the leaf is updated to reflect the insertion.
COSC6341 Information Retreival Project Presentation

23. BRICH*: Instantiation Summary
Cluster features at leaf and non-leaf levels.
Incremental maintenance of Cluster Features at leaf and non leaf nodes
Distance measure between CF* and an object , and between CF*s.
Threshold requirement.
COSC6341 Information Retreival Project Presentation

24. COSC6341 Information Retreival Project Presentation
BUBBLE
BIRCH* instantiated in distance space.
No concept of Centroids.
For a given set of objects O = {O1…On}
Defines:-
Rowsum (O)=
Clustroid (O ) is and object O’ with least Rowsum value.
Radius, r (O ) =
Clustroid distance, D0 (O1 , O 2) = d(O1’,O2’)
COSC6341 Information Retreival Project Presentation

25. BUBBLE: CF at leaf nodes
For a set of objects O = {O1…On} and cluster C . CF is a five tupple defined as: (n, O’, R, RS, r)
n: Number of objects in C .
O’: Clustroid of C .
R : representatives of the Cluster C , (R C ).
RS: The Rowsum of all the representatives.
r: Radius of the Clusters C .
COSC6341 Information Retreival Project Presentation

26. BUBBLE: CF at non-leaf node
A set of sample objects, S (NLi), randomly collected from the subtree NLi form its CFi .
CF at NL =  S (NLi)
Each child node will have at least one representative in S(NL).
If CFi is leaf node then S (NLi) are randomly picked from the clustroids of CFi.
COSC6341 Information Retreival Project Presentation

27. BUBBLE: Incremental Maintenance of CF at leaf
Types of Insertions
Type I: Insertion of a single object
Type II: Insertion of a cluster of objects.
COSC6341 Information Retreival Project Presentation

28. COSC6341 Information Retreival Project Presentation
Type I Insertion
Inserting an object into the leaf
If |C| is small, maintain all the cluster objects and calculate the new clustroid.
If |C | is large, maintain a subset of C of size R that are close to the clustroid.
COSC6341 Information Retreival Project Presentation

29. COSC6341 Information Retreival Project Presentation
Type II Insertion
Inserting a cluster of objects:-
C1 and C2 must be non-overlapping but close clusters.
The location of the new clustroid is between the two old clustroids.
By maintaining few objects far away from the clustriods of C1 and C2 the new clustroid can be calculated.
COSC6341 Information Retreival Project Presentation

30. Incremental Maintenance of CF at non leaf
The sample objects at a non-leaf entry are updated whenever its child node splits.
The distribution of clusters changes significantly whenever a node splits.
To reflect changes in the distribution at all children nodes, update the sample objects at all entries of NL.
COSC6341 Information Retreival Project Presentation

31. COSC6341 Information Retreival Project Presentation
Drawbacks of BUBBLE
BUBBLE computes distance between sample objects which could be expensive.
E.g. edit distance on string
COSC6341 Information Retreival Project Presentation

32. COSC6341 Information Retreival Project Presentation
BUBBLE-FM
Transforms the distance space into an approximate vector space.
Maintains all the sample objects at each non leaf node in vector space.
For a new object O, transforms O to Vector space and uses Euclidean distance metric.
Doesn’t use transformation at leaf node.
COSC6341 Information Retreival Project Presentation

33. COSC6341 Information Retreival Project Presentation
Scalability
COSC6341 Information Retreival Project Presentation

34. COSC6341 Information Retreival Project Presentation
Scalability
COSC6341 Information Retreival Project Presentation

35. COSC6341 Information Retreival Project Presentation
Conclusion
Presented the BIRCH* framework for scalable incremental pre clustering algorithms.
BUBBLE for datasets in arbitrary metric space
Fast map to reduce the number of calls to an expensive distance function.
COSC6341 Information Retreival Project Presentation

36. COSC6341 Information Retreival Project Presentation
References
Primary Source:
Clustering Large Datasets in Arbitrary Metric Spaces (1999)  Venkatesh Ganti Raghu Ramakrishnan Johannes Gehrke Computer Sciences.
Secondary Sources:
BIRCH: An Efficient Data Clustering Method for Very Large Databases (1996)  Tian Zhang, Raghu Ramakrishnan, Miron Livny
CURE: An Efficient Clustering Algorithm for Large Databases (1998) Sudipto Guha, Rajeev Rastogi, Kyuscok Shim.
COSC6341 Information Retreival Project Presentation