1. May 17 2001CASI 2001 Interactive Clustering Overview and Tools Wayne Oldford University of Waterloo

2. May 17 2001CASI 2001 Overview 1. Finding groups in data 2. Interactive data analysis 3. Enlarging the problem 4. Putting it together 5. Software modelling (illustration) 6. Summary

3. May 17 2001CASI 2001 1.Finding groups in data –Web documents as objects to be grouped Objects to be grouped together – –locations – –pairwise (dis)similarities Applications: – –Building groups to understand/model – –Building groups to use later as classification – –Building groups to serve as templates

4. May 17 2001CASI 2001 group definition is a problem Group definition (like with like) homogeneous vs heterogeneous part of pattern

5. May 17 2001CASI 2001 Clustering approaches Agglomerative (near points/clusters are joined) –Single linkage – –Complete linkage – –Average linkage Recursive splitting – –e.g. minimal spanning tree

6. May 17 2001CASI 2001 Cluster hierarchies Clusters are nested Often represented as a tree (dendrogram) Join/split history and ‘strength’ preserved

7. May 17 2001CASI 2001 Other approaches k-means –assign points to k groups –re-assign to improve objective function model-based –likelihood/Bayesian; model search/averaging density estimation –groups = high-density regions classification to cluster visually motivated methods

8. May 17 2001CASI 2001 Visual Empirical Regions of Influence (VERI)

9. May 17 2001CASI 2001 Notes many choices –between and within methods built-in biases for shapes computationally costly –O(n 2 )... Conceptual model: algorithmic, run to completion

10. May 17 2001CASI 2001 typical software resources dedicated to numerical computation –teletype interaction –runs to completion –graphical “output” Compare to interactive data analysis

11. May 17 2001CASI 2001 2. Interactive data analysis

12. May 17 2001CASI 2001 Interactive data analysis exploratory, tentative graphical non-algorithmic –varied granularity integrated deep interaction

13. May 17 2001CASI 2001 3. Enlarging the problem Mutually exclusive and exhaustive groups g 1, g 2, …, g k form a partition P = {g 1, g 2, …, g k } of the set of data objects. Goal:Explore the space of possible partitions.

14. May 17 2001CASI 2001 Structuring the partition space P A ={g 1, g 2, …, g a } and P B ={h 1, h 2, …, h b } When a > b, P A call a finer partition than P B. When a = b, P A is called a reassignment of P B P A is nested in P B only if a > b and every g i is a subset of a single h j - write P A } P B or P B { P A – –P A is called a refinement of P B (or P B a reduction of P A )

15. May 17 2001CASI 2001 Reduction P 1 ={g 1,..., g 6 } -> P 2 ={h 1, …, h 4 } -> P 3 ={m 1, m 2, m 3 } h i = g i i = 1, 2 ; h 3 = join (g 3, g 4 ) ; h 4 = join (g 5, g 6 ) –nesting: P 1 } P 2 – –split (h 4 ) = {h 1 *, h 2 *, h 3 * }; m i = join (h i *, h i ) – –P 2 } P 3 is false disperse elements of h 4 over h i i = 1, 2, 3 to give m i for i = 1, 2, 3.

16. May 17 2001CASI 2001 Reduction decisions/options join operations: which groups? –e.g. inner, outer, centres, … –distance measures to use … dispersal operations: –selecting group(s) Max volume, eigen-value, MST… –determining partitional method random, VERI, MST, … –choosing join …

17. May 17 2001CASI 2001 Refinement P 2 ={h 1, …, h 4 } ---> P 1 ={g 1,..., g 6 } g i = h i i = 1, 2 ; split (h 3 ) -> g 3, g 4 split (h 4 ) -> g 5, g 6 nesting: P 2 { P 1

18. May 17 2001CASI 2001 Refinement decisions/options which groups to split? – –e.g. inner, outer, directions, … – –distance measures to use … how to split? – –MST, outlying points, reassignment,...

19. May 17 2001CASI 2001 Reassignment objective function d(P) to be minimized. P <- P 1 for each object o in g i, assign it to one of g j (j != i) forming a new partition P ij and find largest  ij (o) = d(P) - d(P ij ) repeat for all i, j. If max  ij > 0 move o from g i, to g j Repeat until  max <=0 P 1 ={g 1,..., g k } -> P 2 ={h 1, …, h k }

20. May 17 2001CASI 2001 Reassignment decisions/options Objective function – –distances, centres, … – –within vs between/within,... – –variates/directions Iteration strategy – –single-pass, k-means, complete looping (greedy), start, …

21. May 17 2001CASI 2001 4. Putting it together Series of moves in partition space: 1. Refine (P) -- > P new 2. Reduce (P) -- > P new 3. Reassign (P) -- > P new

22. May 17 2001CASI 2001 Additional ops on partitions Unary: –Subset (P) –Operate any of R (subset (P)) –Manual (P) … change P according to manual intervention (e.g. colouring)

23. May 17 2001CASI 2001 n-ary operators resolve (P 1,..., P m ) --> P new dissimilarity (P i, P j ) --> d i,j display (P 1,..., P m ) – –dendrogram if P 1 { …{ P m – –mds plot of all clusters in P 1, …, P m – –mds plot of all partitions P 1, …, P m

24. May 17 2001CASI 2001 5. Software modelling Principal control panel: –current partition and list of saved partitions –refine, reduce, re-assign, re-start buttons –cluster plot button (mds plot) –random select button –subset focus and join toggle –operation on partitions button –manual button (form partition from point colours)

25. May 17 2001CASI 2001 Secondary panels Refine: –performs refine, offers access to arguments Reduce –performs reduce, offers access to arguments Reassign –performs reassign, offers access to arguments Each will operate on only those points highlighted or on all if none selected.

26. May 17 2001CASI 2001 Secondary panels (continued) Operate on partitions –saved partitions list –resolve selected partition –plot selected partitions using selected dissimilarity –dendrogram of selected partitions (if nested) –cluster-plot for clusters of selected parttitions (esp. for non-nested)

27. May 17 2001CASI 2001 Software modelling (details) Objects: –Point-symbols, case-objects (existing in Quail) –Cluster-points –Clusters –Partitions Methods –Reduce, refine, reassign,...

28. May 17 2001CASI 2001 Software illustration Two prototype displays (buggy) –Single-window –Separate windows Integration with existing Quail graphics Manual, dendrogram, cluster plots, … VERI clustering

29. May 17 2001CASI 2001 6. Summary Cluster analysis is naturally exploratory and needs integration with modern interactive data analysis. Enlarging the problem to partitions: –simplifies and gives structure –encourages exploratory approach –integrates naturally –introduces new possibilities (analysis and research)

30. May 17 2001CASI 2001 Acknowledgements Erin McLeish, several undergraduates and graduate students in statistical computing course at Waterloo Quail: Quantitative Analysis in Lisp http:/www.stats.uwaterloo.ca/Quail