Frequent Item Based Clustering M.Sc Student:Homayoun Afshar Supervisor:Martin Ester

Homayoun AfsharFrequent Item Based Clustering2 Contents Introduction and motivation Frequent item sets Text data as transactional data Cluster set definition Our approach Test data set, results, challenges Related works Conclusion

Homayoun AfsharFrequent Item Based Clustering3 Introduction and Motivation Huge amount of information online Lots of this information is in text format E.G. s, web pages, news group postings, … Need to group related documents Nontrivial task

Homayoun AfsharFrequent Item Based Clustering4 Frequent Item Sets Given a dataset D={t 1,t 2,…,t n } Each t i is a transaction t i  I where I is the set of all items Given a threshold min_sup i  I such that |{t  i  t and t  D}|>min_sup i is a frequent item set with respect to minimum support min_sup

Homayoun AfsharFrequent Item Based Clustering5 Text Data As Transactional Data Assume each word as an item And each document as a transaction Using a minimum support find frequent item sets (frequent word sets) Frequent Word Sets  Frequent Item Sets

Homayoun AfsharFrequent Item Based Clustering6 Cluster Set Definition f={X 1,X 2,…,X n } is the set of all the frequent item sets with respect to some minimum support c={C 1,C 2,…,C m } is a cluster set, where C i is the documents that are covered with some X k  f And…

Homayoun AfsharFrequent Item Based Clustering7 Cluster Set Definition … Each optimal cluster set has to: Cover the whole data set Mutual overlap between clusters in cluster set must be minimized Clusters should be roughly the same size

Homayoun AfsharFrequent Item Based Clustering8 Our Approach: Frequent-Item Based Clustering … Find all the frequent word sets Form cluster sets with just one cluster Overlap is zero Coverage is the support of the frequent item set presenting the cluster Form cluster sets with two clusters Find the overlap and coverage

Homayoun AfsharFrequent Item Based Clustering9 Our Approach: Frequent-Item Based Clustering … Prune the candidate list for cluster sets If Cov(c i )  Cov(c j ) and overlap(c i )>overlap(c j ) c i and c j are candidates in same level remove if Overlap(c i )>= |Cov(c i )| Generate the next level Find Overlap and Coverage, Prune Stop when there are no more candidates left

Homayoun AfsharFrequent Item Based Clustering10 Our Approach: Coverage And Overlap … Using a bit matrix Each column is a document Each row is a frequent word set Coverage: OR, counting the 1s Overlap: XOR, OR, AND, counting 1s

Homayoun AfsharFrequent Item Based Clustering11 Our Approach: Coverage And Overlap … (1st) (2nd) (3rd) Coverage: OR all = count 1s -> coverage = 6 cost = 2 ORs + counting 1s cost for counting 1s = 8 (shifts, ANDs, Adds)

Homayoun AfsharFrequent Item Based Clustering12 Our Approach: Coverage And Overlap … Overlap: (1st) (2nd) AND first two = (i) XOR first two = (ii) (3rd) AND 3rd with (ii) (iii) OR (i) and (iii) now count 1s for overlap -> Overlap = 4

Homayoun AfsharFrequent Item Based Clustering13 Test Data, Results, Challenges Test data set Reuters documents Reuters news 8655 of them have exactly one topic Remove stop words Stem all the words Number of frequent word sets 5% min_sup = % min_sup= % min_sup=78

Homayoun AfsharFrequent Item Based Clustering14 Test Data, Results, Challenges With 20% min support sample 2-cluster candidate set {(said,reuter)(line,ct,vs)} Overlap = 1 Coverage = 5259 sample 5-cluster candidate set {(reuter)(vs)(net)(line,ct,net)(vs,net,shr)} Overlap = 3303 Coverage = 8609

Homayoun AfsharFrequent Item Based Clustering15 Test Data, Results, Challenges More Results With min_sup=10% {(reuter)(includ)(mln,includ)(mln,profit)(year,ct)(year,mln,net)} 6-clusters cluster set Coverage = 8616 Overlap = 2553 {(reuter)(loss)(profit)(year,1986)(mln,profit)(year,ct)(year,mln,net)} 7-clusters cluster set Coverage = 8611 Overlap = 2705 {(reuter)(loss)(profit)(year,1986)(mln,includ)(mln,profit)(year,ct)(year,mln,net)} 8-clusters cluster set Coverage = 8616 Overlap = 3033

Homayoun AfsharFrequent Item Based Clustering16 Test Data, Results, Challenges Lower support values Pruning is very slow 2-cluster set with minSup=20% Creating= seconds. Updating= seconds. (Overlap and Coverage) Pruning= seconds. Sorting= seconds. Number of candidates Before prune=3003 After prune=73

Homayoun AfsharFrequent Item Based Clustering17 Test Data, Results, Challenges Hierarchical clustering Clustering quality In our test data set, entropy Real data sets, classes are not known Test the pruning more efficiently Defining an upper threshold Using following ratios to prune candidates or Using only max item sets

Homayoun AfsharFrequent Item Based Clustering18 Related Works Similar idea Frequent Term-Based Text Clustering [BEX02] Florian Beil, Martin Ester, Xiaowei Xu Focuses on finding one optimal clustering set (non overlapping)-FTC Hierarchical clustering (overlapping)-HFTC

Homayoun AfsharFrequent Item Based Clustering19 Conclusion To get optimal clustering Reduce minimum support Reduce number of frequent items Introduce maximum support Use only max item sets Better pruning (speed) Hierarchical clustering

Homayoun AfsharFrequent Item Based Clustering20 References [AS94] R. Agrawal, R. Sirkant. Fast Algorithms for Mining Association rules in large databases. In Proc Int. Conf. Very Large Data Bases (VLDB’94), pages , Santiago, Chile, Sept [BEX02] F. Beil, M. Ester,X. Xu. Frequent Term-Based Text clustering. J. Han, M. Kamber. Data Mining Concepts and Techniques. Morgan Kaufmann, 2001.