I don’t need a title slide for a lecture Long long ago, in a galaxy far, far away… 11/28/2018
Outline Background Data mining Association Rules Classification Clustering Sequential Patterns Sequence Similarity 11/28/2018
Knowledge Discovery in Databases (KDD) What is it? Finding useful patterns in data Why do we need it? Terabytes of data Impractical to manually search for patterns Where does data mining come in? 11/28/2018
Steps of a KDD process Learn the application domain Create a target dataset Clean and preprocess data Choose type of data mining Pick an algorithm Perform data mining Interpret results 11/28/2018
Databases vs. Data warehousing Storage of all data Details or summaries Metadata Data cleaning, integration Databases Queries over current data Persistent storage Atomic updates 11/28/2018
Databases vs. Data warehouses Databases provide for: Queries over current data Persistent storage Atomic updates Data warehouses provide for: Storage of all data Meta data Data cleaning, integration Fast access to data 11/28/2018
Who’s interested? Databases - large amounts of data Artificial Intelligence - search, planning, machine learning Information Retrieval - searching for similar documents Image Processing - finding similar images 11/28/2018
Types of data mining Association Rules Classification Clustering Sequential Patterns Sequence Similarity 11/28/2018
Association rules What are they? Where are they used? Looking for common causal relationships in basket data Where are they used? Store layout Catalog design Customer segmentation 11/28/2018
Association rules example Find all itemsets that occur at least twice, and the causal relationship of each 11/28/2018
Association rules metrics For a rule a b support = a and b occur together in at least s% of the n baskets confidence = of all of the baskets containing a, at least c% also contain b 11/28/2018
Association rules algorithms Focus on finding support for “itemsets” The naïve method: Combine itemsets of size k-1 that differ only on the last item to find Candidatesk Measure support of itemsets from step 1 to form large itemsetk Increase k and repeat until no new large itemsets 11/28/2018
Itemsets of size 1 Looking for support of 2 11/28/2018
Finding candidate set 2 11/28/2018
Finding candidate set 3 11/28/2018
Apriori algorithm An itemset cannot be a large itemset unless all of its subsets are large itemsets Reduces number of candidate itemsets considered 11/28/2018
Research directions Online construction of rules CARMA (Berkeley) Pre filtering the data a posteriori (Limburgs Universitair Centrum) 11/28/2018
Classification What is it? Where is it used? Rules that partition data into separate groups. Where is it used? to classify people as good/bad credit risks weather prediction fraud detection Variation: best k of n (who to send flyers to) 11/28/2018
Classification example 11/28/2018
Possible solutions Bayesian classification Neural networks Genetic algorithms Decision Trees 11/28/2018
Decision trees Salary < 25,000 no yes Graduate education? Accept no Reject 11/28/2018
Decision trees Build the tree in two steps Build a perfect tree on sample data At each node, pick a “good” attribute Split data according to attribute Recursively build tree on children Prune the tree Minimum Description Length Cost of encoding tree structure Cost of encoding split attribute Cost of encoding leaf data records 11/28/2018
Research directions Integrate building and pruning Incremental Updates PUBLIC (Bell Labs) Incremental Updates BOAT (University of Wisconsin) 11/28/2018
Clustering What is it? Where is it used? Given n points, separate them into k clusters Where is it used? Information retrieval - text classification Identify similar web documents Mapping the universe 11/28/2018
Clustering example 11/28/2018
Traditional clustering algorithms Partitional Determine k partitions that optimize a function Common function is the “square error function” Hierarchical Each point starts as a cluster Clusters are merged until k clusters remain 11/28/2018
Clustering difficulties 11/28/2018
Research directions Higher dimension subspace clustering CLIQUE (IBM Almaden) Incremental clustering Incremental DBScan (University of Munich) Remove problems with outliers CURE (Bell Labs) 11/28/2018
Sequential patterns What is it? Where is it used? Given a set of events, find frequently occurring patterns Where is it used? Analyzing basket data Medical diagnosis 11/28/2018
Sequential patterns example 11/28/2018
AprioriAll Create all large events that occur once Map each subset to numbers While there still are large itemsets: Find candidate itemsets of length k Find large itemsets of length k Increase k 11/28/2018
Mapping the itemsets 11/28/2018
Research directions Time limitations WINEPI (Helsinki/Microsoft) Itemsets over multiple transactions CSP (IBM Almaden) 11/28/2018
Sequence Similarity What is it? Where is it used? Given a number of data sets, look for similar trends Where is it used? Find stocks with similar price movements Find geological irregularities 11/28/2018
Example Are the two sequences similar? 11/28/2018
Basic algorithm Scale data Match all gap-free sequences Form pairs of large similar sequences Find the longest common subsequence 11/28/2018
Research directions Finding surprising patterns IBM Almaden 11/28/2018
Data mining directions Sampling Fractals Pre-partitioning data Making data mining more accessible User defined aggregation support 11/28/2018
References General Data mining: http://www.almaden.ibm.com/cs/quest, www.bell-labs.com/project/serendip Association Rules: “Fast Algorithms for Mining Association Rules”, Agrawal and Srikant; VLDB 94. Classification: “PUBLIC: A Decision Tree Classifier that Integrates Building and Pruning”, Rastogi and Shim; VLDB 98. 11/28/2018
References (cont.) Clustering: “CURE: An Efficient Clustering Algorithm for Large Databases”, Guha, Rastogi, Shim; SIGMOD 98. Sequential Patterns: “Mining Sequential Patterns: Generalizations and Performance Improvements”, Srikant and Agrawal; EDBT 98. Similarity Search: “Fast Similarity Search in the Presence of Noise, Scaling, and Translation in Time-Series Databases”, Agrawal, Nin, Sawhney, and Shim; VLDB 95. 11/28/2018