Download presentation
Presentation is loading. Please wait.
1
Association Rule Mining
Instructor Qiang Yang Slides from Jiawei Han and Jian Pei And from Introduction to Data Mining By Tan, Steinbach, Kumar
2
What Is Frequent Pattern Mining?
Frequent patterns: pattern (set of items, sequence, etc.) that occurs frequently in a database [AIS93] Frequent pattern mining: finding regularities in data What products were often purchased together? What are the subsequent purchases after buying a PC? Frequent-pattern mining methods
3
Why Is Frequent Pattern Mining an Essential Task in Data Mining?
Foundation for many essential data mining tasks Association, correlation, causality Sequential patterns, temporal or cyclic association, partial periodicity, spatial and multimedia association Associative classification, cluster analysis, iceberg cube, fascicles (semantic data compression) Broad applications Basket data analysis, cross-marketing, catalog design, sale campaign analysis Web log (click stream) analysis, DNA sequence analysis, etc. Frequent-pattern mining methods
4
Basic Concepts: Frequent Patterns and Association Rules
Itemset X={x1, …, xk} Find all the rules XY with min confidence and support support, s, probability that a transaction contains XY confidence, c, conditional probability that a transaction having X also contains Y. Transaction-id Items bought 10 A, B, C 20 A, C 30 A, D 40 B, E, F Customer buys diaper buys both buys beer Let min_support = 50%, min_conf = 50%: A C (50%, 66.7%) C A (50%, 100%) Frequent-pattern mining methods
5
Concept: Frequent Itemsets
Outlook Temperature Humidity Play sunny hot high no overcast yes rainy mild cool normal Minimum support=2 {sunny, hot, no} {sunny, hot, high, no} {rainy, normal} Min Support =3 ? How strong is {sunny, no}? Count = Percentage = Frequent-pattern mining methods
6
Concept: Itemset Rules
{sunny, hot, no} = {Outlook=Sunny, Temp=hot, Play=no} Generate a rule: Outlook=sunny and Temp=hot Play=no How strong is this rule? Support of the rule = support of the itemset {sunny, hot, no} = 2 = Pr({sunny, hot, no}) Either expressed in count form or percentage form Confidence = Pr(Play=no | {Outlook=sunny, Temp=hot}) In general LHS RHS, Confidence = Pr(RHS|LHS) Confidence =Pr(RHS|LHS) =count(LHS and RHS) / count(LHS) What is the confidence of Outlook=sunnyPlay=no? Frequent-pattern mining methods
7
Frequent-pattern mining methods
Frequent Patterns Patterns = Item Sets {i1, i2, … in}, where each item is a pair: (Attribute=value) Frequent Patterns Itemsets whose support >= minimum support Support count(itemset)/count(database) Frequent-pattern mining methods
8
Frequent Itemset Generation
Given d items, there are 2d possible candidate itemsets Frequent-pattern mining methods
9
Frequent-pattern mining methods
Max-patterns Max-pattern: frequent patterns without proper frequent super pattern BCDE, ACD are max-patterns BCD is not a max-pattern Tid Items 10 A,B,C,D,E 20 B,C,D,E, 30 A,C,D,F Min_sup=2 Frequent-pattern mining methods
10
Maximal Frequent Itemset
An itemset is maximal frequent if none of its immediate supersets is frequent Maximal Itemsets Infrequent Itemsets Border Frequent-pattern mining methods
11
Frequent-pattern mining methods
Frequent Max Patterns Succinct Expression of frequent patterns Let {a, b, c} be frequent Then, {a, b}, {b, c}, {a, c} must also be frequent Then {a}, {b}, {c}, must also be frequent By writing down {a, b, c} once, we save lots of computation Max Pattern If {a, b, c} is a frequent max pattern, then {a, b, c, x} is NOT a frequent pattern, for any other item x. Frequent-pattern mining methods
12
Find Frequent Max Patterns
Outlook Temperature Humidity Play sunny hot high no overcast yes rainy mild cool normal Minimum support=2 {sunny, hot, no} ?? Frequent-pattern mining methods
13
Frequent-pattern mining methods
Closed Patterns An itemset is closed if none of its immediate supersets has the same support as the itemset {a, b}, {a, b, d}, {a, b, c} are closed patterns But, {a, b} is not a max pattern See where changes happen Reduce # of patterns and rules N. Pasquier et al. In ICDT’99 TID Items 10 a, b, c 20 30 a, b, d 40 a, b, d, 50 c, e, f Frequent-pattern mining methods
14
Maximal vs Closed Itemsets
Transaction Ids indexes beside an item set is the transaction #s. Not supported by any transactions Frequent-pattern mining methods
15
Maximal vs Closed Frequent Itemsets
Closed but not maximal Minimum support = 2 Closed and maximal # Closed = 9 # Maximal = 4 Frequent-pattern mining methods
16
Note on Closed Patterns
Closed patterns have no need to specify the minimum support Given dataset, we can find a set of closed patterns from it, so that for any minimum support values, we can immediately find the set of patterns (a subset of the closed patterns). Closed frequent patterns Both closed and above the min support Frequent-pattern mining methods
17
Maximal vs Closed Itemsets
Frequent-pattern mining methods
18
Mining Association Rules—an Example
Min. support 50% Min. confidence 50% Transaction-id Items bought 10 A, B, C 20 A, C 30 A, D 40 B, E, F Frequent pattern Support {A} 75% {B} 50% {C} {A, C} For rule A C: support = support({A}{C}) = 50% confidence = support({A}{C})/support({A}) = 66.6% Frequent-pattern mining methods
19
Method 1: Apriori: A Candidate Generation-and-test Approach
Any subset of a frequent itemset must be frequent if {beer, diaper, nuts} is frequent, so is {beer, diaper} Every transaction having {beer, diaper, nuts} also contains {beer, diaper} Apriori pruning principle: If there is any itemset which is infrequent, its superset should not be generated/tested! Method: generate length (k+1) candidate itemsets from length k frequent itemsets, and test the candidates against DB The performance studies show its efficiency and scalability Agrawal & Srikant 1994, Mannila, et al. 1994 Frequent-pattern mining methods
20
The Apriori Algorithm — An Example
Itemset sup {A} 2 {B} 3 {C} {D} 1 {E} Database TDB Itemset sup {A} 2 {B} 3 {C} {E} L1 C1 Tid Items 10 A, C, D 20 B, C, E 30 A, B, C, E 40 B, E 1st scan C2 Itemset sup {A, B} 1 {A, C} 2 {A, E} {B, C} {B, E} 3 {C, E} C2 Itemset {A, B} {A, C} {A, E} {B, C} {B, E} {C, E} L2 2nd scan Itemset sup {A, C} 2 {B, C} {B, E} 3 {C, E} C3 L3 Itemset {B, C, E} 3rd scan Itemset sup {B, C, E} 2 Frequent-pattern mining methods
21
Speeding up Association rules
Dynamic Hashing and Pruning technique Thanks to Cheng Hong & Hu Haibo
22
DHP: Reduce the Number of Candidates
A k-itemset whose corresponding hashing bucket count is below the threshold cannot be frequent Candidates: a, b, c, d, e Hash entries: {ab, ad, ae} {bd, be, de} … Frequent 1-itemset: a, b, d, e ab is not a candidate 2-itemset if the sum of count of {ab, ad, ae} is below support threshold J. Park, M. Chen, and P. Yu. An effective hash-based algorithm for mining association rules. In SIGMOD’95 Frequent-pattern mining methods
23
Still challenging, the niche for DHP
DHP ( Park ’95 ): Dynamic Hashing and Pruning Candidate large 2-itemsets are huge. DHP: trim them using hashing Transaction database is huge that one scan per iteration is costly DHP: prune both number of transactions and number of items in each transaction after each iteration Frequent-pattern mining methods
24
Hash Table Construction
Consider two items sets, all itesms are numbered as i1, i2, …in. For any any pair (x, y), has according to Hash function bucket #= h({x y}) = ((order of x)*10+(order of y)) % 7 Example: Items = A, B, C, D, E, Order = , 2, 3 4, 5, H({C, E})= (3*10 + 5)% 7 = 0 Thus, {C, E} belong to bucket 0. Frequent-pattern mining methods
25
How to trim candidate itemsets
In k-iteration, hash all candidate k+1 itemsets in a hash table, and count all the itemsets in each bucket. In k+1 iteration, examine each of the candidate itemset to see if its correspondent bucket value is above the support ( necessary condition ) Frequent-pattern mining methods
26
Example TID Items 100 A C D 200 B C E 300 A B C E 400 B E
Figure1. An example transaction database Frequent-pattern mining methods
27
Generation of C1 & L1(1st iteration)
Itemset Sup {A} 2 {B} 3 {C} {D} 1 {E} C L1 Itemset Sup {A} 2 {B} 3 {C} {E} Frequent-pattern mining methods
28
Hash Table Construction
Find all 2-itemset of each transaction TID 2-itemset 100 {A C} {A D} {C D} 200 {B C} {B E} {C E} 300 {A B} {A C} {A E} {B C} {B E} {C E} 400 {B E} Frequent-pattern mining methods
29
Hash Table Construction (2)
Hash function h({x y}) = ((order of x)*10+(order of y)) % 7 Hash table {C E} {A E} {B C} {B E} {A B} {A C} {C E} {B C} {B E} {C D} {A D} {B E} {A C} bucket 3 1 2 Frequent-pattern mining methods
30
C2 Generation (2nd iteration)
C2 of Apriori {A B} {A C} {A E} {B C} {B E} {C E} L1*L1 # in the bucket {A B} 1 {A C} 3 {A E} {B C} 2 {B E} {C E} Resulted C2 {A C} {B C} {B E} {C E} Frequent-pattern mining methods
31
Effective Database Pruning
Apriori Don’t prune database. Prune Ck by support counting on the original database. DHP More efficient support counting can be achieved on pruned database. Frequent-pattern mining methods
32
Performance Comparison
Frequent-pattern mining methods
33
Performance Comparison (2)
Frequent-pattern mining methods
34
Frequent-pattern mining methods
FP-growth Algorithm Use a compressed representation of the database using an FP-tree Once an FP-tree has been constructed, it uses a recursive divide-and-conquer approach to mine the frequent itemsets Frequent-pattern mining methods
35
Frequent-pattern mining methods
FP-tree construction null After reading TID=1: A:1 B:1 After reading TID=2: null B:1 A:1 B:1 C:1 D:1 Frequent-pattern mining methods
36
Frequent-pattern mining methods
FP-Tree Construction Transaction Database null B:3 A:7 B:5 C:3 C:1 D:1 Header table D:1 C:3 E:1 D:1 E:1 D:1 E:1 D:1 Pointers are used to assist frequent itemset generation Frequent-pattern mining methods
37
Frequent-pattern mining methods
FP-growth null Conditional Pattern base for D: (PB | D) = {(A:1,B:1,C:1), (A:1,B:1), (A:1,C:1), (A:1), (B:1,C:1)} Recursively apply FP-growth on PB, and then append to D Thus, frequent Itemsets found from PB|D (with min support = 2): AD, BD, CD, ABD, ACD, BCD A:7 B:1 B:5 C:1 C:1 D:1 D:1 C:3 D:1 D:1 D:1 Frequent-pattern mining methods
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.