Sampling Large Databases for Association Rules Jingting Zeng CIS 664 Presentation March 13, 2007

Association Rules Outline uAssociation Rules Problem Overview uAssociation Rules Definitions uPrevious Work on Association Rules uToivonen’s Algorithm uExperiments Result uConclusion

Overview uPurpose If people tend to buy A and B together, then a buyer of A is a good target for an advertisement for B.

The Market-Basket Example uItems frequently purchased together: Bread  PeanutButter uUses: wPlacement wAdvertising wSales wCoupons uObjective: increase sales and reduce costs

Other Example uThe same technology has other uses University course enrollment data has been analyzed to find combinations of courses taken by the same students

Scale of Problem uWalMart sells 100,000 items and can store hundreds of millions of baskets. uThe Web has 100,000,000 words and several billion pages.

Association Rule Definitions uSet of items: I={I 1,I 2,…,I m }  Transactions: D={t 1,t 2, …, t n }, t j  I uSupport of an itemset: Percentage of transactions which contain that itemset. uFrequent itemset: Itemset whose number of occurrences is above a threshold.

Association Rule Definitions  Association Rule (AR): implication X  Y where X,Y  I and X  Y = ;  Support of AR (s) X  Y: Percentage of transactions that contain X  Y  Confidence of AR (  ) X  Y: Ratio of number of transactions that contain X  Y to the number that contain X

Example B1 = {m, c, b}B2 = {m, p, j} B3 = {m, b} B4 = {c, j} B5 = {m, p, b}B6 = {m, c, b, j} B7 = {c, b, j}B8 = {b, c} uAssociation Rule w{m, b}  c wSupport = 2/8 = 25% wConfidence = 2/4 = 50%

Association Rule Problem  Given a set of items I={I 1,I 2,…,I m } and a database of transactions D={t 1,t 2, …, t n } where t i ={I i1,I i2, …, I ik } and I ij  I, the Association Rule Problem is to identify all association rules X  Y with a minimum support and confidence threshold.

Association Rule Techniques uFind all frequent itemsets uGenerate strong association rules from the frequent itemsets

APriori Algorithm uA two-pass approach called a-priori limits the need for main memory. uKey idea: monotonicity : if a set of items appears at least s times, so does every subset. wConverse for pairs: if item i does not appear in s baskets, then no pair including i can appear in s baskets.

APriori Algorithm (contd.) uPass 1: Read baskets and count in main memory the occurrences of each item. wRequires only memory proportional to #items. uPass 2: Read baskets again and count in main memory only those pairs both of which were found in Pass 1 to have occurred at least s times. wRequires memory proportional to square of frequent items only.

Partitioning uDivide database into partitions D 1,D 2,…,D p uApply Apriori to each partition uAny large itemset must be large in at least one partition.

Partitioning Algorithm 1.Divide D into partitions D 1,D 2,…,D p; 2.For I = 1 to p do 3. L i = Apriori(D i ); 4.C = L 1  …  L p ; 5.Count C on D to generate L;

Sampling uLarge databases uSample the database and apply Apriori to the sample. uPotentially Frequent Itemsets (PL): Large itemsets from sample uNegative Border (BD - ): wGeneralization of Apriori-Gen applied to itemsets of varying sizes. wMinimal set of itemsets which are not in PL, but whose subsets are all in PL.

Negative Border Example Let Items = {A,…,F} and there are itemsets: {A}, {B}, {C}, {F}, {A,B}, {A,C}, {A,F}, {C,F}, {A,C,F} The whole negative border is: {{B,C}, {B,F}, {D}, {E}}

Toivonen’s Algorithm uStart as in the simple algorithm, but lower the threshold slightly for the sample. wExample: if the sample is 1% of the baskets, use as the support threshold rather than wGoal is to avoid missing any itemset that is frequent in the full set of baskets.

Toivonen’s Algorithm (contd.) uAdd to the itemsets that are frequent in the sample the negative border of these itemsets. uAn itemset is in the negative border if it is not deemed frequent in the sample, but all its immediate subsets are. wExample: ABCD is in the negative border if and only if it is not frequent, but all of ABC, BCD, ACD, and ABD are.

Toivonen’s Algorithm (contd.) uIn a second pass, count all candidate frequent itemsets from the first pass, and also count the negative border. uIf no itemset from the negative border turns out to be frequent, then the candidates found to be frequent in the whole data are exactly the frequent itemsets.

Toivonen’s Algorithm (contd.) uWhat if we find something in the negative border is actually frequent? uWe must start over again! uBut by choosing the support threshold for the sample wisely, we can make the probability of failure low, while still keeping the number of itemsets checked on the second pass low enough for main-memory.

Experiment Synthetic data set characteristics (T = row size on average, I = size of maximal frequent sets on average)

Experiment (contd.) Lowered frequency thresholds (%) for probability of missing any given frequent set is less than δ = 0.001

Number of trials with misses

Conclusions uAdvantages: Reduced failure probability, while keeping candidate-count low enough for memory uDisadvantages: Potentially large number of candidates in second pass

Thank you!