Presentation is loading. Please wait.

Presentation is loading. Please wait.

Date: 2012/07/02 Source: Marina Drosou, Evaggelia Pitoura (CIKM’11) Speaker: Er-Gang Liu Advisor: Dr. Jia-ling Koh 1.

Published byMelinda Mathews Modified over 9 years ago

Similar presentations

Presentation on theme: "Date: 2012/07/02 Source: Marina Drosou, Evaggelia Pitoura (CIKM’11) Speaker: Er-Gang Liu Advisor: Dr. Jia-ling Koh 1."— Presentation transcript:

1 Date: 2012/07/02 Source: Marina Drosou, Evaggelia Pitoura (CIKM’11) Speaker: Er-Gang Liu Advisor: Dr. Jia-ling Koh 1

2 Outline Introduction The ReDRIVE framework FaSets Interesting faSets Top-k faSets computation Recommendations Statistics maintenance Two-Phase algorithm Experiment Conclusion 2

3 Outline Introduction The ReDRIVE framework FaSets Interesting faSets Top-k faSets computation Recommendation Statistics maintenance Two-Phase algorithm Experiment Conclusion 3

4 4 Introduction - Motivation User Database (EX ： IMDB) Not knowing the exact content of the database Query search

5 5 Show me movies directed by F.F. Coppola DirectorTitleYearGenre F.F. CoppolaTetro2009Drama F.F. CoppolaYouth Without Youth2007Fantasy F.F. CoppolaThe Godfather1972Drama F.F. CoppolaRumble Fish1983Drama F.F. CoppolaThe Conversation1974Thriller F.F. CoppolaThe Outsiders1983Drama F.F. CoppolaSupernova2000Thriller F.F. CoppolaApocalypse Now1979Drama Query Result Introduction - Motivation No clear understanding of information needs Users interact with databases by formulating queries

6 6 SELECT title, year, genre FROM movies, directors, genres WHERE director = ‘F.F. Coppola’ AND join(Q) SELECT director FROM movies, directors, genres WHERE year = 1983 AND genre = ‘Drama’ AND join(Q) Query 1 1 Query Result 2 2 Recommendation 3 3 Explorator Query 4 4 Introduction - Goal DirectorTitleYearGenre F.F. CoppolaTetro2009Drama F.F. CoppolaYouth Without Youth2007Fantasy F.F. CoppolaThe Godfather1972Drama F.F. CoppolaRumble Fish1983Drama F.F. CoppolaThe Conversation1974Thriller F.F. CoppolaThe Outsiders1983Drama F.F. CoppolaSupernova2000Thriller F.F. CoppolaApocalypse Now1979Drama Recommendation Drama Drama, 2009 Drama, 1983 Thriller Thriller, 1974 Fantasy Fantasy, 2007 Fantasy, 2007, Youth Without Youth Interesting faSet

7 Outline Introduction The ReDRIVE framework FaSets Interesting faSets Top-k faSets computation Recommendation Statistics maintenance Two-Phase algorithm Experiment Conclusion 7

8 FaSets Facet condition: A condition A i = a i on some attribute of Res(Q) m-FaSet: A set of m facet conditions on m different attributes of Res(Q) 8 DirectorTitleYearGenre F.F. CoppolaTetro2009Drama F.F. CoppolaYouth Without Youth2007Fantasy F.F. CoppolaThe Godfather1972Drama F.F. CoppolaRumble Fish1983Drama F.F. CoppolaThe Conversation1974Thriller F.F. CoppolaThe Outsiders1983Drama F.F. CoppolaSupernova2000Thriller F.F. CoppolaApocalypse Now1979Drama 1-faSet 2-faSet

9 Interestingness score of a FaSet 9 Support of f in Res(Q) Support of f in the database DirectorTitleYearGenre F.F. CoppolaTetro2009Drama F.F. CoppolaYouth Without Youth2007Fantasy F.F. CoppolaThe Godfather1972Drama F.F. CoppolaRumble Fish1983Drama F.F. CoppolaThe Conversation1974Thriller F.F. CoppolaThe Outsiders1983Drama F.F. CoppolaSupernova2000Thriller F.F. CoppolaApocalypse Now1979Drama = 125 = 500 Query Result Score ( f, Q = “F.F. Coppola” ) DB “Drama” : 50 “Thriller” : 5 All tuple: 10000

10 Outline Introduction The ReDRIVE framework FaSets Interesting faSets Top-k faSets computation Recommendation Statistics maintenance Two-Phase algorithm Experiment Conclusion 10

11 Top-k faSets computation To compute the interestingness score of a faSet : p(f |Res(Q)) p(f |D) p(f |Res(Q)) is computed on-line p(f |D) is too expensive ⇒ must be estimated Compute off-line and store statistics that will allow us to estimate p(f |D) for any faSet f. FaSets that appear frequently in the database D are not expected to be interesting. 11

12 It is useful to maintain information about the support of “rare faSets” in D. In correspondence to Data Mining, paper define: Rare faSet (RF) : A faSet with frequency under a threshold Closed Rare faSet (CRF) : A rare faSet with no proper subset with the same frequency Minimal Rare faSet (MRF) : A rare faSet with no rare subset |MRFs| ≤ |CRFs| ≤ |RFs| MRFs can tell us if f is rare but not its frequency CRFs can tell us its frequency but are still too many 12 Estimating p(f |D)

13 13

14 14 Rare faSet (RF) : A faSet with frequency under a threshold Minimal Rare faSet (MRF) : A rare faSet with no rare subset ab : a,b acd: ac,ad,cd ade: ad,de,ae

15 15 abd(1) : ab(2), ad(2), bd(2) bde(0): bd(1),be(1),de(2) bcde(0): bcd(1),bce(1), bde(0),cde(1) Closed Rare faSet (CRF) : A rare faSet with no proper subset with the same frequency Not Closed Rare faSet

16 Statistics Maintaining statistics in the form of -Tolerance Closed Rare FaSets (-CRFs): A faSet f is an -CRF for a set of tuples S if and only if: it is rare for S it has no proper rare subset f’, |f’ |=|f |-1, such that: count(f’,S) < (1+ )count(f,S), ≥ 0 16

17 Outline Introduction The ReDRIVE framework FaSets Interesting faSets Top-k faSets computation Recommendation Statistics maintenance Two-Phase algorithm Experiment Conclusion 17

18 The Two-Phase Algorithm (1/3) Maintain all -CRFs, where rare is defined by minsupp r First Phase: X = {all 1-faSets in Res(Q)} Y = {-CRFs that consist only of 1-faSets in X} 18 DirectorTitleYearGenre F.F. CoppolaTetro2009Drama F.F. CoppolaYouth Without Youth2007Fantasy F.F. CoppolaThe Godfather1972Drama F.F. CoppolaRumble Fish1983Drama F.F. CoppolaThe Conversation1974Thriller F.F. CoppolaThe Outsiders1983Drama F.F. CoppolaSupernova2000Thriller F.F. CoppolaApocalypse Now1979Drama 1-faSet Drama Fantasy Thriller 2009 2007 1972.... Query Result X -CRFs Drama : 50 Thriller : 5...... Collection of maintained Statistics Drama Thiller 2007...... Y

19 The Two-Phase Algorithm (2/3) Maintain all -CRFs, where rare is defined by minsupp r First Phase: Y = {-CRFs that consist only of 1-faSets in X} Z = {faSets in Res(Q) that are supersets of some faSet in Y} Compute scores for faSets in Z 19 DirectorTitleYearGenre F.F. CoppolaTetro2009Drama F.F. CoppolaSupernova2000Thriller Query Result Drama Thiller 2007.... Y...... Z...... { 2009, Drama } { Tetro, 2009, Drama } { 2000, Thriller} {Supernova, 2000, Thriller } { 2009, Drama } { Tetro, 2009, Drama } { 2000, Thriller} {Supernova, 2000, Thriller }

20 The Two-Phase Algorithm (3/3) Let f be a faSet examined in the second phase. This means that p(f |D) > minsupp r Second Phase: Reset the threshold minsupp f by minsupp r Executing a frequent itemset mining algorithm (A-priori) with threshold minsupp f = s * minsupp r (s = k th highest score in Z ) 20 DirectorTitleYearGenre F.F. CoppolaTetro2009Drama F.F. CoppolaYouth Without Youth2007Fantasy F.F. CoppolaThe Godfather1972Drama F.F. CoppolaRumble Fish1983Drama F.F. CoppolaThe Conversation1974Thriller F.F. CoppolaThe Outsiders1983Drama F.F. CoppolaSupernova2000Thriller F.F. CoppolaApocalypse Now1979Drama Query Result “frequent itemset” and “p(f |Res(Q)) > minsupp f ”.... { 2009, Drama } { Tetro, 2009, Drama } { 2000, Thriller} {Supernova, 2000, Thriller } Top K

21 Outline Introduction The ReDRIVE framework FaSets Interesting faSets Top-k faSets computation Recommendation Statistics maintenance Two-Phase algorithm Experiment Conclusion 21

22 Experiment - Datasets Experimenting using real datasets: AUTOS: single-relation, 15191 tuples, 41 attributes MOVIES: 13 relations, 10,000 ~ 1,000,000 tuples, 2 ～ 5 attributes And synthetic ones: ZIPF: single relation, 1000 tuples, 5 attributes 22

23 Experiment Generation 23

24 Top-k faSets discovery Baseline: Consider only frequent faSets in Res(Q) TPA: Two-Phase Algorithm 24

25 Conclusion Introducing ReDRIVE, a novel database exploration framework for recommending to users items which may be of interest to them although not part of the results of their original query Proposing a frequency estimation method based on - CRFs Proposing a Two-Phase Algorithm for locating the top-k most interesting faSets 25

26 26 δ= 0.04 “abcd” is the closest δ-TCFI superset of all its subsets that contain the item “a” “bcd” is the closest δ-TCFI superset of “bc”, “cd” and “c” let Y = abcd, then X1 = {abc, abd, acd}, X2 = {ab, ac, ad} and X3 = {a}.

27 27 the frequency of “abc”, “abd”, “acd” are estimated : (freq(abcd) ・ ext(abcd, 1)) = 100 * 1.03 = 103, the frequency of “ab”, “ac”, “ad” are estimated : : (freq(abcd) ・ ext (abcd, 2)) = 107 frequency of “a” is estimated : (freq(abcd) ・ ext(abcd, 3)) = 111

Download ppt "Date: 2012/07/02 Source: Marina Drosou, Evaggelia Pitoura (CIKM’11) Speaker: Er-Gang Liu Advisor: Dr. Jia-ling Koh 1."

Similar presentations

Association Rule Mining

Association Rule Mining
Recap: Mining association rules from large datasets

Recap: Mining association rules from large datasets
Salvatore Ruggieri SIGKDD2010 Frequent Regular Itemset Mining 2010/9/2 1.

Salvatore Ruggieri SIGKDD2010 Frequent Regular Itemset Mining 2010/9/2 1.
A Phrase Mining Framework for Recursive Construction of a Topical Hierarchy Date ： 2014/04/15 Source ： KDD’13 Authors ： Chi Wang, Marina Danilevsky, Nihit.

A Phrase Mining Framework for Recursive Construction of a Topical Hierarchy Date ： 2014/04/15 Source ： KDD’13 Authors ： Chi Wang, Marina Danilevsky, Nihit.
Frequent Closed Pattern Search By Row and Feature Enumeration

Frequent Closed Pattern Search By Row and Feature Enumeration
LOGO Association Rule Lecturer: Dr. Bo Yuan

LOGO Association Rule Lecturer: Dr. Bo Yuan
Association Rule Mining. 2 The Task Two ways of defining the task General –Input: A collection of instances –Output: rules to predict the values of any.

Association Rule Mining. 2 The Task Two ways of defining the task General –Input: A collection of instances –Output: rules to predict the values of any.
1 Department of Information & Computer Education, NTNU SmartMiner: A Depth First Algorithm Guided by Tail Information for Mining Maximal Frequent Itemsets.

1 Department of Information & Computer Education, NTNU SmartMiner: A Depth First Algorithm Guided by Tail Information for Mining Maximal Frequent Itemsets.
Association Rule Mining. Mining Association Rules in Large Databases  Association rule mining  Algorithms Apriori and FP-Growth  Max and closed patterns.

Association Rule Mining. Mining Association Rules in Large Databases  Association rule mining  Algorithms Apriori and FP-Growth  Max and closed patterns.
FP (FREQUENT PATTERN)-GROWTH ALGORITHM ERTAN LJAJIĆ, 3392/2013 Elektrotehnički fakultet Univerziteta u Beogradu.

FP (FREQUENT PATTERN)-GROWTH ALGORITHM ERTAN LJAJIĆ, 3392/2013 Elektrotehnički fakultet Univerziteta u Beogradu.
Sampling Large Databases for Association Rules ( Toivenon’s Approach, 1996) Farzaneh Mirzazadeh Fall 2007.

Sampling Large Databases for Association Rules ( Toivenon’s Approach, 1996) Farzaneh Mirzazadeh Fall 2007.
Data Mining Techniques So Far: Cluster analysis K-means Classification Decision Trees J48 (C4.5) Rule-based classification JRIP (RIPPER) Logistic Regression.

Data Mining Techniques So Far: Cluster analysis K-means Classification Decision Trees J48 (C4.5) Rule-based classification JRIP (RIPPER) Logistic Regression.
Data Mining Association Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 6 Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach,

Data Mining Association Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 6 Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach,
Data Mining Association Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 6 Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach,

Data Mining Association Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 6 Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach,
Data Mining Association Analysis: Basic Concepts and Algorithms

Data Mining Association Analysis: Basic Concepts and Algorithms
Data Mining Association Analysis: Basic Concepts and Algorithms

Data Mining Association Analysis: Basic Concepts and Algorithms
Data Mining Association Analysis: Basic Concepts and Algorithms

Data Mining Association Analysis: Basic Concepts and Algorithms
Is Sampling Useful in Data Mining? A Case in the Maintenance of Discovered Association Rules S.D. Lee, D. W. Cheung, B. Kao Department of Computer Science.

Is Sampling Useful in Data Mining? A Case in the Maintenance of Discovered Association Rules S.D. Lee, D. W. Cheung, B. Kao Department of Computer Science.
Intelligent Information Directory System for Clinical Documents Qinghua Zou 6/3/2005 Dr. Wesley W. Chu (Advisor)

Intelligent Information Directory System for Clinical Documents Qinghua Zou 6/3/2005 Dr. Wesley W. Chu (Advisor)
Association Rule Mining - MaxMiner. Mining Association Rules in Large Databases  Association rule mining  Algorithms Apriori and FP-Growth  Max and.

Association Rule Mining - MaxMiner. Mining Association Rules in Large Databases  Association rule mining  Algorithms Apriori and FP-Growth  Max and.

Similar presentations

Ads by Google