Download presentation
Presentation is loading. Please wait.
Published byDouglas Williams Modified over 9 years ago
2
Learning Statistical Models From Relational Data Lise Getoor University of Maryland, College Park Includes work done by: Nir Friedman, Hebrew U. Daphne Koller, Stanford Avi Pfeffer, Harvard Ben Taskar, Stanford Slides taken from the presentation (subset only)
3
Outline Motivation and Background PRMs w/ Attribute Uncertainty PRMs w/ Link Uncertainty PRMs w/ Class Hierarchies
4
Discovering Patterns in Structured Data Patient Treatment Strain Contact
5
Learning Statistical Models Traditional approaches –work well with flat representations –fixed length attribute-value vectors –assume independent (IID) sample Patient flatten Problems: –introduces statistical skew –loses relational structure incapable of detecting link-based patterns –must fix attributes in advance Contact
6
Probabilistic Relational Models Combine advantages of relational logic & Bayesian networks: –natural domain modeling: objects, properties, relations; –generalization over a variety of situations; –compact, natural probability models. Integrate uncertainty with relational model: –properties of domain entities can depend on properties of related entities; –uncertainty over relational structure of domain.
7
Relational Schema Author Good Writer Author of Has Review Describes the types of objects and relations in the database Review Paper Quality Accepted Mood Length Smart
8
Probabilistic Relational Model Length Mood Author Good Writer Paper Quality Accepted Review Smart
9
Probabilistic Relational Model Length Mood Author Good Writer Paper Quality Accepted Review Smart Paper.Review.Mood Paper.Quality, Paper.Accepted | P
10
Probabilistic Relational Model Length Mood Author Good Writer Paper Quality Accepted Review Smart 3.07.0 4.06.0 8.02.0 9.01.0,,,,, tt ft tf ff P(A | Q, M) MQ
11
Fixed relational skeleton : set of objects in each class relations between them Author A1 Paper P1 Author: A1 Review: R1 Review R2 Review R1 Author A2 Relational Skeleton Paper P2 Author: A1 Review: R2 Paper P3 Author: A2 Review: R2 Primary Keys Foreign Keys Review R2
12
Author A1 Paper P1 Author: A1 Review: R1 Review R2 Review R1 Author A2 PRM defines distribution over instantiations of attributes PRM w/ Attribute Uncertainty Paper P2 Author: A1 Review: R2 Paper P3 Author: A2 Review: R2 Good WriterSmart Length Mood Quality Accepted Length Mood Review R3 Length Mood Quality Accepted Quality Accepted Good WriterSmart
13
P2.Accepted P2.Quality r2.Mood P3.Accepted P3.Quality 3.07.0 4.06.0 8.02.0 9.01.0,,,,, tt ft tf ff P(A | Q, M) MQ Bad Low 3.07.0 4.06.0 8.02.0 9.01.0,,,,, tt ft tf ff P(A | Q, M) MQ r3.Mood A Portion of the BN
14
P2.Accepted P2.Quality r2.Mood P3.Accepted P3.Quality Bad Low r3.Mood High 3.07.0 4.06.0 8.02.0 9.01.0,,,,, tt ft tf ff P(A | Q, M) MQ Bad A Portion of the BN
15
P2.Accepted P2.Quality r2.Mood P3.Accepted P3.Quality 3.07.0 4.06.0 8.02.0 9.01.0,,,,, tt ft tf ff P(A | Q, M) MQ Pissy Low 3.07.0 4.06.0 8.02.0 9.01.0,,,,, tt ft tf ff P(A | Q, M) MQ A Portion of the BN
16
3.07.0 4.06.0 8.02.0 9.01.0,,,,, tt ft tf ff P(A | Q, M) MQ P2.Accepted P2.Quality r2.Mood P3.Accepted P3.Quality Pissy LowHigh A Portion of the BN
17
Length Mood Paper Quality Accepted Review Review R1 Length Mood Review R2 Length Mood Review R3 Length Mood Paper P1 Accepted Quality PRM: Aggregate Dependencies
18
sum, min, max, avg, mode, count Length Mood Paper Quality Accepted Review Review R1 Length Mood Review R2 Length Mood Review R3 Length Mood Paper P1 Accepted Quality mode 3.07.0 4.06.0 8.02.0 9.01.0,,,,, tt ft tf ff P(A | Q, M) MQ PRM: Aggregate Dependencies
19
PRM with AU Semantics Attributes Objects probability distribution over completions I: PRM relational skeleton += Author Paper Review Author A1 Paper P2 Paper P1 Review R3 Review R2 Review R1 Author A2 Paper P3
20
Learning PRMs w/ AU Database Patient Strain Contact Relational Schema Patient Contact Strain Parameter estimation Structure selection
21
Parameter Estimation in PRMs Assume known dependency structure S Goal: estimate PRM parameters –entries in local probability models, is good if it is likely to generate the observed data, instance I. MLE Principle: Choose so as to maximize l
22
Learning PRMs w/ AU Database Paper Author Review Relational Schema Paper Review Author Parameter estimation Structure selection
23
Paper Quality Accepted Review Mood Length where is the number of accepted, low quality papers whose reviewer was in a poor mood,,,,, tt ft tf ff P(A | Q, M) MQ ? ? ? ? ? ? ? ? ML Parameter Estimation
24
Paper Quality Accepted Review Mood Length ,,,,, tt ft tf ff P(A | Q, M) MQ ? ? ? ? ? ? ? ? Count Query for counts: Review table Paper table ML Parameter Estimation
25
Structure Selection Idea: –define scoring function –do local search over legal structures Key Components: –legal models –scoring models –searching model space
26
Structure Selection Idea: –define scoring function –do local search over legal structures Key Components: »legal models –scoring models –searching model space
27
Legal Models author-of PRM defines a coherent probability model over a skeleton if the dependencies between object attributes are acyclic (prop. BN). How do we guarantee that a PRM is acyclic for every skeleton? Researcher Prof. Gump Reputation high Paper P1 Accepted yes Paper P2 Accepted yes sum
28
Attribute Stratification PRM dependency structure S dependency graph Paper.Accecpted Researcher.Reputation if Researcher.Reputation depends directly on Paper.Accepted dependency graph acyclic acyclic for any Attribute stratification: Algorithm more flexible; allows certain cycles along guaranteed acyclic relations
29
Blood Type M-chromosome P-chromosome Person Result Contaminated Blood Test Blood Type M-chromosome P-chromosome Person Blood Type M-chromosome P-chromosome Person (Father) (Mother)
30
Structure Selection Idea: –define scoring function –do local search over legal structures Key Components: –legal models »scoring models –searching model space
31
Scoring Models Bayesian approach: Standard approach to scoring models; used in Bayesian network learning
32
Structure Selection Idea: –define scoring function –do local search over legal structures Key Components: –legal models –scoring models »searching model space
33
Searching Model Space Review Author Paper score Delete R.M R.L Review Author Paper score Add A.S A.W Author Review Paper Phase 0: consider only dependencies within a class
34
Review Author Paper score Add A.S P.A score Add P.A R.M Review Author Paper Review Paper Author Phase 1: consider dependencies from “neighboring” classes, via schema relations Phased Structure Search
35
score Add A.S R.M score Add R.M A.W Phase 2: consider dependencies from “further” classes, via relation chains Review Author Paper Review Author Paper Review Author Paper
36
Issue PRM w/ AU applicable only in domains where we have full knowledge of the relational structure Next we introduce PRMs which allow uncertainty over relational structure…
37
Kinds of structural uncertainty How many objects does an object relate to? –how many Authors does Paper1 have? Which object is an object related to? –does Paper1 cite Paper2 or Paper3? Which class does an object belong to? –is Paper1 a JournalArticle or a ConferencePaper? Does an object actually exist? Are two objects identical?
38
Structural Uncertainty Motivation: PRM with AU only well-defined when the skeleton structure is known May be uncertain about relational structure itself Construct probabilistic models of relational structure that capture structural uncertainty Mechanisms: –Reference uncertainty –Existence uncertainty –Number uncertainty –Type uncertainty –Identity uncertainty
39
PRMs w/ Link Uncertainty Advantages: –Applicable in cases where we do not have full knowledge of relational structure –Incorporating uncertainty over relational structure into probabilistic model can improve predictive accuracy Two approaches: –Reference uncertainty –Existence uncertainty Different probabilistic models; varying amount of background knowledge required for each
40
Citation Relational Schema Wrote Paper Topic Word1 WordN … Word2 Paper Topic Word1 WordN … Word2 Cites Count Citing Paper Cited Paper Author Institution Research Area
41
Attribute Uncertainty Paper Word1 Topic WordN Wrote Author... Research Area P( WordN | Topic) P( Topic | Paper.Author.Research Area Institution P( Institution | Research Area)
42
Reference Uncertainty Bibliography Scientific Paper ` 1. ----- 2. ----- 3. ----- ? ? ? Document Collection
43
PRM w/ Reference Uncertainty Cites Cited Citing Dependency model for foreign keys Paper Topic Words Paper Topic Words Naïve Approach: multinomial over primary key noncompact limits ability to generalize
44
Reference Uncertainty Example Paper P5 Topic AI Paper P4 Topic AI Paper P3 Topic AI Paper M2 Topic AI Paper P1 Topic Theory Cites Citing Cited Paper P5 Topic AI Paper P3 Topic AI Paper P4 Topic Theory Paper P2 Topic Theory Paper P1 Topic Theory Paper.Topic = AI Paper.Topic = Theory C1 C2 C1 C2 3.07.0
45
Reference Uncertainty Example Paper P5 Topic AI Paper P4 Topic AI Paper P3 Topic AI Paper M2 Topic AI Paper P1 Topic Theory Cites Citing Cited Paper P5 Topic AI Paper P3 Topic AI Paper P4 Topic Theory Paper P2 Topic Theory Paper P1 Topic Theory Paper.Topic = AI Paper.Topic = Theory P1 P2 Paper Topic Words P1 P2 3.07.0 P1 P2 1.09.0 Cited.Topic 99.001.0 Theory AI
46
Introduce Selector RVs Cites1.Cited Cites1.Selector P1.Topic P2.Topic P3.Topic P4.Topic P5.Topic P6.Topic Cites2.Cited Cites2.Selector Introduce Selector RV, whose domain is {C1,C2} The distribution over Cited depends on all of the topics, and the selector
47
PRMs w/ RU Semantics PRM-RU + entity skeleton probability distribution over full instantiations I Cites Cited Citing Paper Topic Words Paper Topic Words PRM RU Paper P5 Topic AI Paper P4 Topic Theory Paper P2 Topic Theory Paper P3 Topic AI Paper P1 Topic ??? Paper P5 Topic AI Paper P4 Topic Theory Paper P2 Topic Theory Paper P3 Topic AI Paper P1 Topic ??? Reg Cites entity skeleton
48
Learning PRMs w/ RU Idea: just like in PRMs w/ AU –define scoring function –do greedy local structure search Issues: –expanded search space construct partitions new operators
49
Learning Idea: –define scoring function –do phased local search over legal structures Key Components: –legal models –scoring models –searching model space PRMs w/ RU model new dependencies new operators unchanged
50
Legal Models Cites Citing Cited Mood Paper Important Accepted Review Paper Important Accepted
51
Legal Models P1.Accepted When a node’s parent is defined using an uncertain relation, the reference RV must be a parent of the node as well. Cites1.Cited Cites1.Selector R1.Mood P2.Important P3.Important P4.Important
52
Structure Search Cites Citing Cited Paper Topic Words Paper Topic Words Cited Papers 1.0 Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Author Institution
53
Structure Search: New Operators Cites Citing Cited Paper Topic Words Paper Topic Words Cited Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Topic Δscore Refine on Topic Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Author Institution
54
Structure Search: New Operators Cites Cited Citing Paper Topic Words Paper Topic Words Citing Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Topic = AI Δscore Refine on Topic Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Δscore Refine on Author.Instition Author Institution Institution = MIT
55
PRMs w/ RU Summary Define semantics for uncertainty over foreign-key values Search now includes operators Refine and Abstract for constructing foreign-key dependency model Provides one simple mechanism for link uncertainty
56
Existence Uncertainty Document Collection ? ? ?
57
PRM w/ Exists Uncertainty Cites Dependency model for existence of relationship Paper Topic Words Paper Topic Words Exists
58
Exists Uncertainty Example Cites Paper Topic Words Paper Topic Words Exists Citer.Topic Cited.Topic 0.9950.005 Theory FalseTrue AI Theory0.9990.001 AI 0.9920.008 AI Theory0.9970.003
59
Paper#2 Topic Paper#3 Topic WordN Paper#1 Word1 Topic... Author #1 Area Inst #1-#2 Author #2 Area Inst Exists #2-#3 Exists #2-#1 Exists #3-#1 Exists #1-#3 Exists WordN Word1 WordN Word1 Exists #3-#2 Introduce Exists RVs
60
Paper#2 Topic Paper#3 Topic WordN Paper#1 Word1 Topic... Author #1 Area Inst #1-#2 Author #2 Area Inst Exists #2-#3 Exists #2-#1 Exists #3-#1 Exists #1-#3 Exists WordN Word1 WordN Word1 Exists WordN Word1 WordN Word1 WordN Word1 Exists #3-#2 Introduce Exists RVs
61
PRMs w/ EU Semantics PRM-EU + object skeleton probability distribution over full instantiations I Paper P5 Topic AI Paper P4 Topic Theory Paper P2 Topic Theory Paper P3 Topic AI Paper P1 Topic ??? Paper P5 Topic AI Paper P4 Topic Theory Paper P2 Topic Theory Paper P3 Topic AI Paper P1 Topic ??? object skeleton ??? PRM EU Cites Exists Paper Topic Words Paper Topic Words
62
Learning PRMs w/ EU Idea: just like in PRMs w/ AU –define scoring function –do greedy local structure search Issues: –efficiency Computation of sufficient statistics for exists attribute Do not explicitly consider relations that do not exist
63
Structure Selection Idea: –define scoring function –do phased local search over legal structures Key Components: –legal models –scoring models –searching model space PRMs w/ EU model new dependencies unchanged
64
Results
65
PRMs w/ Class Hierarchies Allows us to: Refine a “heterogenous” class into more coherent subclasses Refine probabilistic model along class hierarchy –Can specialize/inherit CPDs –Construct new dependencies that were originally “acyclic” Provides bridge from class-based model to instance-based model
66
Learning PRM-CHs Relational Schema Database: TVProgram Person Vote Person Vote TVProgram Instance I Class hierarchy provided Learn class hierarchy
67
Journal Topic Quality Accepted Conf-Paper Topic Quality Accepted Journal.Accepted Conf-Paper.Accepted Paper Topic Quality Accepted Paper.Accepted Paper.Class Guaranteeing Acyclicity w/ Subclasses
68
Learning PRM-CH Scenario 1: Class hierarchy is provided New Operators Specialize/Inherit Accepted Paper Accepted Journal Accepted Conference Accepted Workshop
69
Learning Class Hierarchy Issue: partially observable data set Construct decision tree for class defined over attributes observed in training set Paper.Venue conference workshop class1 class3 journal class2 class4 high Paper.Author.Fame class5 medium class6 low New operator Split on class attribute Related class attribute
70
PRM-CH Summary PRMs with class hierarchies are a natural extension of PRMs: –Specialization/Inheritance of CPDs –Allows new dependency structures Provide bridge from class-based to instance- based models Learning techniques proposed –Need efficient heuristics –Empirical validation on real-world domains
71
Conclusions PRMs can represent distribution over attributes from multiple tables PRMs can capture link uncertainty PRMs allow inferences about individuals while taking into account relational structure (they do not make inappropriate independence assumptions)
72
Selected Publications “Learning Probabilistic Models of Link Structure”, L. Getoor, N. Friedman, D. Koller and B. Taskar, JMLR 2002. “Probabilistic Models of Text and Link Structure for Hypertext Classification”, L. Getoor, E. Segal, B. Taskar and D. Koller, IJCAI WS ‘Text Learning: Beyond Classification’, 2001. “Selectivity Estimation using Probabilistic Models”, L. Getoor, B. Taskar and D. Koller, SIGMOD-01. “Learning Probabilistic Relational Models”, L. Getoor, N. Friedman, D. Koller, and A. Pfeffer, chapter in Relation Data Mining, eds. S. Dzeroski and N. Lavrac, 2001. –see also N. Friedman, L. Getoor, D. Koller, and A. Pfeffer, IJCAI-99. “Learning Probabilistic Models of Relational Structure”, L. Getoor, N. Friedman, D. Koller, and B. Taskar, ICML-01. “From Instances to Classes in Probabilistic Relational Models”, L. Getoor, D. Koller and N. Friedman, ICML Workshop on Attribute-Value and Relational Learning: Crossing the Boundaries, 2000. Notes from AAAI Workshop on Learning Statistical Models from Relational Data, eds. L.Getoor and D. Jensen, 2000. Notes from IJCAI Workshop on Learning Statistical Models from Relational Data, eds. L.Getoor and D. Jensen, 2003. See http://www.cs.umd.edu/~getoor
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.