Download presentation
Presentation is loading. Please wait.
Published byMatilda Collins Modified over 9 years ago
1
1 Lifted First-Order Probabilistic Inference Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir and Dan Roth
2
Page 2 Motivation We want to be able to do inference with Probabilistic First-order Logic if epidemic(Disease) then sick(Person, Disease) [0.2, 0.05] if sick(Person, Disease) then hospital(Person) [0.3, 0.01] sick(bob, measles) or sick(bob, flu) 0.6 Expressiveness of logic Robustness of probabilistic models Goal: Probabilistic Inference at First-order level (scalability one of main advantages)
3
Page 3 Current approaches - Propositionalization Exact inference: Knowledge-based construction (Breese, 1992) Probabilistic Logic Programming (Ng & Subrahmanian, 1992) Probabilistic Logic Programming (Ngo and Haddawy, 1995) Probabilistic Relational Models (Friendman et al., 1999) Relational Bayesian networks (Jaeger, 1997) Bayesian Logic Programs (Kersting & DeRaedt, 2001) Probabilistic Abduction (Poole, 1993) MEBN (Laskey, 2004) Markov Logic (Richardson & Domingos, 2004) Sampling: PRISM (Saito, 1995) BLOG (Milch et al, 2005)
4
Page 4 Unexploited structure (epidemic(Disease1), epidemic(Disease2)) epidemic(measles) epidemic(flu) epidemic(rubella) … propositionalized epidemic(D2) epidemic(D1) D1 D2 first-order as in first-order theorem proving
5
Page 5 This Talk Representation Inference Inversion Elimination (IE) (Poole, 2003) Formalization of IE Identification of conditions for IE Counting Elimination Experiment & Conclusions
6
Page 6 Representation sick(mary,measles) hospital(mary) epidemic(measles)epidemic(flu) sick(mary,flu) … … sick(bob,measles) hospital(bob) sick(bob,flu) …… … …… ……
7
Page 7 Representation sick(mary,measles) hospital(mary) epidemic(measles)epidemic(flu) sick(mary,flu) … … sick(bob,measles) hospital(bob) sick(bob,flu) …… … …… …… sick(mary,measles), epidemic(measles))
8
Page 8 Representation sick(mary,measles) hospital(mary) epidemic(measles)epidemic(flu) sick(mary,flu) … … sick(bob,measles) hospital(bob) sick(bob,flu) …… … …… ……
9
Page 9 Representation - Lots of Redundancy! sick(mary,measles) hospital(mary) epidemic(measles)epidemic(flu) sick(mary,flu) … … sick(bob,measles) hospital(bob) sick(bob,flu) …… … …… ……
10
Page 10 Representing structure sick(mary,measles) epidemic(measles)epidemic(flu) sick(mary,flu) … … sick(bob,measles)sick(bob,flu) …… …… sick(P,D) epidemic(D) Poole (2003) named these parfactors, for “parameterized factors”
11
Page 11 Parfactor sick(Person,Disease) epidemic(Disease) 8 Person, Disease sick(Person,Disease), epidemic(Disease))
12
Page 12 Parfactor sick(Person,Disease) epidemic(Disease) 8 Person, Disease sick(Person,Disease), epidemic(Disease)), Person mary, Disease flu
13
Page 13 Representing structure sick(P,D) hospital(P) epidemic(D) More intutive More compact Represents structure explicitly Generalization of graphical models Atoms represent a set of random variables
14
Page 14 Making use of structure in Inference Task: given a condition, what is the marginal probability of a set of random variables? P(sick(bob, measles) | sick(mary,measles)) = ? Three approaches plain propositionalization dynamic construction (“smart” propositionalization) lifted inference
15
Page 15 Inference - Plain Propositionalization Instantiation of potential function for each instantiation Lots of redundant computation Lots of unnecessary random variables
16
Page 16 Inference - Dynamic construction Instantiation of potential function for relevant parts P(hospital(mary) | sick(mary, measles)) = ? sick(mary,measles) hospital(mary) sick(mary, flu) … epidemic(measles)epidemic(flu) sick(mary, rubella) epidemic(rubella) …
17
Page 17 Inference - Dynamic construction Instantiation of potential function for relevant parts P(hospital(mary) | sick(mary, measles)) = ? sick(mary,measles) hospital(mary) sick(mary, flu) … epidemic(measles)epidemic(flu) sick(mary, rubella) epidemic(rubella) … Much redundancy still
18
Page 18 Inference - Dynamic construction Most common approach for exact First-order Probabilistic inference: Knowledge-based construction (Breese, 1992) Probabilistic Logic Programming (Ng & Subrahmanian, 1992) Probabilistic Logic Programming (Ngo and Haddawy, 1995) Probabilistic Relational Models (Friendman et al., 1999) Relational Bayesian networks (Jaeger, 1997) Bayesian Logic Programs (Kersting & DeRaedt, 2001) MEBN (Laskey, 2004) Markov Logic (Richardson & Domingos, 2004)
19
Page 19 Inference - Lifted inference Inference on parameterized, or first-order, level; Performs certain inference steps once for a class of random variables; Poole (2003) describes a generalized Variable Elimination algorithm which we call Inversion Elimination.
20
Page 20 Inference - Inversion Elimination (IE) P(hospital(mary) | sick(mary, measles)) = ? hospital(mary) sick(mary, D) epidemic(D)
21
Page 21 Inference - Inversion Elimination (IE) P(hospital(mary) | sick(mary, measles)) = ? hospital(mary) sick(mary, D) epidemic(D) = Unification
22
Page 22 Inference - Inversion Elimination (IE) P(hospital(mary) | sick(mary, measles)) = ? sick(mary,measles) hospital(mary) sick(mary, D) D measles epidemic(measles)epidemic(D) D measles
23
Page 23 Inference - Inversion Elimination (IE) P(hospital(mary) | sick(mary, measles)) = ? sick(mary,measles) hospital(mary) sick(mary, D) D measles epidemic(measles)epidemic(D) D measles =
24
Page 24 Inference - Inversion Elimination (IE) P(hospital(mary) | sick(mary, measles)) = ? sick(mary,measles) hospital(mary) sick(mary, D) D measles epidemic(measles)epidemic(D) D measles
25
Page 25 Inference - Inversion Elimination (IE) P(hospital(mary) | sick(mary, measles)) = ? sick(mary,measles) hospital(mary) sick(mary, D) D measles epidemic(D) D measles
26
Page 26 Inference - Inversion Elimination (IE) hospital(mary) sick(mary, D) D measles epidemic(D) D measles P(hospital(mary) | sick(mary, measles)) = ?
27
Page 27 Inference - Inversion Elimination (IE) P(hospital(mary) | sick(mary, measles)) = ? hospital(mary) sick(mary, D) D measles
28
Page 28 Inference - Inversion Elimination (IE) P(hospital(mary) | sick(mary, measles)) = ? hospital(mary)
29
Page 29 … Inference - Inversion Elimination (IE) Does not depend on domain size. hospital(mary) sick(mary,measles)sick(mary, flu) hospital(mary) sick(mary, D) epidemic(D)epidemic(measles) epidemic(flu)
30
Page 30 First contribution - Formalization of IE Joint (A ) Example X (p(X)) X,Y (p(X),q(X,Y)) Marginalization by eliminating class q(X,Y): q(X,Y) X (p(X)) X,Y (p(X),q(X,Y)) X (p(X)) q(X,Y) X,Y (p(X),q(X,Y))
31
Page 31 First contribution - Formalization of IE q(X,Y) X,Y (p(X),q(X,Y)) = q(x1,y1)... q(xn,ym) (p(x 1 ),q(x 1,y 1 ))... (p(x n ),q(x n,y m )) = ( q(x1,y1) (p(x 1 ),q(x 1,y 1 )) )... ( q(xn,ym) (p(x n ),q(x n,y m )) ) = X,Y q(X,Y) (p(X),q(X,Y)) = X,Y (p(X)) = X (p(X))
32
Page 32 First contribution - Formalization of IE q(X,Y) X,Y (p(X),q(X,Y)) = q(x1,y1)... q(xn,ym) (p(x 1 ),q(x 1,y 1 ))... (p(x n ),q(x n,y m )) = ( q(x1,y1) (p(x 1 ),q(x 1,y 1 )) )... ( q(xn,ym) (p(x n ),q(x n,y m )) ) = X,Y q(X,Y) (p(X),q(X,Y)) = X,Y (p(X)) = X (p(X))
33
Page 33 First contribution - Formalization of IE By formalizing the problem of Inversion Elimination, we determined conditions for its application: Eliminated atom must contain all logical variables in parfactors involved; Eliminated atom instances must not occur together in the same instances of parfactor.
34
Page 34 Inversion Elimination - Limitations - I Eliminated atom must contain all logical variables in parfactors involved. sick(P,D) epidemic(D)
35
Page 35 Inversion Elimination - Limitations - I Eliminated atom must contain all logical variables in parfactors involved. sick(P,D) epidemic(D) Ok, contains both P and D
36
Page 36 Inversion Elimination - Limitations - I Eliminated atom must contain all logical variables in parfactors involved. sick(P,D) epidemic(D) Not Ok, missing P sick(P,D)
37
Page 37 Inversion Elimination - Limitations - I Eliminated atom must contain all logical variables in parfactors involved. q(Y,Z) p(X,Y) No atom can be eliminated
38
Page 38 Inversion Elimination - Limitations - I Marginalization by eliminating class p(X): p(X) X,Y (p(X),q(X,Y)) = p(x1)... p(xn) Y (p(x 1 ),q(x 1,Y))... Y (p(x n ),q(x n,Y)) = ( p(x1) Y (p(x 1 ),q(x 1,Y)) )... ( p(xn) Y (p(x n ),q(x n,Y)) ) = X p(X) Y (p(X),q(X,Y))
39
Page 39 Inversion Elimination - Limitations - II Requires eliminated RVs to occur in separate instances of parfactor … sick(mary, flu) epidemic(flu) sick(mary, rubella) epidemic(rubella) … sick(mary, D) epidemic(D) D measles Inversion Elimination Ok
40
Page 40 Inversion Elimination - Limitations - II epidemic(measles) epidemic(flu) epidemic(D2) epidemic(D1) epidemic(rubella) … Inversion Elimination Not Ok D1 D2 Requires eliminated RVs to occur in separate instances of parfactor
41
Page 41 Inversion Elimination - Limitations - II e(D) D1 D2 (e(D 1 ),e(D 2 )) = e(d1)... e(dn) (e(d 1 ), e(d 2 ))... (e(d n-1 ),e(d n )) = e(d1) (e(d 1 ), e(d 2 ))... e(dn) (e(d n-1 ),e(d n ))
42
Page 42 e(D) D1 D2 (e(D 1 ),e(D 2 )) = e(D) (0,0) #(0,0) in e(D),D1 D2 (0,1) #(0,1) in e(D),D1 D2 (1,0) #(1,0) in e(D),D1 D2 (1,1) #(1,1) in e(D),D1 D2 = e(D) v (v) #v in e(D),D1 D2 Second Contribution - Counting Elimination = ( ) v (v) #v in e(D),D1 D2 (from i) |e(D)| i i=0
43
Page 43 Second Contribution - Counting Elimination Does depend on domain size, but exponentially less so than brute force; More general than Inversion Elimination, but still has conditions of its own; inter-atom logical variables must be in a dominance ordering p(X,Y), q(Y,X), r (Y) OK, Y dominates X p(X), q(X,Y), r (Y) not OK, no dominance
44
Page 44 A Simple Experiment
45
Page 45 Conclusions Contributions: Formalization and Identification of conditions for Inverse Elimination (Poole); Counting Elimination; Much faster than propositionalization in certain cases; Basis for probabilistic theorem proving on the First-order level, as it is already the case with regular logic.
46
Page 46 Future Directions Approximate inference Parameterized queries “what is the most likely D such that sick(mary, D)?” Function symbols sick(motherOf(mary), D) sequence(S, [g, c, t]) Equality MPE, MAP Summer project at Cyc
47
Page 47 The End
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.