1. Lifted First-Order Probabilistic Inference [de Salvo Braz, Amir, and Roth, 2005]
Daniel Lowd
5/11/2005

2. Key Ideas
Do exact inference at the first-order level, rather than grounding out the network
When we have no evidence concerning many objects, we can treat them identically
Allows for queries that primarily depend on the size of a domain

3. Background: variable elimination
Key idea: compute exact marginal probability by iteratively summing out variables.
Example: want to compute P(A,C) in a Markov network: A B D C

4. Background: variable elimination
1. Distribute across sums:

5. Background: variable elimination
1. Distribute across sums:
2. Sum out D:

6. Background: variable elimination
1. Distribute across sums:
2. Sum out D:

7. Background: variable elimination
1. Distribute across sums:
2. Sum out D:
3. Sum out B:

8. Background: variable elimination
1. Distribute across sums:
2. Sum out D:
3. Sum out B:

9. First-order variable elimination
Instead of factors  and , use parameterized factors, or parfactors:
 – potential function
A – set of atoms (may be parameterized)
C – set of constraints on groundings  of the atoms in A

10. Example parfactor Given MLN clause:
{w, Friends(A,B) => Friends(B,A)}
One parfactor might be:
: 0.7 if Friends(A,B) => Friends(B,A) is true
0.3 otherwise
A: Friends(A,B), Friends(B,A)
C: A != bob

11. Definitions Logical variable: a predicate parameter
(e.g., in Friends(A, B), A and B are logical variables)
Notation: LV(S)  logical vars in S
Random variable: the value of a functor (e.g., Friends(anna, bob) is a random variable in the Friends/Smokes domain)
Notation: RV(S)  random vars in S

12. Joint distribution We can represent any MLN as a set of parfactors G.
Joint probability of a world:
All random variables
(i.e., predicate truth
assignments) in the world

13. Joint distribution We can represent any MLN as a set of parfactors G.
Joint probability of a world:
All random variables
(i.e., predicate truth
assignments) in the world
Parfactors
Groundings

14. Joint distribution We can represent any MLN as a set of parfactors G.
Joint probability of a world:
Potential of ground atoms
All random variables
(i.e., predicate truth
assignments) in the world
Parfactors
Groundings

15. Query probability
TODO – fix, clarify

16. Query probability
TODO – fix, clarify
Split apart summation

17. Query probability Push first factor before summation
TODO – fix, clarify
Push first factor before summation

18. Query probability
Substitute parfactor g’:

19. How to find g’? Inversion elimination Counting elimination
Complexity is independent of number of groundings
Not always applicable
Counting elimination
Always applicable
Requires computing multinomial distributions (potentially large factorials)

20. Shattering
Using unification, split parfactors as necessary to ensure two conditions:
For every atom pair (p,q) in G, RV(p) and RV(q) are either identical or disjoint.
Good: Friends(anna, B) and Friends(bob, B)
Bad: Friends(anna, B) and Friends(A, bob)
Incomplete overlap prevents us from reordering terms for inversion elimination.

21. Shattering (cont.)
The second condition is used by counting elimination:
For every atom pair (p,q) in every parfactor g in G, p and q are never instantiated to the same random variable.
Good: Friends(A, B) and Friends(B, A), A != B
Bad: Friends(A, B) and Friends(B, A)
Friends(A,B) and Friends(B,A) may be instantiated to the same random variable, e.g., Friends(anna, anna), and thus are not independent.

22. Inversion elimination
Requirements
E = {e}
(a single atom, possibly parameterized)
LV(e) = LV(g)
(all logical variables that appear in e’s parfactor, g, also appear as parameters in e)
Example: suppose Ag = {Friends(A,B), Smokes(A), and Smokes(B)}
Good: E = {Friends(A,B)}
Bad: E = {Smokes(B)}
Fewer instantiations of Smokes(B) than of parfactor g, since g is over more logical variables.

23. Inversion Elimination
Because our parfactors were shattered, every single term is independent, allowing us to invert the sum and the product:
(see paper for full details)

24. Counting elimination Inversion elimination cannot be applied to:
Ag = {Professor(A), IsQualsCourse(B)}
Set E consists of multiple atoms, so that remaining atoms in g are ground.
Each atom may take on one of several values (e.g., for predicates, True or False)
Key idea: Sum out atoms by counting the number of groundings for each configuration (independent assignment of atoms to values).
(See paper for further details.)