1. Tutorial #9 by Ma’ayan Fishelson
Bucket Elimination
Tutorial #9
by Ma’ayan Fishelson
References:
Kenneth Lange “Mathematical and Statistical Methods for Genetic Analysis”
Jurg Ott “Analysis of Human Genetic Linkage”

2. Bucket Elimination Algorithm
An algorithm for performing inference in a Bayesian network.
Similar algorithms can be used in other domains, such as constraint satisfaction.

3. Bayesian Network X = {X1,…,Xn} is a set of random variables.
A BN is a pair (G,P):
G is a directed acyclic graph over nodes that represent the random variables X.
P = {Pi|1 ≤i ≤n}. Pi, defined on , is the conditional probability table associated with node Xi.
Pi = P(Xi | pa(Xi))
The BN represents a probability distribution over X.

4. Bayesian Network - ExampleC E F D
Bayesian Network
P(A)
P(C|A)
P(B|A)
P(E|B,C)
P(F|E)
P(D|A,B) A B C E F D
Moralized Graph
P(A,B,C,D,E,F) = P(A)P(B|A)P(C|A)P(D|A,B)P(E|B,C)P(F|E)

5. Some Probabilistic Inference Tasks…
Belief Assessment: find P(Xi = xi | e).
Most Probable Explanation (MPE):
find

6. Solution – Bucket Elimination Algorithms
Given an ordering of the variables X1,…Xn:
Distribute P1,…,Pn into buckets B1,…,Bn.
is the highest in order in Ai.
Process the buckets in reverse order: BnB1.
(When processing bucket Bi, multiply all the probability tables in Bi and eliminate the bucket’s variable Xi by summing over all its possible values.)
Place the resulting function in the bucket of the highest variable (in the order) that is in its scope.

7. Example – Belief Assessment
Compute:
P(A=a) = Σ{A=a,B,C,D,E,F}P(A,B,C,D,E,F) =
Σ{A=a,B,C,D,E,F}P(A)P(B|A)P(C|A)P(D|A,B)P(E|B,C)P(F|E)
Suppose an order A,C,B,E,D,F. and evidence that F=1.
The distribution into buckets is as follows:
P(A)
P(C|A)
P(B|A)
P(E|B,C)
P(D|A,B)
P(F|E)
B1(A)
B2(C)
B3(B)
B4(E)
B5(D)
B6(F)

8. Example – Computing P(A=a)
P(F|E)
P(D|A,B)
P(E|B,C)
P(B|A)
P(A)
P(C|A)
B1(A)
B2(C)
B3(B)
B4(E)
B5(D)
B6(F)
To process B6: Assign F=1, get λF(E) = P(F=1|E)
Place λF(E) in bucket B4(E).
P(D|A,B)
P(E|B,C)
λF(E)
P(B|A)
P(A)
P(C|A)
B1(A)
B2(C)
B3(B)
B4(E)
B5(D)
B6(F)

9. Computing P(A=a) (cont. 1)
P(D|A,B)
P(E|B,C)
λF(E)
P(B|A)
P(A)
P(C|A)
B1(A)
B2(C)
B3(B)
B4(E)
B5(D)
B6(F)
To process B5: λD(A,B) = ΣDP(D|A,B)
Place λD(A,B) in bucket B3(B).
P(E|B,C)
λF(E)
P(B|A)
λD(A,B)
P(A)
P(C|A)
B1(A)
B2(C)
B3(B)
B4(E)
B5(D)
B6(F)

10. Computing P(A=a) (cont. 2)
P(E|B,C)
λF(E)
P(B|A)
λD(A,B)
P(A)
P(C|A)
B1(A)
B2(C)
B3(B)
B4(E)
B5(D)
B6(F)
To process B4: λE(B,C) = ΣEP(E|B,C) λF(E)
Place λE(B,C) in bucket B3(B).
B3(B)
P(B|A)
λD(A,B)
λE(B,C)
P(A)
P(C|A)
B1(A)
B2(C)
B4(E)
B5(D)
B6(F)

11. Computing P(A=a) (cont. 3)
P(C|A)
B1(A)
B2(C)
B3(B)
B4(E)
B5(D)
B6(F)
P(B|A)
λD(A,B)
λE(B,C)
To process B3: λB(A,C) = ΣBP(B|A) λD(A,B) λE(B,C)
Place λB(A,C) in bucket B2(C).
P(A)
P(C|A)
λB(A,C)
B1(A)
B2(C)
B3(B)
B4(E)
B5(D)
B6(F)

12. Computing P(A=a) (cont. 4)
P(C|A)
λB(A,C)
B1(A)
B2(C)
B3(B)
B4(E)
B5(D)
B6(F)
To process B2: λC(A) = ΣCP(C|A) λB(A,C)
Place λC(A) in bucket B1(A).
P(A)
λC(A)
B1(A)
B2(C)
B3(B)
B4(E)
B5(D)
B6(F)

13. Computing P(A=a) (cont. 5)
λC(A)
B1(A)
B2(C)
B3(B)
B4(E)
B5(D)
B6(F)
To compute the belief P(A=a), we multiply P(A) λC(A).
We obtain a function of A.

14. Bucket Elimination Complexity
The time and space complexity is bounded by the size of the probability tables that are created by the algorithm.
w*(d) is the induced width (max clique size) of the moral graph along ordering d. A B C E
Moral graph D
Order B,C,D,E,A.
w*(d1) =4
Order E,D,C,B,A.
w*(d2) =2

15. Constraint Satisfaction Problem (CSP)
Input:
A set of variables: X = {X1,…,Xn}.
A set of constraints: R = {R1,…,Rm}.
For each constraint Ri:
Scope of Ri:
Ri is a relation over the variables in Ai.
Output:
An assignment to X that is consistent with R.

16. Constraint Graph Nodes: {X1,…Xn}. Edges:
For simplicity, assume that constraints are defined on
maximal cliques of the graph.

17. Example of CSP: Find Possible Genotypes
A pedigree typed at the ABO and
AK1 loci:
Possible haplotypes:
h1=A,A1; h2 = A,A2;
h3=O,A1; h4 = O,A2;
Possible genotypes (ordered):
gij = hi/hj (father/mother)
Ind #1: g11, g13, g31
Ind #2: g44
Ind #3: g12, g21, g14, g41, g23, g32
Ind #4: g22, g24, g42
Ind #5: g34, g43 1 2 3 4 5 A
A1/A1 O
A2/A2
A1/A2
Constraints: R13 = {<g11,g{12,14}>, <g{13,31},g{12,14,32}>},
similarly for R23, R35, and R45.

18. Bucket Elimination for CSP
Given an order X1,…, Xn:
Distribute constraints into buckets, according to the highest variable in the constraint’s scope.
Process the buckets one by one starting from Xn:
Join the constraints (with equal value for the eliminated variable), in the eliminated bucket and remember the value of the eliminated variable.
Put the new constraint in the bucket of the highest variable in its scope.

19. Example of CSP: Find Possible Genotypes (cont. 1)
Constraints:
RAC = {<g11,g{12,14}>,
<g{13,31},g{12,14,32}>};
RBC = {<g44, g14>};
RCE = {<g{14,41,23,32} g{34,43}};
RDE = {<g{24,42} g34>} A B C E D
Constraint graph

20. Find Possible Genotypes (cont. 2)
Suppose an order: A, B, C, D, E.
The distribution into buckets is as follows: BA BB BC BD BE
Initial
RAC = {<g11,g{12,14}>,
<g{13,31},g{12,14,32}>}
RBC = {<g44, g14>}
RCE =
{<g{23,32,14,41},g{34,43}>}
RDE = {<g{24,42} g34>}

21. Find Possible Genotypes (cont. 3)BA BB BC BD BE
Initial
RAC = {<g11,g{12,14}>,
<g{13,31},g{12,14,32}>}
RBC = {<g44, g14>}
RCE =
{<g{23,32,14,41},g{34,43}>}
RDE = {<g{24,42}, g34>}
When processing BE:
compute RCD = {<g{23,32,14,41}, g{24,42}>}.
put RCD into bucket BD.
remember the value g34 for E.

22. Find Possible Genotypes (cont. 4)BA BB BC BD BE
After
Processing BE.
RAC = {<g11,g{12,14}>,
<g{13,31},g{12,14,32}>}
RBC = {<g44, g14>}
RCD =
{<g{23,32,14,41},g{24,42}>}
When processing BD:
compute RC = {<g{23,32,14,41}}.
put RC into bucket BC.
remember the values g24,g42 for D.

23. Find Possible Genotypes (cont. 5)BA BB BC BD BE
After
Processing BD.
RAC = {<g11,g{12,14}>,
<g{13,31},g{12,14,32}>}
RBC = {<g44, g14>}
RC = {<g{23,32,14,41}}.
When processing BC:
compute RAB = {<g{11,13, 31}, g44>}.
put RC into bucket BB.
remember the value g14 for C.

24. Find Possible Genotypes (cont. 6)BA BB BC BD BE
After
Processing BD.
RAB = {<g{11,13, 31}, g44>}.
When processing BB:
compute RA = {<g{11,13, 31}>}.
remember the value g44 for B.

25. Find Possible Genotypes (cont. 7)
By backtracking we get the
possible values:
Ind #1 (A): g11, g13, g31
Ind #2 (B): g44
Ind #3 (C): g14
Ind #4 (D): g24, g42
Ind #5 (E): g34 1 2 3 4 5 A
A1/A1 O
A2/A2
A1/A2 1 2 3 4 5
A|A or A|O or O|A
A1|A1
O|O
A2|A2
A|O
A1|A2
A|O or O|A
A2/A2

26. Bucket Elimination Algorithm

27. Bucket Elimination Algorithm

28. Bucket Elimination Algorithm
More variations of the algorithm