Bayesian Networks Clique tree algorithm Presented by Sergey Vichik

Algorithm sequence 1. Translate a BN to Markov graph (moralization) 2. Add edges to create chordal graph 3. Find cliques 4. Construct clique tree 5. Enter evidences 6. Calc a posteriori probability (inference) Moralization Add edges Find Cliques Enter evidences Create clique tree Inference Bayesian network Markov graph Chordal graph Cliques graph Clique tree Tree with Evidences Evidences

BN to Markov u Add an edge between parents. B D E F C A G B D E F C A G Moralization Add edges Find Cliques Enter evidences Create clique tree Inference Bayesian network Markov graph Chordal graph Cliques graph Clique tree Tree with Evidences Evidences

Create Chordal graph u Add edges to form a chordal graph. B D E F C A G B D E F C A G Moralization Add edges Find Cliques Enter evidences Create clique tree Inference Bayesian network Markov graph Chordal graph Cliques graph Clique tree Tree with Evidences Evidences

Cliques graph u Find all cliques, connect them to form a clique graph u Cliques are connected if they are sharing a variable B D E F C A G AGC CDE ADC EF ABD Moralization Add edges Find Cliques Enter evidences Create clique tree Inference Bayesian network Markov graph Chordal graph Cliques graph Clique tree Tree with Evidences Evidences

Running Intersection property u Many trees may be embedded in the cliques graph. What tree to choose? u Lets try: {ADC}-{AGC}-{ABD}-{CDE}-{EF} u Now lets follow this order of cliques and perform an elimination:  Eliminate F, Eliminate E  Now, in order to continue we need to eliminate either C or D. Eliminating any of them will result in creating an extra edge : CB or GD, thus enlarging the probability tables. u The required property : If variable x is contained in cliques Y and Z, it must be contained in every. clique on path from Y to Z. AGC CDE ADC EF ABD B D E F C A G

Clique tree construction u A maximal spanning tree of the clique graph, is the required clique tree. u Proof: At the end of the lecture. AGC CDE ADC EF ABD Moralization Add edges Find Cliques Enter evidences Create clique tree Inference Bayesian network Markov graph Chordal graph Cliques graph Clique tree Tree with Evidences Evidences

Enter evidence u Entering evidence is eqivalent to removing the evidence variables and recalculating the effected probability functions. u General approach : build the clique tree without the evidence, and then recalculate the effected cliques. Actually it means reducing the tables to specific values :  f(E,F) -> f(E,F=f). Moralization Add edges Find Cliques Enter evidences Create clique tree Inference Bayesian network Markov graph Chordal graph Cliques graph Clique tree Tree with Evidences Evidences

Efficient Calculation of probabilities u m ij – a message from i to j u Wait for all messages excluding j (immediately for leaves). u Store messages on the edge for efficient update. AGC CDE ADC EF ABD ACAD CD E Moralization Add edges Find Cliques Enter evidences Create clique tree Inference Bayesian network Markov graph Chordal graph Cliques graph Clique tree Tree with Evidences Evidences

Calculation of a marginal probability 1) Select clique with a variable. 2) Multiply all incoming messages. 3) Marginalize to the required variable. P=f(ADC)f(AGC)f(ABD)f(CDE)f(EF) AGC CDE ADC EF ABD ACAD CD E

Maximum Spanning tree is a Clique Tree – a Proof../1 1) From graph theory : Cycle property:  For any cycle C in the graph, if the weight of an edge e of C is smaller than the weights of other edges of C, then this edge cannot belong to an MST. 2) For any chordal graph exists a clique tree.  Every chordal graph have a perfect elimination order.

Proof../2 u Lets take a clique tree that has as much common edges with MST but different. u For the purpose of contradiction, assume that edge (K1,K2) in MST is not an edge in CT (Clique Tree). u take the cut set associated with (K1,K2) in MST from the full graph. u There must be another edge (K3,K4) not equal to (K1,K2) in this cut set, and (K3,K4) is in CT but not in MST. K1 K2 K3K4 (K1,K2)  MST (K3,K4)  CT (K3,K4)  MST Cut set - set of all edges connecting 2 graph partitions

Proof../3 u But from properties of clique tree : K1∩K2  K3∩K4. if K1∩K2  K3∩K4, it contradicts the fact that MST is maximal. u Therefore K1∩K2 = K3∩K4. u We can replace the (K3,K4) with (K1,K2) in CT, and still remain with a clique tree.  Lets take K5 and K6 belong to different cut set sides. From properties of CT, K5∩K6  K3∩K4 and thus K5∩K6  K1∩K2.  Therefore the clique intersection property holds u We have contradicted the fact that CT is different from MST and proposed an algorithm to make them equal. K1 K2 K3K4 (K1,K2)  MST (K3,K4)  CT (K3,K4)  MST CT=> K1∩K2  K3∩K4 MST=> K1∩K2  K3∩K4 => K1∩K2 = K3∩K4 //