On Distributing a Bayesian Network Thor Whalen Metron, Inc.

Major discussion points Functionality afforded by old code Issues and problems with old approach Functionality trying to add Functionality successfully added What is not yet working correctly What sill have to try to add

Outline Need a Review of JT? Distributing a BN using the JT: Issues. Directions for Solutions Matlab: New functionality and experiments And now what?

Bayesian Network Secondary Structure/ Junction Tree abd ade ace ceg egh def ad ae ce de eg a b c d e f g h Bayesian Network • one-dim. random variables • conditional probabilities Secondary Structure/ Junction Tree • multi-dim. random variables • joint probabilities (potentials)

Building a Junction Tree DAG Moral Graph Triangulated Graph Identifying Cliques Junction Tree

Step 1: Moralization G = ( V , E ) GM 1. For all w  V: b c d e f g h a b c d e f g h a b c d e f g h G = ( V , E ) GM 1. For all w  V: • For all u,vpa(w) add an edge e=u-v. 2. Undirect all edges.

Step 2: Triangulation GM GT b c d e f g h GM GT Add edges to GM such that there is no cycle with length  4 that does not contain a chord. NO YES

a b c d e f g h a b c d e f g h a b c d e f g h a b d c e f g h Bayesian Network G = ( V , E ) Moral graph GM Triangulated graph GT a b d c e f g h abd a ace ad ae ce ade e ceg e de e eg seperators def e egh Cliques e.g. ceg  egh = eg Junction graph GJ (not complete)

There are several methods to find MST. Kruskal’s algorithm: choose successively a link of maximal weight unless it creates a cycle. abd ade ace ceg egh def ad ae ce de eg abd ade ace ceg egh def ad ae ce de eg e a Junction tree GJT Junction graph GJ (not complete)

GJT a b d c e f g h abd ade ace ceg egh def ad ae ce de eg In JT cliques becomes vertices sepsets Ex: ceg  egh = eg

Propagating potentials Message Passing from clique A to clique B 1. Project the potential of A into SAB 2. Absorb the potential of SAB into B Projection Absorption

Global Propagation 1. COLLECT-EVIDENCE messages 1-5 2. DISTRIBUTE-EVIDENCE messages 6-10 Root abd ade ace ceg egh def ad ae ce de eg 3 2 7 9 6 5 1 8 4 10

Using the JT for the MSBN How? Issue: Clique Granularity Issue: Clique Association Issue: MSBN must have tree structure

Take a JT

Partition JT into sub-trees

Issue: Clique Granularity

Issue: Clique Granularity

Issue: Clique association

Issue: Clique association

Issue: MSBN has a tree structure

Issue: MSBN has a tree structure Can’t communicate here!

Issue: MSBN has a tree structure Can’t communicate here! Though these two subnets may share many variables.

Directions for Solutions Controlling Granularity Increasing Granularity Choosing the JT Alternative communication graphs

Controlling Granularity Moralization and Triangulation → Cliques Moralization: No choice. Triangulation: Some choice. Triangulate keeping the subnets in mind.

Increasing Granularity Can we break down cliques? Conjecture: Given two sets of random variables X and Y, let Given a clique Z whose set of random variables Z is covered by sets X and Y. If ∆(X,Y) is small enough then we may replace Z by X and Y. Y Z X

Choosing The JT Once the cliques have been formed (hence the JG), any maximal weight spanning tree of the JG will do for the JT. Again, we should choose this tree so as to reduce the overhead when connecting subnets internally.

Alternative communication Graphs We raise the question as to wether it is possible to perform intra- and extra-subnet calibration using some non-tree sub-graph of the JG…

MatLab: The new Stuff: Netica can talk to the Matlab tool. View/Interact tool is nicer JG for communication graph… (or not) Query cliques

Propagating with cycles BC BCE ABC BE B E BDE DEF DE

Mysterious convergence ABC E B E BDE DEF DE