1. Lecture 3: Exact Inference in GMs
see also Rina Drechter’s course
on Bayesian networks.

2. Elimination
Does the structure of UGM & DGM imply tractible algorithms for computing conditional distributions?
Elimination: algorithm to compute conditional distributions:
- choose ordering of nodes: last node is target node
- maintain active stack of functions
- when eliminating a node, pick all functions that depend on it,
sum out the variable, put new function on stack.
Graph elimination: when eliminating node connect all neighbors
The reconstituted graph is triangulated (no chord-less cycles)
Maximal cliques in rec. graph determine complexity.
Tree-width: maximal clique-size minus one, minimized over all triangulations.

3. Belief Propagation
For every conditional distr. we need new elimination – can we do better?
- full answer: Junction Tree algorithm.
- partial answer for trees: belief propagation.
Undirected trees  Directed trees. Polytrees
Directed trees can be rewritten using potential functions.
- however unobserved branches can be pruned away.
Messages towards the root (inward) node are just elimination
However, to avoid rerunning elimination over and over we can run messages outward only once.
Protocol: send message only when received all other messages

4. Factor Graphs
If potentials contain > 2 nodes (e.g. polytrees) we convert to factor graphs (FG).
FG make factorization explicit, does not help for reading out conditional independencies, but makes inference more efficient.
BP rules for factor trees: compute exact marginal or conditional distributions on variables and factors.
BP on FG is one unified inference algorithm for both undirected and directed GMs.
Applying same rules to graphs with loops often gives good results: “loopy BP”.