Bayesian Networks, Winter Yoav Haimovitch & Ariel Raviv 1

 Chordal Graph  Warm up  Theorem 7  Perfect Vertex Elimination Scheme  Maximal cliques Tree Bibliography M.C.Golumbic Algorithmic Graph Theory and Perfect Graphs. Second edition Pearl, J. Probabilistic Reasoning in Intelligent Systems. 2

 Definition: Undirected graph G is chordal if every cycle of length greater than 3 has a chord.  Remark: Chordality is a hereditary property, All induced subgraphs of a chordal graph are chordal!  Chordal graphs are also called triangulated and perfect elimination graphs. 3

4

 A vertex x of G is called simplicial if its adjacency set Adj(x) is a clique (not necessarily maximal)  Lemma 1[Dirac 1961]: Each chordal graph has a simplicial vertex and if G is not a clique it has two non adjacent simplicial vertices. 5

 If G is a clique, done.  If not, assume that be has two non-adjacent nodes a and b and that the lemma is true to all graphs with fewer vertices than G. Let S be a minimal vertex separator for a and b. Let G a and G b be the connected components of a and b respectively. 6

 S is a clique – if not, then there exists x and y such that there is no edge between them, and because S is minimal vertex separator there is a cycle. It is easy to show that there exist a minimal cycle with length no less then 4 with no chords in it, contradiction. 7

 S is a clique. G A +S is smaller than G therefore by induction the lemma holds, i.e. G A +S is a clique or has two non adjacent simplicial vertices, one of each must be in G A. Any simplicial vertex in G A is a simplicial vertex in G because all elements of Adj(A) are inside G A +S. Thus from G A and G B we get two simplicial vertices in G. 8

 Definition: The following are equivalent: 1.G is chordal 2.The edges of G can be directed acyclically so that every pair of converging arrows comes from two adjacent vertices. 3.G has a perfect vertex elimination scheme. 4.There is a tree T with maximal cliques of G as vertices. Any two cliques containing v are either adjacent in T or connected by a path of cliques that contain v. 9

 By lemma 1 every chordal graph has a simplicial vertex  Direct all of its edges to it  Repeat the process with the rest of the graph (can be done because being chordal is hereditary)  From the definition of simplicial vertex corollary that each two converging edges coming from adjacent vertices. 10

 Let there be a cycle of size larger than 3.  There exist a valid direction which is acyclic  There are two converging edges, which come from two adjacent vertices  The cycle has a chord. 11

 Given a Markov network, we would like to calculate a probability function over it’s variables  What if we want to calculate the probability of one variable? 12

P(X 1,X 2,X 3,X 4,X 5 )=K∙f 1 (X 1,X 2 )f 2 (X 2,X 3 )f 3 (X 3,X 4 )f 4 (X 4,X 5 )f 5 (X 5,X 1 ) P(X 1,X 2,X 3,X 4 )=Σ X5 K∙f 1 (X 1,X 2 )f2(X 2,X 3 )f 3 (X 3,X 4 )f 4 (X 4,X 5 )f 5 (X 5,X 1 )= =K∙f 1 (X 1,X 2 )f 2 (X 2,X 3 )f 3 (X 3,X 4 ) Σ X5 f 4 (X 4,X 5 )f 5 (X 5,X 1 ) = K`∙f 1 (X 1,X 2 )f 2 (X 2,X 3 )f 3 (X 3,X 4 ) F(X 4, X 1 )  Explanation: Each time we sum over a variable we automatically add edges between all it’s adjacent vertices (creating a clique) 13

 Here is an example of how bad (i.e. big cliques) things can happen with incorrect order of summing P(X 1,X 2,X 3,X 4,X 5 )=Σ X0 K∙f 1 (X 0,X 1 )f 2 (X 0,X 2 )f 3 (X 0,X 3 )f 4 (X 0,X 4 )f 5 (X 0,X 5 ) = K`∙F(X 1,X 2,X 3,X 4,X 5 )  Explanation: For wrong order of summing we get big cliques and thus huge tables of probabilities to store, which are exponential in number of variables. 14

 Let G=(V,E) be an undirected graph and let σ=[v 1,v 2,…,v n ] be an ordering of the vertices. We say that σ is a perfect vertex elimination scheme if for each i: X i = { v j in Adj(v i ) | j>I } is complete. 15

 Definition: The following are equivalent: 1.G is chordal 2.The edges of G can be directed acyclically so that every pair of converging arrows comes from two adjacent vertices. 3.G has a perfect vertex elimination scheme. 4.There is a tree T with maximal cliques of G as vertices. Any two cliques containing v are either adjacent in T or connected by a path of cliques that contain v. 16

 By lemma 1, G has a simplicial vertex.  The subgraph induced after removing this vertex is also chordal.  By induction, there is an elimination order. 17

 Let us assume a cycle of length 4 or greater.  Let x be the first vertex on the elimination scheme of this cycle.  x is simplicial at this point.  Two vertices adjacent to x from the cycle are also adjacent between themselves. 18

3.All vertices of G can be deleted by arranging them in separate piles, one for each clique, and then repeatedly applying the following two operations:  Delete a vertex that occurs in only one pile.  Delete a pile if all its vertices appear in another pile. 19

20 A A B B C C A A C C B B D D E E B B C C D D D D E E

21 A A C C B B D D E E B B C C D D D D E E A A B B A A C C D D

 Definition: The following are equivalent: 1.G is chordal 2.The edges of G can be directed acyclically so that every pair of converging arrows comes from two adjacent vertices. 3.G has a perfect vertex elimination scheme. 4.There is a tree T with maximal cliques of G as vertices. Any two cliques containing v are either adjacent in T or connected by a path of cliques that contain v. 22

23 a a c c b b d d e e C 2 C 3 C 1 a a c c b b C 1 c c b b d d b b d d e e C 2 C 3

24 b b a a d d c c C2C2 C1C1 C3C3 C4C4 b b a a C 1 b b d d d d c c C 4 C 3 a a c c C 2

 Lemma 2: A vertex is simplicial iff it belongs to exactly one clique.  Let G =(V,E) be a chordal graph and |V|=k  Proof by induction: Assume the claim holds for all graphs of size < k.  By Lemma 1 G has a simplicial vertex v.  By Lemma 2 v belongs to exactly one clique C. 25

 For G’ =(V\{v},E’) there exists a tree T’ which satisfies the claim. Split into two cases: ◦ C’ = C – {v} is maximal in G’. Add v to C’ to build T from T’. T is a Clique-Tree as needed. 26 a a c c b b d d e e C2C2 C1C1 a a b b C1C1 b b d d C2C2 e e c c

◦ C’ = C – {v} is not maximal in G’. There is a maximal clique P in G’ such that C’⊂P. Add C and (C, P) to T’ to build T. T is a Clique- Tree as needed. 27 b b a a d d c c C2C2 C1C1 C1C1 a a d d C2C2 c c b b a a d d

 Let T be a Clique-Tree of graph G=(V,E) and |V|=k.  Proof by induction: Assume the claim holds for all graphs of size < k.  Let L be a leaf in T, and P its parent in T.  Let v∈L\P (exists from maximality).  v cannot be in any other clique.  by Lemma 2, v is simplicial. 28

 Let T’ be T with v removed. T’ is a Clique- Tree.  By assumption G’ =(V\{v},E’) is chordal.  By part 3 of Theorem 7, there exists σ’, a Perfect Elimination Order for G’.  Let σ = [v, σ’] be a perfect elimination order for G.  By the converse of part 3, we can conclude that G is chordal 29