Download presentation
Presentation is loading. Please wait.
Published byLetitia Stafford Modified over 9 years ago
1
1 Bayesian Networks (Directed Acyclic Graphical Models) The situation of a bell that rings whenever the outcome of two coins are equal can not be well represented by undirected graphical models. A clique will be formed because of induced dependency of the two coins given the bell. Coin 1 Bell Coin 2
2
2 Bayesian Networks (BNs) Examples of models for diseases & symptoms & risk factors One variable for all diseases (values are diseases) One variable per disease (values are True/False) Naïve Bayesian Networks versus Bipartite BNs
3
3 Boundary Basis for Dependency Models Let M be a dependency model over U={X 1,…,X n }. Let d be an ordering of these elements. A boundary basis wrt d of M is a set of independence statements I(X i, B i, U i -B i ) that hold in M where U i ={X 1,X 2,…,X i-1 }, i=1,..n. A boundary basis is minimal if every B i is minimal. Example I: What is the boundary basis for P(X1,X2,X3,X4) = P(X1)P(X2|X1)P(X3|X2)P(X4|X3)?
4
4 Example I I ( X 3, X 2, X 1 ) I ( X 4, X 3, { X 1, X 2 }) X1X1 X2X2 X3X3 X4X4 A boundary basis and a boundary DAG for: P(X1,X2,X3,X4) = P(X1)P(X2|X1)P(X3|X2)P(X4|X3)? The directed acyclic graph (DAG) created by assigning each vertex X i the parents B i is called the boundary DAG of M relative to order d.
5
5 Example II I ( coin 1, { },coin 2 ) Coin 1 Bell Coin 2 A boundary basis and a boundary DAG for: P(coin 1,coin 2,bell) =P(coin 1 )P(coin 2 )P(bell|coin 1,coin 2 )
6
6 Example III In the order V,S,T,L,B,A,X,D, we have a boundary basis: I( S, { }, V ) I( T, V, S) I( l, S, {T, V}) … I( X,A, { V,S,T,L,B,D}) V S L T A B XD Does I ( {X, D},A,V) also hold in the dependency model P ?
7
7 1. A Directed Acyclic Graph (DAG) D=(U,E) is an I-map of a dependency model M over U if I D (X,Z,Y) I M (X,Z,Y) for all disjoint subsets X,Y, Z of U. 2.D is a minimal I-map of M if by removing any edge, D ceases to be an I-map. 3. D is a perfect map of M if I D (X,Z,Y) I M (X,Z,Y) for all disjoint subsets X,Y, Z of U. Definitions Can we define “Independence” I D (X,Z,Y) graphically that answers these probabilistic independence questions ?
8
8 From Separation in UGs To d-Separation in DAGs
9
9 Paths u Intuition: dependency must “flow” along paths in the graph u A path is a sequence of neighboring variables Examples: u X A D B u A L S B V S L T A B XD
10
10 Path blockage u Every path is classified given the evidence: active -- creates a dependency between the end nodes blocked – does not create a dependency between the end nodes Evidence means the assignment of a value to a subset of nodes.
11
11 Blocked S L B S L B Path Blockage Three cases: Common cause Blocked Active
12
12 Blocked S A L S A L Path Blockage Three cases: Common cause Intermediate cause Blocked Active
13
13 Blocked T L X A T L X A T L X A Path Blockage Three cases: Common cause Intermediate cause Common Effect Blocked Active
14
14 Definition of Path Blockage Definition: A path is active, given evidence Z, if Whenever we have the configuration then either A or one of its descendents is in Z No other nodes in the path are in Z. Definition: A path is blocked, given evidence Z, if it is not active. T L A Definition: X is d-separated from Y, given Z, if all paths from a node in X and a node in Y are blocked, given Z.
15
15 d-Separation
16
16 I D (T,S| ) = yes Example V S L T A B XD
17
17 V S L T A B XD I D (T,S | ) = yes I D (T,S|D) = no Example
18
18 I D (T,S | ) = yes I D (T,S|D) = no I D (T,S|{D,L,B}) = yes Example V S L T A B XD
19
19 Example In the order V,S,T,L,B,A,X,D, we get from the boundary basis: I D ( S, { }, V ) I D ( T, V, S) I D ( l, S, {T, V}) … I D ( X,A, { V,S,T,L,B,D}) V S L T A B XD
20
20 Main Result - Soundness
21
21 Bayesian Networks (Directed Acyclic Graphical Models) Definition: Given a probability distribution P on a set of variables U, a DAG D = (U,E) is called a Bayesian Network of P iff D is a minimal I-map of P.
22
22 First claim holds because any probability distribution is a semi graphoid (Symmetry, Decomposition, Contraction, Weak union).
23
23 Second claim of uniqueness of parents sets holds due to. I(X,ZW 1,YW 2 ) and I(X,ZW 2,YW 1 ) I(X,Z,YW 1 W 2 ) Proof: (1) I(X, ZW 1,YW 2 ). Given. (2) I(X, ZW 2,YW 1 ). Given. (3) I(X, ZW 1 W 2,Y) by weak union from (1). (4) I(X, ZYW 1,W 2 ) by weak union from (1). (5) I(X, ZYW 2,W 1 ) by weak union from (2). (6) I(X, ZY, W 1 W 2 ) by intersection from (4) and (5). I(X, Z, YW 1 W 2 ) by intersection from (3) and (6).
24
24 d-separation The definition of I D (X, Z, Y) is such that: Soundness [Theorem 9]: I D (X, Z, Y) = yes implies I P (X, Z, Y) follows from the boundary Basis(D). Completeness [Theorem 10]: I D (X, Z, Y) = no implies I P (X, Z, Y) does not follow from the boundary Basis(D).
25
25 Revisiting Example II V S L T A B XD So does I P ( {X, D},A, V) hold ? Enough to check d-separation !
26
26 Bayesian Networks with numbers p(t|v) V S L T A B XD p(x|a) p(d|a,b) p(a|t,l) p(b|s) p(l|s) p(s)p(s) p(v)p(v)
27
27 Bayesian Network (cont.) Each Directed Acyclic Graph defines a factorization of the form: p(t|v) V S L T A B XD p(x|a) p(d|a,b) p(a|t,l) p(b|s) p(l|s) p(s)p(s) p(v)p(v)
28
28 Independence in Bayesian networks This set of independence assertions is denoted Basis(G). All other independence assertions that are entailed by (*) are derivable using the semi-graphoid axioms. I P ( X i ; { X 1,…,X i-1 }\Pa i | Pa i )
29
29 Local distributions- Asymmetric independence Table: p(A=y|L=n, T=n) = 0.02 p(A=y|L=n, T=y) = 0.60 p(A=y|L=y, T=n) = 0.99 p(A=y|L=y, T=y) = 0.99 Lung Cancer (Yes/No) Tuberculosis (Yes/No) Abnormality in Chest (Yes/no) p(A|T,L)
30
30 COROLLARY 4: D is an I-map of P iff each variable X is conditionally independent in P of all its non-descendants, given its parents. Proof : Each variable X is conditionally independent of all its non-descendants, given its parents implies using decomposition that it is also independent of its predecessors in a particular order d. Proof : X is d-separated of all its non-descendants, given its parents. Since D is an I-map, by the soundness theorem the claim holds.
31
31 COROLLARY 5: If D=(U,E) is a boundary DAG of P constructed in some order d, then any topological order d’ of U will yield the same boundary DAG of P. (Hence construction order can be forgotten). Proof : By Corollary 4, each variable X is d-separated of all its non-descendants, given its parents in the boundary DAG of P. In particular, due to decomposition, X is independent given its parents from all previous variables in any topological order d’.
32
32 Extension of the Markov Chain Property I(X k, X k-1, X 1 … X k-2 ) I(X k, X k-1 X k+1, X 1 … X k-2 X k+2 … X n ) Holds due to the soundness theorem. Converse holds when Intersection is assumed. Markov Blankets in DAGs
33
33 Consequence: There is no improvement to d-separation and no statement escapes graphical representation. Reasoning: (1) If there were an independence statement not shown by d-separation, then must be true in all distributions that satisfy the basis. But Theorem 10 states that there exists a distribution that satisfies the basis and violates . (2) Same argument. [Note that (2) is a stronger claim.]
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.