Bayesian Networks (Directed Acyclic Graphical Models) Coin1 Coin2 Bell The situation of a bell that rings whenever the outcome of two coins are equal can not be well represented by undirected graphical models. A clique will be formed because of induced dependency of the two coins given the bell.

Bayesian Networks (Directed Graphical Models)

Example I X1 X2 X3 X4 ID( X3 ; X1 | X2) ID( X4 ; {X1, X2}| X3)

Example II In the order V,S,T,L,B,A,X,D, we have: ID( S; V ) ID( T; S | V ) ID( l; {T, V} | S ) … ID( X; {V,S,T,L,B,D} | A) Does ID( {X, D} ; V | A ) also hold ? To answer this question one needs to analyze the types of paths that connect {X, D} and V.

Paths Intuition: dependency must “flow” along paths in the graph A path is a sequence of neighboring variables Examples: X  A  D  B A  L  S  B V S L T A B X D

Path blockage Every path is classified given the evidence: active -- creates a dependency between the end nodes blocked – does not create a dependency between the end nodes Evidence means the assignment of a value to a subset of nodes.

Path Blockage Three cases: Common cause Blocked Blocked Active S L B S

Path Blockage Three cases: Common cause Intermediate cause Blocked Active Blocked S A L

Path Blockage Three cases: Common cause Intermediate cause Common Effect Blocked Active Blocked T L X A T L X A

Definition of Path Blockage Definition: A path is active, given evidence Z, if Whenever we have the configuration then either A or one of its descendents is in Z No other nodes in the path are in Z. Definition: A path is blocked, given evidence Z, if it is not active. T L A Definition: X is d-separated from Y, given Z, if all paths from a node in X and a node in Y are blocked, given Z.

Example ID(T,S|) = yes V S L T A B X D

Example ID (T,S |) = yes ID(T,S|D) = no V S L T A B X D

Example ID (T,S |) = yes ID(T,S|D) = no ID(T,S|{D,L,B}) = yes V S L T

d-separation The definition of ID (X; Y | Z) is such that: Soundness [Theorem 9]: ID (X; Y | Z) = yes implies IP(X;Y|Z) follows from Basis(G) Completeness [Theorem 10]: ID (X; Y | Z) = no implies IP(X;Y|Z) does not follow from Basis(G)

Revisiting Example II So does IP( {X, D} ; V | A ) hold ? V S L T A B

Extension of the Markov Chain Property

How Expressive are Bayesian Networks

Quantifying the links of Bayesian Networks p(v) p(s) V S L T A B X D p(t|v) p(l|s) p(b|s) p(a|t,l) p(d|a,b) p(x|a) Bayesian network = Directed Acyclic Graph (DAG), annotated with conditional probability distributions.

Bayesian Network (cont.) Each Directed Acyclic Graph defines a factorization of the form: p(t|v) V S L T A B X D p(x|a) p(d|a,b) p(a|t,l) p(b|s) p(l|s) p(s) p(v)

Independence in Bayesian networks IP( Xi ; {X1,…,Xi-1}\Pai | Pai ) This set of independence assertions is denoted Basis(G) . All other independence assertions that are entailed by (*) are derivable using the semi-graphoid axioms.

Local distributions- Asymmetric independence Lung Cancer (Yes/No) Tuberculosis Abnormality in Chest (Yes/no) p(A|T,L) Table: p(A=y|L=n, T=n) = 0.02 p(A=y|L=n, T=y) = 0.60 p(A=y|L=y, T=n) = 0.99 p(A=y|L=y, T=y) = 0.99