Local structures; Causal Independence, Context-sepcific independance COMPSCI 276 Fall 2007

Local structure2 Reducing parameters of families Determinizm Causal independence Context-specific independanc Continunous variables

Local structure3

4 Causal Independence Event X has two possible causes: A,B. It is hard to elicit P(X|A,B) but it is easy to determine P(X|A) and P(X|B). Example: several diseases causes a symptom. Effect of A on X is independent from the effect of B on X  Causal Independence, using canonical models: Noisy-O, Noisy AND, noisy-max AB X

Local structure5 Binary OR AB X ABP(X=0|A,B) 001 P(X=1|A,B)

Local structure6 Noisy-OR “noise” is associated with each edge described by noise parameter  [0,1] : Let q b =0.2, q a =0.1 P(x=0|a,b)= (1- a ) (1- b ) P(x=1|a,b)=1-(1- a ) (1- b ) AB X ABP(X=0|A,B) 001 P(X=1|A,B) 0 a b qi=P(X=0|A_i=1,…else =0)

Local structure7 Noisy-OR with Leak Use leak probability 0  [0,1] when both parents are false: Let a =0.2, b =0.1, 0 = P(x=0|a,b)= (1- 0 )(1- a ) a (1- b ) b P(x=0|a,b)=1-(1- 0 )(1- a ) a (1- b ) b AB X ABP(X=0|A,B) P(X=1|A,B) a b

Local structure8 Formal Definition for Noisy-Or Definition 1 Let Y be a binary-valued random variable with k binary-valued parents X 1,…,X k. The CPT P(Y|X 1,…X k ) is a noisy-or if there are k+1 noise parameters 0, 1,… k such that P(y=0| X 1,…,X k ) = (1- 0 )  i,Xi=1 (1- i )

Local structure9 Closed Form Bel(X) - 1 Given: noisy-or CPT P(x|u) noise parameters i T u = {i: U i = 1} Define: q i = 1 - I, Then: q_i is the probability that the inhibitor for u_i is active while the

Local structure10 Closed Form Bel(X) - 2 Using Iterative Belief Propagation : Set pi ix = pi x (u k =1). Then we can show that:

Local structure11 Closed Form Bel(X) - 2 Using Iterative Belief Propagation : Set pi ix = pi x (u k =1). Then we can show that:

Local structure12 Causal Influence Defined Definition 2 Let Y be a random variable with k parents X 1,…,X k. The CPT P(Y|X 1,…X k ) exhibits independence of causal influence (ICI) if it is described via a network fragment of the structure shown in on the left where CPT of Z is a deterministic functions f. Z Y X1X1 X1X1 X1X1 Z0Z0 Z1Z1 Z2Z2 ZkZk

Local structure13

Local structure14

Local structure15

Local structure16

Local structure17

Local structure18 Context Specific Independence When there is conditional independence in some specific variable assignment

Local structure19

Local structure20

Local structure21

Local structure22

Local structure23 The impact during inference Causal independence in polytrees is linear during inference Causal independence in general can sometime be exploited but not always CSI can be exploited by using operation (product and summation) over trees.

Local structure24 Representing CSI Using decision trees Using decision graphs

Local structure25 IntelligenceDifficulty Grade Letter SAT Job Apply A student’s example

Local structure26 A S L (0.8,0.2) (0.9,0.1)(0.4,0.6) (0.1,0.9) s1s1 a0a0 a1a1 s0s0 l1l1 l0l0 Tree CPD If the student does not apply, SAT and L are irrelevant Tree-CPD for job

Local structure27 Definition of CPD-tree A CPD-tree of a CPD P(Z|pa_Z) is a tree whose leaves are labeled by P(Z) and internal nodes correspond to parents branching over their values.

Local structure28 C L2 (0.1,0.9) l2 1 c1c1 c2c2 l2 0 L1 (0.8,0.2)(0.3,0.7) l1 1 l1 0 (0.9,0.1) Letter1 Job Letter2 Choice Captures irrelevant variables

Local structure29 Multiplexer CPD A CPD P(Y|A,Z1,Z2,…,Zk) is a multiplexer iff Val(A)=1,2,…k, and P(Y|A,Z1,…Zk)=Z_a Letter1 Letter Letter2 Choice Job

Local structure30 A B C (0.3,0.7)(0.4,0.6) (0.1,0.9) b1b1 a0a0 a1a1 b0b0 c1c1 c0c0 C B (0.3,0.7)(0.5,0.5) (0.2,0.8) c1c1 c0c0 b1b1 b0b0 Rule-based representation A CPD-tree that correponds to rules.

Continuous Variables ICS 275b 2002

Local structure32 Gaussian Distribution N( ,  )

Local structure gaussian(x,0,1) gaussian(x,1,1) N( ,  )

Local structure gaussian(x,0,1) gaussian(x,0,2) N( ,  )

Local structure35 Multivariate Gaussian Definition: Let X 1,…,X n. Be a set of random variables. A multivariate Gaussian distribution over X 1,…,X n is a parameterized by an n-dimensional mean vector  and an n x n positive definitive covariance matrix . It defines a joint density via:

Local structure36 Linear Gaussian Distribution Definition: Let Y be a continuous node with continuous parents X 1,…,X k. We say that Y has a linear Gaussian model if it can be described using parameters  0, …,  k and  2 such that: P(y| x 1,…,x k )=N (  0 +  1 x 1 +…,  k x k ;  2 )

Local structure37 XY XYXY

Local structure38

Local structure39 Linear Gaussian Network Definition Linear Gaussian Bayesian network is a Bayesian network all of whose variables are continuous and where all of the CPTs are linear Gaussians. Linear Gaussian BN  Multivariate Gaussian =>Linear Gaussian BN has a compact representation

Local structure40 Hybrid Models Continuous Node, Discrete Parents (CLG) –Define density function for each instantiation of parents Discrete Node, Continuous Parents –Treshold –Sigmoid

Local structure41 Continuous Node, Discrete Parents Definition: Let X be a continuous node, and let U={U 1,U 2,…,U n } be its discrete parents and Y={Y 1,Y 2,…,Y k } be its continuous parents. We say that X has a conditional linear Gaussian (CLG) CPT if, for every value u  D(U), we have a a set of (k+1) coefficients a u,0, a u,1, …, a u,k+1 and a variance  u 2 such that:

Local structure42 CLG Network Definition: A Bayesian network is called a CLG network if every discrete node has only discrete parents, and every continuous node has a CLG CPT.

Local structure43 Discrete Node, Continuous Parents Threshold Model

Local structure44 Discrete Node, Continuous Parents Sigmoid Binomial Logit Definition: Let Y be a binary-valued random variable with k continuous-valued parents X 1,…X k. The CPT P(Y|X 1 …X k ) is a linear sigmoid (also called binomial logit) if there are (k+1) weights w 0,w 1,…,w k such that:

Local structure sigmoid(0.1*x) sigmoid(0.5*x) sigmoid(0.9*x)

Local structure46 References Judea Pearl “Probabilistic Reasoning in Inteeligent Systems”, section 4.3 Nir Friedman, Daphne Koller “Bayesian Network and Beyond”