Bayesian Networks

Quiz: Probabilistic Reasoning 1.What is P(F), the probability that some creature can fly? 2.Creature b is a bumble bee. What’s P(F|B), the probability that b can fly given that it’s a bumble bee? 3.b has unfortunately met a malicious child, who has torn off b’s wings. What is P(F|B,N), the probability that b can fly given that it has no wings? 4.b somehow makes its way onto a jumbo jet, where it survives by drinking juice spilled by passengers. What is P(F|B, N, L=j), the probability that b can fly given that it has no wings and its location is a jet?

Example BN = (V, E, P) V = a set of random variables E = directed edges between them (cycles not allowed) P = for every node in the network, a conditional probability distribution for that random variable, given its parents in the graph Has diabetes? (D or  D) Test was positive? (+ or -) Observable node Unobservable node Diab?P(D) D0.01 DD 0.99 Diab?Test?P(T|D) D+0.9 D-0.1 DD +0.2 DD -0.8

Simple probabilistic reasoning You already know how to figure out: P(D)  stored in the Bayes Net P(+|D)  stored in the Bayes Net P(D,+)  multiply P(D)P(+|D) P(+)  apply marginalization to P(D, +) P(D|+)  apply Bayes’ Rule

Purpose behind Bayes Networks Bayes Nets help figure out more difficult cases: What’s P(Battery Dead | Car won’t start, Battery is 5 years old)? or P(Alternator broken | Car won’t start, oil light is on, lights are dim)? Battery dead Battery age Fan belt broken Battery meter No oil Battery flat Alternator broken Not charging No gas Starter broken Fuel line blocked Lights Gas gauge Oil light Car won’t start dipstick

Types of Bayes Net Queries Bayes Nets let you solve “queries”, or probabilistic questions. There are different types of queries for a Bayes Net with random variables X1, …, XN: 1.Joint queries: What is P(car starts, oil light on)? 2.Conditional queries: What is P(alternator broken, battery light dim | oil light off, lights dim)? 3.Maximum a posteriori (MAP): what values (true or false) for “Will Car Start?” makes this probability the biggest: P(Will Car Start? | battery is 5 years old, lights dim)

The Bayes Net Equation

Example P(Diab, Test) = P(Diab|parents(Diab)) *P(Test|parents(Test)) =P(Diab) *P(Test|Diab) Has diabetes? (D or  D) Test was positive? (+ or -)

Quiz: Two-test Diabetes 1.What is P(Test1=+|D)? 2. What is P(Test1=+|D,Test2=+)? 3. What is P(D|Test1=+,Test2=+)? 4. What is P(D|Test1=+,Test2=-)? Has diabetes? (D or  D) Test 1 was positive? (+ or -) Test 2 was positive? (+ or -) Diab?P(D) D0.01 DD 0.99 Diab?Test1?P(T1|D) D+0.9 D-0.1 DD +0.2 DD -0.8 Diab?Test2?P(T2|D) D+0.9 D-0.1 DD +0.2 DD -0.8

Conditional Independence in a BN In this BN, T1  T2 | D This means, e.g.: P(T1=+|D, T2=+) is the same as P(T1=+|D) Has diabetes? (D or  D) Test 1 was positive? (+ or -) Test 2 was positive? (+ or -)

Quiz: Two-test Diabetes What is P(T1=+|T2=+)? Has diabetes? (D or  D) Test 1 was positive? (+ or -) Test 2 was positive? (+ or -)

Absolute vs. Conditional Independence Remember: T1  T2 | D Does this mean that T1  T2 ? In other words, P(T1) =? P(T1 | T2) Has diabetes? (D or  D) Test 1 was positive? (+ or -) Test 2 was positive? (+ or -)

Confounding Cause 1.What is P(R | S)? 2.What is P(R | H, S)? 3.What is P(R | H,  S)? 4.What is P(R | H)? Happy? (H or  H) Sunny? (S or  S) Raise? (R or  R) S?S?P(D) S0.7 SS 0.3 R?R?P(R) R0.01 RR 0.99 Happy?Sunny?Raise?P(H|S,R)? HSR1.0 HS RR 0.7 H SS R0.9 H SS RR 0.1

Absolute vs. Conditional Independence Remember: R  S Does this mean that R  S | H ? In other words, P(R | H) =? P(R | H, S) Happy? (H or  H) Sunny? (S or  S) Raise? (R or  R)

D-Separation D-separation is the technical method for determining conditional independence in a BN. Active TripletsInactive Triplets …

D-Separation Node A is d-separated (short for directional-separated) from node B if all paths from A to B contain at least one inactive triplet. A  B | K 1, …, K m  nodes A and B are d-separated when nodes K 1, …, K m are known

D-Separation Quiz 1 C  A? C  A | B? C  D? C  D | A? E  C | D? D A B C E

D-Separation Quiz 2 A  E? A  E | B? A  E | C? A  B? A  B | C? C AB DE

D-Separation Quiz F  A? F  A | D? F  A | G? F  A | H? B AC D F E G H

Counting BN Parameters A complete joint distribution over 5 binary variables would require 31 = parameters. This BN requires 10 = parameters. C AB DE

Quiz A full joint over 6 binary variables requires = 63 parameters. How many parameters does this network require? C A BD E F

Quiz A full joint distribution over 7 binary variables requires = 127 parameters. How many parameters does this network require? D AC EG F B

Quiz A full joint distribution over 16 binary variables requires = 65,535 parameters. How many parameters does this network require? Battery dead Battery age Fan belt broken Battery meter No oil Battery flat Alternator broken Not charging No gas Starter broken Fuel line blocked Lights Gas gauge Oil light Car won’t start dipstick