1. Probabilistic Reasoning Bayesian Belief Networks Constructing Bayesian Networks Representing Conditional Distributions Summary

2. Bayesian Belief Networks (BBN) A Bayesian Belief Network is a method to describe the joint probability distribution of a set of variables. Let x1, x2, …, xn be a set of random variables. A Bayesian Belief Network or BBN will tell us the probability of any combination of x1, x2,.., xn.

3. Representation A BBN represents the joint probability distribution of a set of variables by explicitly indicating the assumptions of conditional independence through the following: a)Nodes representing random variables b)Directed links representing relations. c)Conditional probability distributions. d) The graph is a directed acyclic graph.

4. Example 1 Weather Cavity Toothache Catch

5. Example

6. Representation Each variable is independent of its non-descendants given its predecessors. We say x1 is a descendant of x2 if there is a direct path from x2 to x1. Example: Predecessors of Alarm: Burglary, Earthquake.

7. Joint Probability Distribution To compute the joint probability distribution of a set of variables given a Bayesian Belief Network we simply use the following formula: P(x1,x2,…,xn) = Π P(xi | Parents(xi)) Where parents are the immediate predecessors of xi.

8. Joint Probability Distribution Example: P(John, Mary,Alarm,~Burglary,~Earthquake) : P(John|Alarm) P(Mary|Alarm) P(Alarm|~Burglary ^ ~Earthquake) P(~Burglary) P(~Earthquake) = 0.00062

9. Conditional Probabilities Alarm Burglary Earthquake B E P(A) t t 0.95 t f 0.94 f t 0.29 f f 0.001

10. Probabilistic Reasoning Bayesian Belief Networks Constructing Bayesian Networks Representing Conditional Distributions Summary

11. Constructing Bayesian Networks Choose the right order from causes to effects. P(x1,x2,…,xn) = P(xn|xn-1,..,x1)P(xn-1,…,x1) = Π P(xi|xi-1,…,x1) -- chain rule Example: P(x1,x2,x3) = P(x1|x2,x3)P(x2|x3)P(x3)

12. How to construct BBN P(x1,x2,x3) x3 x2 x1 root cause leaf Correct order: add root causes first, and then “leaves”, with no influence on other nodes.

13. Compactness BBN are locally structured systems. They represent joint distributions compactly. Assume n random variables, each influenced by k nodes. Size BBN: n2 k Full size: 2 n

14. Probabilistic Reasoning Bayesian Belief Networks Constructing Bayesian Networks Representing Conditional Distributions Summary

15. Representing Conditional Distributions Even if k is small O(2 k ) may be unmanageable. Solution: use canonical distributions. Example: U.S. Canada Mexico North America simple disjunction

16. Noisy-OR Cold Flu Malaria Fever A link may be inhibited due to uncertainty

17. Noisy-OR Inhibitions probabilities: P(~fever | cold, ~flu, ~malaria) = 0.6 P(~fever | ~cold, flu, ~malaria) = 0.2 P(~fever | ~cold, ~flu, malaria) = 0.1

18. Noisy-OR Now the whole probability can be built: P(~fever | cold, ~flu, malaria) = 0.6 x 0.1 P(~fever | cold, flu, ~malaria) = 0.6 x 0.2 P(~fever | ~cold, flu, malaria) = 0.2 x 0.1 P(~fever | cold, flu, malaria) = 0.6 x 0.2 x 0.1 P(~fever | ~cold, ~flu, ~malaria) = 1.0

19. Continuous Variables Continuous variables can be discretized. Or define probability density functions Example: Gaussian distribution. A network with both variables is called a Hybrid Bayesian Network.

20. Continuous Variables Subsidy Harvest Cost Buys

21. Continuous Variables P(cost | harvest, subsidy) P(cost | harvest, ~subsidy) Normal distribution x P(x)

22. Probabilistic Reasoning Bayesian Belief Networks Constructing Bayesian Networks Representing Conditional Distributions Summary

23. Bayesian networks are directed acyclic graphs that concisely represent conditional independence relations among random variables. BBN specify the full joint probability distribution of a set of variables. BBN can by hybrid, combining categorical variables with numeric variables.