1. Bayesian Nets and Applications

2. Naïve Bayes 2  What happens if we have more than one piece of evidence?  If we can assume conditional independence  Overslept and trafficjam are independent, given late  A and B are conditionally independent given C just in case B doesn't tell us anything about A if we already know C:  P(late|overslept Λ trafficjam) = α P(overslept Λ trafficjam)|late)P(late) = α P(overslept)|late)P(trafficjam|late)P(late)  Na ï ve Bayes where a single cause directly influences a number of effects, all conditionally independent  Independence often assumed even when not so

3. Bayesian Networks 3  A directed acyclic graph in which each node is annotated with quantitative probability information  A set of random variables makes up the network nodes  A set of directed links connects pairs of nodes. If there is an arrow from node X to node Y, X is a parent of Y  Each node X i has a conditional probability distributionP(X i |Parents(X i ) that quantifies the effect of the parents on the node

4. Example 4  Topology of network encodes conditional independence assumptions

5. 5 Smart Good test taker Understands material Hard working Exam GradeHomework Grade

6. 6 Smart Good test taker Understands material Hard working Exam GradeHomework Grade Smart TrueFalse.5 Hard Working TrueFalse.7.3 SGood Test Taker TrueFalse True.75.25 False.25.75 SHWUM TrueFalse True.95.05 TrueFalse.6.4 FalseTrue.6.4 False.2.8

7. Conditional Probability Tables Smart TrueFalse.5 Hard Working TrueFalse.7.3 SGood Test Taker TrueFalse True.75.25 False.25.75 SHWUM TrueFalse True.95.05 TrueFalse.6.4 FalseTrue.6.4 False.2.8 7 GTTUMExam Grade ABCDF True.7.25.03.01 TrueFalse.3.4.2.05 FalseTrue.4.3.2.08.02 False.05.2.3.15 Homework Grade UMABCDF True.7.25.03.01 False.2.3.4.05

8. Compactness 8  A CPT for Boolean X i with k Boolean parents has 2 k rows for the combinations of parent values  Each row requires one number p for X i =true (the number for X i =false is just 1-p)  If each variable has no more than k parents, the complete network requires O(nx2 k ) numbers  Grows linearly with n vs O(2 n ) for the full joint distribution  Student net: 1+1+2+2+5+5=11 numbers (vs. 26-1)=31

9. Conditional Probability 9

10. Global Semantics/Evaluation  Global semantics defines the full joint distribution as the product of the local conditional distributions: P(x 1,…,x n )=∏ i n =1 P(x i | Parents(X i )) e.g.,  P(EG=A Λ GT Λ ⌐ UM Λ S Λ HW) 10

11. Global Semantics  Global semantics defines the full joint distribution as the product of the local conditional distributions: P(X 1,…,X n )=∏ i n =1 P(X i |Parents(X i )) e.g., Observations:S, HW, not UM, will I get an A?  P(EG=A Λ GT Λ ⌐ UM Λ S Λ HW) = P(EG=A|GT Λ ⌐ UM)*P(GT|S)*P( ⌐ UM |HW Λ S)*P(S)*P(HW) 11

12. Conditional Independence and Network Structure 12  The graphical structure of a Bayesian network forces certain conditional independences to hold regardless of the CPTs.  This can be determined by the d-separation criteria

13. 13 a b c a b c b a c Linear Converging Diverging

14. D-separation (opposite of d-connecting) 14  A path from q to r is d-connecting with respect to the evidence nodes E if every interior node n in the path has the property that either  It is linear or diverging and is not a member of E  It is converging and either n or one of its decendents is in E  If a path is not d-connecting (is d-separated), the nodes are conditionally independent given E

15. 15 Smart Good test taker Understands material Hard working Exam GradeHomework Grade

16. 16  S and EG are not independent given GTT  S and HG are independent given UM

17. Medical Application of Bayesian Networks: Pathfinder

18. Pathfinder 18  Domain: hematopathology diagnosis  Microscopic interpretation of lymph-node biopsies  Given: 100s of histologic features appearing in lymph node sections  Goal: identify disease type malignant or benign  Difficult for physicians

19. Pathfinder System 19  Bayesian Net implementation  Reasons about 60 malignant and benign diseases of the lymph node  Considers evidence about status of up to 100 morphological features presenting in lymph node tissue  Contains 105,000 subjectively-derived probabilities

20. 20

21. Commercialization 21  Intellipath  Integrates with videodisc libraries of histopathology slides  Pathologists working with the system make significantly more correct diagnoses than those working without  Several hundred commercial systems in place worldwide

22. 22 Sequential Diagnosis

23. Features 23  Structured into a set of 2-10 mutually exclusive values  Pseudofollicularity  Absent, slight, moderate, prominent  Represent evidence provided by a feature as F 1,F 2, … F n

24. Value of information 24  User enters findings from microscopic analysis of tissue  Probabilistic reasoner assigns level of belief to different diagnoses  Value of information determines which tests to perform next  Full disease utility model making use of life and death decision making  Cost of tests  Cost of misdiagnoses

25. 25

26. 26

27. Group Discrimination Strategy 27  Select questions based on their ability to discriminate between disease classes  For given differential diagnosis, select most specific level of hierarchy and selects questions to discriminate among groups  Less efficient  Larger number of questions asked

28. 28

29. 29

30. Other Bayesian Net Applications 30  Lumiere – Who knows what it is?

31. Other Bayesian Net Applications 31  Lumiere  Single most widely distributed application of BN  Microsoft Office Assistant  Infer a user’s goals and needs using evidence about user background, actions and queries  VISTA  Help NASA engineers in round-the-clock monitoring of each of the Space Shuttle’s orbiters subsystem  Time critical, high impact  Interpret telemetry and provide advice about likely failures  Direct engineers to the best information  In use for several years  Microsoft Pregnancy and Child Care  What questions to ask next to diagnose illness of a child

32. Other Bayesian Net Applications 32  Speech Recognition  Text Summarization  Language processing tasks in general