Bayesian Nets and Applications Today’s Reading: C. 14 Next class: machine learning C. 18.1, 18.2 Questions on the homework?

2

3

4

5

6

7

8 Why is this useful?  Useful for assessing diagnostic probability from causal probability  P(cause|effect)= P(effect|cause)P(cause) P(effect)  Let M be meningitus, S be stiff neck P(m|s)=P(s|m)P(m) = 0.8 X = P(s) 0.1  Note: posterior probability of meningitus is still very small!

9 Naïve Bayes  What happens if we have more than one piece of evidence?  If we can assume conditional independence  Overslept and trafficjam are independent, given late  P(late|overslept Λ trafficjam) = αP(overslept Λ trafficjam)|late)P(late) = αP(overslept)|late)P(trafficjam|late)P(late)  Na ï ve Bayes where a single cause directly influences a number of effects, all conditionally independent  Independence often assumed even when not so

10 Bayesian Networks  A directed acyclic graph in which each node is annotated with quantitative probability information A set of random variables makes up the network nodes A set of directed links connects pairs of nodes. If there is an arrow from node X to node Y, X is a parent of Y Each node X i has a conditional probability distributionP(X i |Parents(X i ) that quantifies the effect of the parents on the node

11 Example  Topology of network encodes conditional independence assumptions

12 Smart Good test taker Understands material Hard working Exam GradeHomework Grade

13 Smart Good test taker Understands material Hard working Exam GradeHomework Grade Smart TrueFalse.5 Hard Working TrueFalse.7.3 SGood Test Taker TrueFalse True False SHWUM TrueFalse True TrueFalse.6.4 FalseTrue.6.4 False.2.8

14 Conditional Probability Tables Smart TrueFalse.5 Hard Working TrueFalse.7.3 SGood Test Taker TrueFalse True False SHWUM TrueFalse True TrueFalse.6.4 FalseTrue.6.4 False.2.8 GTTUMExam Grade ABCDF True TrueFalse FalseTrue False Homework Grade UMABCDF True False

15 Compactness  A CPT for Boolean X i with k Boolean parents has 2 k rows for the combinations of parent values  Each row requires one number p for X i =true (the number for X i =false is just 1- p)  If each variable has no more than k parents, the complete network requires O(nx2 k ) numbers  Grows linearly with n vs O(2 n ) for the full joint distribution  Student net: =11 numbers (vs. 26-1)=31

16 Conditional Probability A general version holds for joint distributions: P(PlayerWins,HostOpensDoor1)=P(PlayerWins|HostOpensDoor1)*P(Ho stOpensDoor1)

17 Global Semantics/Evaluation  Global semantics defines the full joint distribution as the product of the local conditional distributions: P(x 1,…,x n )=∏ i n =1 P(x i | Parents(X i )) e.g.,  P(EG=AΛGTΛ ⌐ UMΛSΛHW)

18 Global Semantics  Global semantics defines the full joint distribution as the product of the local conditional distributions: P(X 1,…,X n )=∏ i n =1 P(X i |Parents(X i )) e.g., Observations:S, HW, not UM, will I get an A?  P(EG=AΛGTΛ ⌐ UMΛSΛHW) = P(EG=A|GT Λ ⌐ UM)*P(GT|S)*P( ⌐ UM |HW ΛS)*P(S)*P(HW)

19 Conditional Independence and Network Structure  The graphical structure of a Bayesian network forces certain conditional independences to hold regardless of the CPTs.  This can be determined by the d- separation criteria

20 a b c a b c b a c Linear Converging Diverging

21 D-separation (opposite of d-connecting)  A path from q to r is d-connecting with respect to the evidence nodes E if every interior node n in the path has the property that either  It is linear or diverging and is not a member of E  It is converging and either n or one of its decendents is in E  If a path is not d-connecting (is d-separated), the nodes are conditionally independent given E

22 Smart Good test taker Understands material Hard working Exam GradeHomework Grade

23  S and EG are not independent given GTT  S and HG are independent given UM

Medical Application of Bayesian Networks: Pathfinder

25 Pathfinder  Domain: hematopathology diagnosis Microscopic interpretation of lymph- node biopsies Given: 100s of histologic features appearing in lymph node sections Goal: identify disease type malignant or benign Difficult for physicians

26 Pathfinder System  Bayesian Net implementation  Reasons about 60 malignant and benign diseases of the lymph node  Considers evidence about status of up to 100 morphological features presenting in lymph node tissue  Contains 105,000 subjectively- derived probabilities

27

28 Commercialization  Intellipath  Integrates with videodisc libraries of histopathology slides  Pathologists working with the system make significantly more correct diagnoses than those working without  Several hundred commercial systems in place worldwide

29 Sequential Diagnosis

30 Features  Structured into a set of 2-10 mutually exclusive values  Pseudofollicularity  Absent, slight, moderate, prominent  Represent evidence provided by a feature as F 1,F 2, … F n

31 Value of information  User enters findings from microscopic analysis of tissue  Probabilistic reasoner assigns level of belief to different diagnoses  Value of information determines which tests to perform next  Full disease utility model making use of life and death decision making  Cost of tests  Cost of misdiagnoses

32

33

34 Group Discrimination Strategy  Select questions based on their ability to discriminate between disease classes  For given differential diagnosis, select most specific level of hierarchy and selects questions to discriminate among groups  Less efficient  Larger number of questions asked

35

36

37 Other Bayesian Net Applications  Lumiere – Who knows what it is?

38 Other Bayesian Net Applications  Lumiere  Single most widely distributed application of BN  Microsoft Office Assistant  Infer a user’s goals and needs using evidence about user background, actions and queries  VISTA  Help NASA engineers in round-the-clock monitoring of each of the Space Shuttle’s orbiters subsystem  Time critical, high impact  Interpret telemetry and provide advice about likely failures  Direct engineers to the best information  In use for several years  Microsoft Pregnancy and Child Care  What questions to ask next to diagnose illness of a child

39 Other Bayesian Net Applications  Speech Recognition  Text Summarization  Language processing tasks in general