1. (Medical) Diagnostic Testing

2. The situation Patient presents with symptoms, and is suspected of having some disease. Patient either has the disease or does not have the disease. Physician performs a diagnostic test to assist in making a diagnosis. Test result is either positive (diseased) or negative (healthy).

3. Situation presented in a two-way table Test Result True Disease Status Diseased (+) Healthy (-) Diseased (+) Correct False Negative Healthy (-) False Positive Correct

4. Definitions False Positive: Healthy person incorrectly receives a positive (diseased) test result. False Negative: Diseased person incorrectly receives a negative (healthy) test result.

5. Goal Minimize chance (probability) of false positive and false negative test results. Or, equivalently, maximize probability of correct results.

6. Accuracy of tests in development Sensitivity: probability that a person who truly has the disease correctly receives a positive test result. Specificity: probability that a person who is truly healthy correctly receives a negative test result.

7. In conditional probability notation Sensitivity= P(Test +|True +) = P(Test+ and True+) ÷ P(True +) Specificity= P(Test -|True -) = P(Test- and True-) ÷ P(True -)

8. Example

9. Example (continued) Sensitivity = 49/85 = 0.58 Specificity = 1475/2893 = 0.51 CDC screening questionnaire about as good as tossing a coin in identifying kids with elevated and normal lead levels.

10. Interpretation of accuracy (Conditional) probabilities, so numbers between 0 and 1 Closer sensitivity is to 1, the more accurate the test is in identifying diseased individuals Closer specificity is to 1, the more accurate the test is in identifying healthy individuals

11. Alternatively False negative rate = P(Test -|True +) = 1 - P(Test +|True+) = 1 - sensitivity False positive rate = P(Test +|True -) = 1 - P(Test -|True -) = 1 - specificity

12. Example

13. Example (continued) False negative rate = 1 - 49/85 = 36/85 = 0.42 False positive rate= 1 -1475/2893 = 1418/2893 = 0.49

14. Accuracy of tests in use Positive predictive value: probability that a person who has a positive test result really has the disease. Negative predictive value: probability that a person who has a negative test result really is healthy.

15. In conditional probability notation Positive predictive value = P(True +|Test +) = P(True + and Test +) ÷ P(Test +) Negative predictive value = P(True -|Test -) = P(True - and Test -) ÷ P(Test -)

16. Example

17. Example (continued) Positive predictive value = 49/1467 = 0.033 Negative predictive value = 1475/1511 = 0.98 Kids who test positive have small chance in having elevated lead levels, while kids who test negative can be quite confident that they have normal lead levels.

18. Caution about predictive values! Reading positive and negative predictive values directly from table is accurate only if the proportion of diseased people in the sample is representative of the proportion of diseased people in the population. (Random sample!)

19. Example

20. Example (continued) Sens = 392/400 = 0.98 Spec = 576/600 = 0.96 PPV = 392/416 = 0.94 NPV = 576/584 = 0.99 Note prevalence of disease is 400/1000 or 40% Looks good?

21. Example

22. Example (continued) Sens = 49/50 = 0.98 Spec = 912/950 = 0.96 PPV = 49/87 = 0.56 NPV = 912/913 = 0.999 Sensitivity and specificity the same, and yet PPV smaller -- because prevalence of disease is smaller, namely 50/1000 or 5%.

23. Morals Don’t recklessly read PPV and NPV directly from tables without knowing prevalence of disease. PPV in screening tests naturally low, but not all that bad since NPV generally high.

24. Find correct predictive values by knowing…. True proportion of diseased people in the population. Sensitivity of the test Specificity of the test

25. Example: PPV of pap smears? Rate of atypia in normal population is 0.001 Sensitivity = 0.70 Specificity = 0.90 Find probability that a woman will have atypical cervical cells given that she had a positive pap smear.

26. Example

31. Example (continued) PPV = 70/10,060 = 0.00696 NPV = 89,910/89,940 = 0.999 Person with positive pap has tiny chance (0.6%) of truly having disease, while person with negative pap almost certainly will be disease free.

32. What to know How to calculate sensitivity and specificity Relationship between false negative (positive) rate and sensitivity (specificity) How disease prevalence affects predictive values How to calculate predictive values correctly Methodological standards…how good is that test?