Diagnosis II Dr. Brent E. Faught, Ph.D. Assistant Professor Faculty of Applied Health Sciences Brock University St.Catharines, ON, Canada DIAGNOSIS II Published reports of the performance of a diagnostic test typically contain values for sensitivity and specificity, as well as an estimate of the test’s predictive value for the particular study setting. The physician who wishes to use the test in a different practice setting must take into account the effect of any differences between the study population and the practice population on the predictive ability of the test. It is especially critical to assess the impact of any alteration in disease prevalence (Knapp and Miller, 1992). The objective of this lecture is to examine the overall accuracy of a test, likelihood of disease given a positive or negative clinical outcome as well as the predictive estimate of clinical tests as well as how these predictive estimates can be changed or revised based on the prevalence. Again, the following Website may help you if you are new to the concept of validating diagnostic tests. http://bmj.com/cgi/content/full/315/7107/540

Conditional Probabilities Gold Standard (D +) (D -) True Positive False Positive (T +) Clinical Test False Negative True Negative CONDITIONAL PROBABILITIES 2 X 2 Contingency Table: “ . . . the probability of an event occurring given () that another event has already occurred” (Knapp and Miller, 1992). True positive: “ . . . individuals with the condition who are correctly identified as diseased by the new test” (Knapp and Miller, 1992). False positive: “ . . . individuals without the condition who are falsely identified as diseased by the new test” (Knapp and Miller, 1992). This is also referred to as a mis-diagnosis. True negative: “ . . . individuals without the condition who are correctly identified as diseased-free by the new test” (Knapp and Miller, 1992). False negative: “ . . . individuals with the condition who are falsely identified as disease-free by the new test” (Knapp and Miller, 1992). This is also referred to as a missed diagnosis. (T -)

Interpretation of Diagnostic Data P (T +  D +) P (T +  D –) (T +) A B OR A / (A + C) B / (B + D) (T -) C D INTERPRETATION OF DIAGNOSTIC DATA Positive Likelihood Ratio: “ . . . the odds that an individual will encounter the disease given a positive test result” (Knapp and Miller, 1992). Formula: P (T +  D +) OR A / (A + C) P (T +  D –) B / (B + D)

Interpretation of Diagnostic Data P (T –  D +) P (T –  D –) (T +) A B OR C / (A + C) D / (B + D) (T -) C D INTERPRETATION OF DIAGNOSTIC DATA Negative Likelihood Ratio: “ . . . the odds that an individual will not encounter the disease given a negative test result” (Knapp and Miller, 1992). Formula: P (T –  D +) OR C / (A + C) P (T –  D –) D / (B + D)

Interpretation of Diagnostic Data (A + D) (T +) A B  (T -) C D INTERPRETATION OF DIAGNOSTIC DATA Accuracy: “ . . . proportion of individuals accurately identified as diseased or disease-free according to the clinical test results” (Knapp and Miller, 1992). Formula: (A + D) / 

Interpretation of Diagnostic Data P ( D + (T +) A  T + ) B OR __A__ (T -) C D INTERPRETATION OF DIAGNOSTIC DATA We often evaluate positive and negative predictive value since we usually don’t actually know whether the individual has the disease or not. Therefore, positive and negative predictive values are more valuable in terms of the diagnostic algorithm of disease, whereby we initially conduct a clinical evaluation and then follow-up with a confirmatory examination (Knapp and Miller, 1992). Predictive Value Positive: “ . . . probability that an individual with a positive test result has the disease. It is also the proportion of diseased individuals in the population of individuals with a positive test result. PVP is also known as the posterior probability, positive predictive value or posttest probability or disease” (Knapp and Miller, 1992). Formula: P (D +  T +) OR A / (A + B) (A + B)

Interpretation of Diagnostic Data P ( D - (T +) A  T - ) B OR __D__ (T -) C D INTERPRETATION OF DIAGNOSTIC DATA Predictive Value Negative: “ . . . probability that an individual with a negative test result does not have the disease. It is also the proportion of disease-free individuals in the population of individuals with a negative test result. PVN is also known as negative predictive value” (Knapp and Miller, 1992). Formula: P (D -  T -) OR D / (C + D) (C + D)

Revising Estimates of Diagnostic Test Performance 1. Select an arbitrary sample size. 2. Based on prevalence, calculate the diseased individuals. (T +) 76 4 228 1292 3. Calculate the disease-free individuals. 4. Calculate D + based on fixed sensitivity. (T -) 5. Calculate D - based on fixed specificity. REVISING ESTIMATES OF DIAGNOSTIC TEST PERFORMANCE Back Calculation Method: “A physician wishes to assess the predictive ability of a clinical test in a population in which the prevalence of breast cancer is 5%. Test sensitivity and specificity are unchanged; that is, they equal .95 and .85, respectively”. Step 1. Select an arbitrary sample size, for example 1600. Step 2. Based on a prevalence P (D+), of 5%, calculate the number of diseased individuals: 1600 x .05 = 80 Step 3. Calculate the number of disease-free individuals: 1600 - 80 = 1520 Step 4. Calculate the proportion of diseased individuals with a positive test, based on a sensitivity of .95: .95 x 80 = 76 . . . . then 80 - 76 = 4 Step 5. Calculate the proportion of disease-free individuals with a negative test, based on a specificity of .85: .85 x 1520 = 1292 . . . . Then 1520 - 1292 = 228 80 1520 1600

Effect of Prevalence on Predictive Values Rules of Prevalence and Predictive Values: - As prevalence rate increases; so to does positive predictive value while negative predictive value decreases. - As prevalence rate decreases; so to does positive predictive value, while negative predictive value increases. Prevalence (%)