1. EPI-820 Evidence-Based Medicine LECTURE 3: DIAGNOSIS I Mat Reeves BVSc, PhD

2. Objectives: 1. Understand the four main types of clinical diagnostic strategies. 2.Define and understand test characteristics (Se & Sp). 3.Understand how test characteristics influence ability to rule-in and rule-out disease. 4. Understand the construction and application of ROC curves. 5.Define and understand PV+ and PV- and the influence of prevalence.

3. I. Clinical Diagnostic Strategies Diagnosis = the process of discovering a patient’s underlying disease by: ascertaining the patient’s history, signs and symptoms, choosing appropriate tests, interpreting the results and, making correct conclusions. Highly complicated, not well understood process.

4. Four Basic Methods: A. Gestalt or Pattern Recognition recognition of a pattern of signs and/or symptoms that leads to an immediate diagnosis e.g., Parkinson’s disease, catatonia, myasthenia gravis signs or symptoms immediately rule-in diagnosis = pathognomonic very effective but limited to a few diseases. Equivalent to Specificity = 100% (i.e., no FP’s)

5. B. Exhaustion collect very complete patient history and physical examination. then hope diagnosis will become apparent. routinely taught, but rarely used (inefficient!!).

6. C. Branching algorithms clinical work-up presented as a series of logical steps results in a diagnosis, or a management plan. represents close to the idealized diagnostic process. useful esp. if don’t have extensive experience or rare condition.

7. D. Hypothetico-deductive reasoning strategy used by nearly all clinicians most all the time. formulate differential diagnoses list (hypotheses), then use tests to identify final diagnosis. 1. Formulate explanations (hypotheses) for the patient’s primary problem. 2. Consider those that are most likely and/or that are particularly harmful to miss (e.g., cerebral aneurysm for headache). 3. Simultaneously rule-out harmful/catastrophic and rule-in most likely. 4. Continue until one candidate has a very high likelihood (i.e., > 90%).

8. II. Clinical Test Characteristics A. Sensitivity and Specificity "Diagnostic test" can be applied to any piece of clinical information - patient’s history, physical examination or diagnostic procedures. Assume, dichotomous disease and diagnostic test results – hence, 4 possible interpretations: Two are correct true positive (TP) and true negative (TN) Two are incorrect false positive (FP) and false negative (FN)

9. Figure 1. Relationship between Diagnostic Test Result and Disease Status FP DISEASE PRESENT (D +) ABSENT (D -) TEST POSITIVE (T+) NEGATIVE (T-) TP FNTN a b cd FP

10. Sensitivity & Specificity Interpretation of diagnostic tests is concerned with comparing the relative frequencies and “costs” of the incorrect results (FNs and FPs) versus the correct results (TPs and TNs). Degree of overlap is a measure of the test effectiveness which is quantified by Se and Sp

11. D - D+D+ Diagnostic test result (continuous) T -T + Cut -point TNTP = FN = FP Figure 2. Results for a Typical Diagnostic Test Illustrating Overlap Between Disease (D+) and Non- disease (D-) Populations

12. Sensitivity (Se): defined as the proportion of individuals with disease that have a positive test result, or Se = TP = a. TP + FN a + c conditional probability of being test positive given that disease is present Se = P(T+ | D+). referred to as the true-positive rate. calculated solely from diseased individuals (LH column).

13. Specificity (Sp) defined as the proportion of individuals without disease that have a negative test result, or Sp = TN = d. TN + FP d + b conditional probability of being test negative given that disease is absent Sp = P(T-|D-) referred to as the true-negative rate. calculated solely from non-diseased individuals (RH Column).

14. Figure 3. Example of the Sensitivity and Specificity of Impendence Plethysmography and I-125 Fibrinogen Leg Scanning for DVT (Ref: Hull et al, Ann Int Med, 1981) FP DVT PRESENT (D +) ABSENT (D -) TEST Either or both POS (T+) Both NEG (T-) 103 11 152 a b cd 8 Se = 103/114 = 90% Sp = 152/160 = 95%

15. Figure 4. Example of a Perfectly Sensitive Test D - D+D+ Diagnostic test result T - T + Cut -point TNTP = FP

16. Tests with High Sensitivity perfectly sensitive test (Se = 100%), all diseased patients are test positive (no FN’s) all test negative patients are disease free (TNs) but usually many FPs exist. highly sensitive tests are used to rule-out disease - if the test is negative you can be confident that disease is absent (FN results are rare!). highly sensitive tests do not tell you if disease is present, because they provides no information regarding FP’s (see Sp). SnNout = if a sign, symptom or other diagnostic tests has a sufficiently high Sensitivity, a Negative result rules out disease.

17. Tests with High Sensitivity Three clinical scenarios where high sensitivity test should be used: 1) Early stages of a diagnostic work-up. –large number of potential diseases are being considered. –a negative result indicates a particular disease can be dropped (i.e., ruled out). 2) Important penalty for missing a disease. –Examples - TB, syphilis - dangerous but treatable conditions. –don’t want to miss cases, hence avoid false negative results 3) Screening tests. –the probability of disease is relatively low (i.e., low prevalence) –want to find as many asymptomatic cases as possible (incr. yield)

18. Table 1. Examples of Tests with High Sensitivities Disease/ConditionTestSensitivity Duodenal ulcerHistory of ulcer, 50+ yrs, pain relieved by eating or pain after eating 95% Favourable prognosis following non-traumatic coma Corneal reflex92% High intracranial pressureAbsence of spont. pulsation of retinal veins100% Deep vein thrombosisPositive to impedance plethysmography and/or fibrinogen leg scanning 92% Pancreatic cancerEndoscopic retrograde cholangio- pancreatography (ERCP) 95%

19. Tests with High Specificity perfectly specific test (Sp = 100%), all non-diseased patients test negative (no FP’s) all test positive patients have disease (TPs) but usually sizeable number of FNs highly specific tests are used to rule-in disease - if the test is positive you can be confident that disease is present (FPs are rare). highly specific tests do not tell you if disease is absent, because Sp provides no information regarding FNs (see Se). SpPin = if a sign, symptom or other diagnostic tests has a sufficiently high Specificity, a Positive result rules in disease.

20. D - D+D+ Diagnostic test result T - T + Cut -point TN TP = FN Figure 5. Example of a Perfectly Specific Test

21. Tests with High Specificity Clinical scenarios when high specificity tests should be used: 1) To rule-in a diagnosis suggested by other tests –specific tests are therefore used at the end of a work-up to rule-in a final diagnosis e.g., biopsy, culture, CT scan. 2) False positive tests results can harm patient –want to be absolutely sure that disease is present. –example, the confirmation of HIV positive status or the confirmation of cancer prior to chemotherapy.

22. Table 2. Examples of Tests with High Specificities Disease/ConditionTestSpecificity Alcohol dependencyYes to 3 or more of the 4 CAGE questions99.7% Iron-deficiency anemiaSerum ferritin90% Deep vein thrombosisNegative to impedance plethysmography and/or fibrinogen leg scanning 92% Pancreatic cancerEndoscopic retrograde cholangio- pancreatography (ERCP) 97% Breast cancerFine needle aspirate98% Strep throatPharyngeal gram stain96%

23. B. Trade-off Between Sensitivity and Specificity inherent trade-off between sensitivity and specificity. the location of the cut-point is arbitrary, and should be modified according to the purposes of the test. trade-off between Sp and Se cannot be avoided ideal cut-point depends on what the purpose of the test is - do you want high Se or high Sp or a balance between the two?

24. D - D+D+ Diagnostic test result T - T + Cut -point TNTP = FN = FP Figure 6. Trade-off Between Se and Sp: Lowering the Test Cut-point Increases Se but Decreases Sp

25. III. Receiver Operator Characteristic (ROC) Curves an alternative way to demonstrate the relationship between Se and Sp constructed by plotting sensitivity (or true positive rate) against the false positive rate (1 - Specificity) using various cut points.

26. Figure 7. A Receiver Operator Character- istic Curve (The Accuracy of the CK Test in the Diagnosis of Myocardial Infarction)

27. Uses for the ROC Curve 1. Comparing the usefulness of different tests increase Se and Sp: the further curve is pushed into the top left hand corner - best tests lie "to the north-west" worthless test has equal TP and FP rates - indicated the 45 degree diagonal (LR= 1.0). compare discriminating ability of tests by calculating the Area Under the ROC Curve – AUROCC varies from 0.5 (no ability) to 1.0 (perfect accuracy)

28. 2. Deciding on the Best Cut-point best cut-point is influenced by: the likelihood of disease (i.e., its prevalence), and the relative costs (or risk-benefit ratio) associated with FPs and FNs. optimal operating position is where the slope of the ROC curve equals: AC fp P(D-) AC fn P(D+)

29. IV. Prevalence and Predictive Values In terms of conditional probabilities Se and Sp can be defined as: Se = P(T+|D+) Sp = P(T-|D-) Problem: can only be calculated if the true disease status is known! But: the clinician is using a test precisely because the disease status is unknown! Clinician actually wants the conditional probability of disease given the test result, OR P(D+|T+)andP(D-|T-)

30. Predictive Value Positive (PVP) the probability of disease in a patient with a positive (abnormal) test. PVP = TP = a. TP + FP a + b conditional probability of being diseased given the test was positive, or PVP = P(D+|T+) note link between Sp and PVP via FP rate. A highly specific test rules-in disease because PVP is maximized.

31. Predictive Value Negative (PVN) the probability of not having disease when the test result is negative (normal). PVN = TN = d. TN + FN d + c conditional probability of not being diseased given the test was negative PVN = P(D-|T-). note the link between Se and PVN via the FN rate. A highly sensitive test rule-out disease because PVN is maximized.

32. Prevalence the proportion of the total population tested that have disease, or P = Total Number of Diseased Total Population (N) = TP + FN = a + c. TP+FN+FP+TN a + b + c + d very important

33. Prevalence can have a dramatic influence on predictive values Prevalence is the "third force" - often goes unnoticed only to reveal its influence in dramatic fashion! Equivalent names: the likelihood of disease, prior probability, prior belief, prior odds, pre-test probability and pre-test odds.

34. Figure 8. The PVP and PVN of the Combined IP and I-125 FS Tests in DVT When Prevalence of DVT is 42% FP DVT PRESENT (D +) ABSENT (D -) Either or both POS (T+) Both NEG (T-) 103 11152 a b cd 8 Se = 90%Sp = 95% PVP = 103/111 = 92.8% N = 114N = 160N = 274 PVN = 152/163 = 93.3%

35. Figure 9. The PVP and PVN of the Combined IP and I-125 FS Tests in DVT When Prevalence of DVT is 15% FP DVT PRESENT (D +) ABSENT (D -) Either or both POS (T+) Both NEG (T-) 27 3161 a b cd 9 Se = 90%Sp = 95% PVP = 27/36 = 75.0% N = 30N = 170N = 200 PVN = 161/164 = 98.2%

36. .20 0.40.60.80 Predictive Value Prevalence or Prior probability PV- 1.0 PV+ Figure 10. The PVP and PVN as a Function of Prevalence for a Typical Diagnostic Test

37. As prevalence falls, positive predictive value must fall along with it, and negative predictive value must rise. Conversely, as prevalence increases, positive predictive value will increase and negative predictive value will fall.

38. Summary I. Test Operating Characteristics Also Called Derived From Useful Result Most Affects Sensitivitytrue positive rate patients with disease negativeNegative predictive value Specificitytrue negative rate patients without disease positivePositive predictive value

39. II. Testing Situations Likely disease prevalence Need good....Use a test which is... Rule OutlowNegative predictive value Sensitive Rule InhighPositive predictive value Specific