EBM-Diagnostic Testing K. Mae Hla, MD, MHS Primary Care Faculty Development Fellowship November 13, 2010

Objectives Develop pre-test probabilities Derive treatment thresholds Appraise evidence about a diagnostic test -validity, accuracy and applicability Calculate the results of diagnostic tests - sensitivity, specificity and likelihood ratios Apply evidence to patient care decisions

The Diagnostic Process Working diagnosis- pretest probability With each new finding/test we move from the pre-test probability to a new post-test probability Clinicians estimate probability of disease using probabilistic, prognostic and pragmatic approaches Compare disease probabilities to two thresholds

Applying Diagnostic Tests Example #1 8-year-old with fever, sore throat, swollen cervical glands and tonsillar exudates. No h/o cough. You order a rapid strep test.

What’s your pretest probability of the patient having group A strep pharyngitis? How much of a change would help you decide to treat, not treat or test further?

Treatment Thresholds ZONE OF UNCERTAINTY No Tx Tx 0% 100% Probability of Strep Pharyngitis 5%90%

Rapid Strep Test Results Rapid Strep test result comes back negative How does the rapid strep test result change the probability of the patient having or not having the disease? A positive rapid strep test raises post test probability of strep pharyngitis to 85% in one study A negative strep test decreases probability to 12 %

Pre-test Prob = 40% LR+ = LR- =

Treatment Thresholds ZONE OF UNCERTAINTY No Tx Tx 0% 12% 100% Probability of strep when rapid strep test is negative X 5%90%

Example #2 18-year-old female with ankle pain after a roller-blading accident. States unable to walk on her injured ankle. Exam demonstrates a slightly swollen ankle but no tenderness noted. Able to bear weight and take 4 full steps upon encouragement. What is the probability of ankle fracture? Do you need to order an ankle x-ray?

Ottowa Ankle rule An ankle x-ray is only necessary if there is pain near the malleoli and any of the following findings are present: inability to bear wt. both immediately and in the ED bone tenderness at the posterior edge or tip of the malleolus

How accurate is the Ottowa ankle rule in ruling out ankle fracture? A prospective validation study in > 1000 pts presenting to the ED with ankle pain Likelihood ratio + = 1.96 Likelihood ratio - = 0

Pre-test Prob = 10% LR+ = LR- =

Case 1 75-year-old woman with a hemoglobin of 10, MCV was 80 on routine checkup, a negative history and physical except osteoarthritis, and on no meds likely to suppress her marrow or cause a bleed Her probability of iron deficiency was 50% You want to avoid doing a bone marrow and order serum ferritin to diagnose iron deficiency anemia

Case 1 P: In an elderly symptomless woman with mild anemia I: how useful is serum ferritin C: O: in diagnosing iron deficiency anemia T(ype of question): Diagnosis T(ype of study): Prospective Cohort *Diagnosis of Iron Deficiency Anemia in the Elderly (Guyatt, et al. Am J Med, 1990;(88):

Three Main Questions Validity-Is this evidence about the accuracy of a diagnostic test valid? Results-Does this evidence show that this test can adequately distinguish patients who do and do not have the disorder? Applicability-How can I apply this valid, accurate diagnostic test to a specific patient?

Validity Measurement: was the gold standard measured independently? Representative: was the test evaluated in appropriate spectrum of patients? Ascertainment: was the reference test ascertained regardless of the diagnostic test?

All patients in study should have both the diagnostic test in question (blood test, history, physical exam) and the gold/reference standard test (autopsy, bone marrow, biopsy, angiogram) Independent- test not part of gold standard, decision to perform gold standard should not depend on result of diagnostic test under study Blinding- reference test readers should be unaware of results to avoid bias if tests/gold standard have subjective component- x-rays, biopsy, slides Validity- Measurement

Validity: Measurement Was there an independent blind comparison with a reference gold standard? The gold standard was the bone marrow aspirate results All patients got the serum ferritin and bone marrow done independently Marrow aspirates and iron deficiency status was determined by 2 hematologists unaware of the lab result

Validity: Representative Was the diagnostic test evaluated in an appropriate spectrum of patients? Examples: risk markers such as CEA were initially done in high risk patients

Validity: Representative Diagnostic uncertainty Patients with mild as well as severe symptoms Patients with early as well as late disease Patients with other commonly confused diagnoses

Study spectrum representative? Consecutive patients age 65 or older with anemia were recruited 36% of patients had iron deficiency anemia 44% had anemia of chronic disease 8% megaloblastic anemia Patients with other commonly confused disorders- different types of anemia and chronic medical conditions were included

Validity: Ascertainment Was the reference standard ascertained regardless of the diagnostic test result? Did all patients in the study both with and without iron deficiency anemia get the bone marrow done? Yes

Patients with negative diagnostic test may not get the gold standard done if the latter is invasive How do we prove for sure that the ones with negative tests truly do not have the disease or vice versa? Other ways to establish reference test Ascertainment-Continued

In the Pioped study looking at the utility of V/Q scan in patients with suspected pulmonary embolism-all patients with negative V/Q scan did not get pulmonary angiogram Clinical followup in a year was the additional reference standard to not miss patients with false negative VQ results

Results Does the test accurately distinguish between patients with and without the disorder? – –Sensitivity and specificity – –Likelihood ratios

Disease PresentAbsent Test Positive aba+b Negative cdc+d a+cb+d

Disease PresentAbsent Test Positive aTPb FP FPa+b Negative cFNd TN TNc+d a+cb+d

Disease PresentAbsent Test Positive aTPb FP FPa+b Negative cFNdTNc+d a+cSen=a/a+cb+dSp=d/d+b

Disease PresentAbsent Test Positive aTPb a+b Negative cFNdTNc+d a+cSen=a/a+cSp=d/d+bb+d“PID”“NIH”

Sensitive test-rules out the disease (SnNout) Test with high sensitivity (high TPR and very low false negative rate), negative test rules out the disease Examples: loss of retinal vein pulsation in increased intracranial pressure-the presence of pulsation (negative test) rules out IIP HIV antibody- negative test rules out HIV

Specific test – rules in the disease (SpPIN) Test with high specificity (high TNR, very low FPR)-positive test rules in the diagnosis Features of child with Down’s syndrome- very specific Presence of features (positive test) rules in the diagnosis Western blot confirmatory testing for HIV- high specificity: positive test rules in HIV disease

Likelihood Ratio likelihood of the test result in patients with the disease likelihood of the same result in patients without disease LR =

Disease PresentAbsent Test Positive aTPb FP FPa+b Negative cFNdTNc+d a+cSen=a/a+cb+dSp=d/d+b (+)LR= + test result in pts with dz (-)LR= - test result in pts with dz + test result in pts without dz - test result in pts without dz + test result in pts without dz - test result in pts without dz

Disease PresentAbsent Test Positive aTPb FP FPa+b Negative cFNdTNc+d a+cSen=a/a+cb+dSp=d/d+b (+)LR= + test result in pts with dz (-)LR= - test result in pts with dz + test result in pts without dz - test result in pts without dz + test result in pts without dz - test result in pts without dz = Sn/1-Sp = 1-Sn/Sp

Likelihood Ratio probability of the test result in patients with the disease probability of the same result in patients without disease LR =

Pre-Test Probability Pre-Test Odds Post-Test Odds Post-Test Probability Odds = Probability/1-probability Probability = Odds/1 + Odds

What do all these numbers mean?!? L.R.s indicate by how much a given diagnostic test result will raise or lower the pre-test probability of the target disorder L.R. of 1 = post-test probability is same as pre- test probability L.R. > 1 increases the probability that the target disorder is present; the higher the L.R., the greater the increase L.R. < 1 decreases the probability of the target disorder; the smaller the L.R., the greater the decrease

Effects of different likelihood ratios >10 or <0.1 generate large and often conclusive changes from pre- to post- test probability 5-10 and generate moderate shifts in pre- to post-test probability Depending on pre-test probability, change may or may not be large enough to influence Rx decision

Back to our patient Our patient’s serum ferritin comes back at 40 mmol/L How should we put all this together?

Iron Deficiency Anemia PresentAbsent Test Positive < a 15 15b Negative>45 c d135 a+c a+cb+da+b+c+d Totals

Low ferritin (<45) in diagnosing Fe def anemia Prevalence (study pre-test probability) = 85/235= 36% Sensitivity= True positive / all with disease = a/a+c = 70/85 = 82% Specificity = True neg / all without disease = d/b+d = 135/150 = 90%

Low ferritin (<45) in diagnosing iron deficiency anemia L.R.+ = sensitivity/(1-specificity) = 82%/10% = 8.2 L.R. -= (1-sens)/spec = 18%/90% = 0.2

Simplifying Likelihood Ratio Calculations Bone Marrow: iron deficient Bone Marrow: normal iron Test Results: < > Totals85150

Calculating Likelihood Ratios at 45 cut point Bone Marrow: iron deficient Bone Marrow: normal iron Likelihood Ratios Test Results: < /85= /150= /0.1= > /85= /150= /0.9= Totals85150

Pre-test Prob = 36% LR+ = LR- =

Applying the Test to the Patient Is the diagnostic test available, affordable, accurate and precise in our setting? Yes

Applicability (cont’d) Test needs to be available, affordable Interpreted in competent, reproducible fashion in clinical setting Potential consequences should justify the cost

Applicability (Cont’d) Are the study patients similar to our own? Are the results applicable to the patient in my practice? Will the patient be better off as a result of the test?

Applicability-study patients’ characteristics 235/259 patients had interpretable aspirates Mean age 79.7, 46% men, 72% had no medical diagnosis other than anemia Early dementia 25 CHF 25 COPD 25 Rheumatoid arthritis 17 Osteoarthritis 14 Pneumonia 13

Can we generate a clinically sensible estimate of our patient’s pre-test probability? How can we estimate pre-test probability? Clinical experience Regional or national prevalence statistics Practice databases Pretest probability observed in the study itself Studies of pre-test probabilities

Low = 3.6% (2-6) Wells PS, Anderson DR, Rodger M, et al. Derivation of a simple clinical model to categorize patients' probability of pulmonary embolism: increasing the model's utility with the SimpliRED D-dimer. Thromb Haemost 2000;83: Wells PS, Anderson DR, Rodger M, et al. Derivation of a simple clinical model to categorize patients' probability of pulmonary embolism: increasing the model's utility with the SimpliRED D-dimer. Thromb Haemost 2000;83: Risk of PE Inter = 20% (17-24) High = 67% (54-77)

Will the post-test probabilities affect our management and help our patient? Could the test result move us across a test-treatment threshold? Would the patient be willing to undergo the test? Would it help the patient?

Pre-test Prob = 50% LR+ = LR- =

Treatment Thresholds ZONE OF UNCERTAINTY No Tx Tx 0% 100% 90% Probability of Fe def Anemia when Ferritin is <45 10%90% x

Likelihood Ratios for 4 levels of Serum Ferritin Ferritin Fe def # Not Fe def L.R. < >18 18< >45 45< > Total85150

Calculating Likelihood Ratios Bone Marrow: iron deficient Bone Marrow: normal iron Likelihood Ratios Test Results: < /85= /150= /0.013= /85= /150= /0.087= /85= /150= /0.18= >10088/85= /150= /0.72= Totals85150

Clinical Scenario 52 y.o. male admitted to Orthopedics 3 days ago for a R femur fracture after falling from a ladder 52 y.o. male admitted to Orthopedics 3 days ago for a R femur fracture after falling from a ladder Underwent ORIF 2 days ago Underwent ORIF 2 days ago Last night developed SOB Last night developed SOB No CP or cough No CP or cough PE: Afeb 115/70 HR 110 RR 18 95% on 4L PE: Afeb 115/70 HR 110 RR 18 95% on 4L –Otherwise unremarkable (Lungs clear, No elevated JVP or RV heave, no lower ext swelling) Labs: EKG sinus tach, CXR clear Labs: EKG sinus tach, CXR clear D-dimer 0.5 mcg/mL (nl <0.6) D-dimer 0.5 mcg/mL (nl <0.6)

PICO P: In a patient with acute onset of SOB, hypoxia and sinus tachycardia after a major orthopedic surgery for femur fracture I: Does a negative D-dimer result C: O: Rule out pulmonary embolism T(ype of question): Diagnosis T(ype of study): Prospective Cohort *De Monye W et al: The performance of two rapid d-dimer assays in 287 patients with clinically suspected pulmonary embolism. Thrombosis Res 2002;(107):

Low = 3.6% (2-6) Wells PS, Anderson DR, Rodger M, et al. Derivation of a simple clinical model to categorize patients' probability of pulmonary embolism: increasing the model's utility with the SimpliRED D-dimer. Thromb Haemost 2000;83: Wells PS, Anderson DR, Rodger M, et al. Derivation of a simple clinical model to categorize patients' probability of pulmonary embolism: increasing the model's utility with the SimpliRED D-dimer. Thromb Haemost 2000;83: Risk of PE Inter = 20% (17-24) High = 67% (54-77)

Treatment Thresholds ZONE OF UNCERTAINTY No Tx Tx 0% 20% 100% Probability of Pulm Embolism X 5%90%

Sensitivity and Specificity Sensitivity (“Positive in disease”) = true pos / all disease = 74 / 90 = 82.2% Specificity (“Negative in health”) = true neg/all disease free = 120 / 197 = 60.3%

Likelihood Ratios of D-Dimer Test Likelihood ratio (+) = (74 / 90) / (77/197) = 2.1 Likelihood ratio (-) = (16/90) / (120/197) = 0.29

LR+ = LR- = Pre-test Prob = 20%

Treatment Thresholds ZONE OF UNCERTAINTY No Tx Tx 0% 5% 100% Probability of Pulm Embolism if D-Dimer is negative X 5%90%

LR (using lower cut off 95% sensitive D-Dimer value) Likelihood ratio + test = (86/90) / (149/197) = 1.3 Likelihood ratio - test = (4/90) / (48/197) = 0.18 probability of the test result in patients with the disease probability of the same result in patients without disease LR =

LR+ = LR- = Pre-test Prob = 20%

Treatment Thresholds ZONE OF UNCERTAINTY No Tx Tx 0% 100% 3% Probability of PE if D-Dimer is negative 3% 90% x

Summary Develop pre-test probabilities Steps in appraising a diagnostic test Calculate and interpret sensitivity, specificity and likelihood ratios Evaluate how pre-test probability and likelihood ratio results affect post-test probability of disease to influence further testing or treatment decisions