Diagnostic Test Characteristics: What does this result mean Diagnostic Test Characteristics: What does this result mean? Basic Research Lecture Series Adam J. Singer, MD Professor and Vice Chairman for Research Department of Emergency Medicine Stony Brook University

Why get diagnostic tests? Rarely confirm or exclude disease with certainty Results strengthen clinical estimate that disease likely or unlikely in particular patient Test results often labeled as “positive” “negative”, “high” or “low probability” Do not guarantee magnitude by which test results strengthen clinical assessment

Traditional measures of the diagnostic value of a test Sensitivity Specificity These measure a test’s diagnostic discrimination compared to that of a criterion standard (that has 100% sensitivity and specificity) Inherent characteristics of test unaffected by disease prevalence Positive Predictive Value Negative Predictive Value

Sensitivity Measures the proportion of those with disease correctly identified by test If my patient has disease, what is the chance that my test will detect it? True Positives/all people with disease = TP/(TP+FN)

Specificity Measures the proportion of those without disease correctly identified as disease free If my patient is healthy, what is the chance the test will be negative? True Negatives/all healthy people TN/(TN+FP)

Positive Predictive Value The proportion of those with positive test who have disease If the test is positive, how likely is it that my patient has the disease? PPV=TP/all positives PPV=TP/(TP+FP)

Negative Predictive Value The proportion of those with a negative test who do not have the disease If the test is negative, how likely is is that my patient is healthy? NPV=TN/all negatives NPV=TN/(TN+FN)

Two by Two Table Gold Standard Diseased Gold Standard Disease Free Test Positive A=number of diseased and + TP B=number disease free and + FP Test Negative C=number diseased and – FN D=number disease free and – TN A+C=number with disease B+D=number disease free

Example: Sensitivity=80%, Specificity=90%, Prevalence=10% Disease PPP=TP/(TP+FP) PPP=80/(170)=47% NPV=TN/(TN+FN) NPV=810/830=98% + - Test 80 90 20 810 + 170 - 830 100 900

Example: Sensitivity=80%, Specificity=90%, Prevalence=1% Disease PPP=TP/(TP+FP) PPP=8/(107)=7% NPV=TN/(TN+FN) NPV=891/893=99.8% + - Test 8 99 2 891 + 107 - 893 10 990

Problems with traditional measures of tests Sensitivity and specificity Refer to characteristics of test Do not help determine likelihood of disease in any given patient PPV and NPV Highly dependent on prevalence of disease in given population Little interest in test quality in patients with known disease

Baye’s Theorem Expresses the results of test in terms of how much it increases or decreases the existing prior clinical probability of disease The likelihood that a positive or negative test is a true positive or negative depends on sensitivity and specificity of test as well as pretest probability that patients has the disease Calculates probability of event given another event

Calculation of Baye’s Theorem P (D+)P(T+D+) P(D+)P(T+D+) + [1-P(D+)][1-P(T-D-) P(D+T+)= where: P(D+T+) = probability of disease given a positive test P(D+) = probability of disease P(T+D+) = probability of a positive test given presence of disease P(T-D-) =probability of a negative test given absence of disease

Likelihood Ratios or Playing the Odds The likelihood of disease BEFORE testing (pre-test probability) is its prevalence in that particular population (local or published data) The likelihood of disease AFTER knowing the test result is the post-test probability LR measures the MAGNITUDE of change from initial assessment to post test assessment of disease probability How will results of test change likelihood of disease

Likelihood Ratios Diagnostic tests only useful if results substantially alter pre=test probability Treatment threshold Level of disease probability requiring no further testing and prompts treatment Test threshold Level of disease probability that effectively rules out disease requiring no further testing

Likelihood Ratios Measure accuracy of test Ratio of a given test result in patients with disease to probability of the same test result in patients without disease Indicates how much a given test result will increase or decrease probability of disease

Calculating Likelihood Ratios: Sensitivity 80%, Specificity 90% +LR= +LR= 0.8/0.1=8 1-Sensitivity Specificity -LR= -LR= 0.2/0.9=0.2

Application of LR to clinical decision making A useful diagnostic test has a very high or very low likelihood ratio As LR approaches 1, utility of test decreases (probability remains the same) Use of nomogram easiest way for clinician to calculate post test probability

Calculating post test probability from pretest probability and the LR Pretest probability 5%, LR = 3.7 Convert probability to odds Calculate posttest odds from pretest odds and LR Convert odds back to probability Probability 0.5 1 1-Probability 0.95 19 Pretest odds = = = 1 19 Posttest odds=Pretest odds x LR = X 3.7 = 0.19 Odds 0.19 1+odds 1+0.19 Post test probability = = = 0.16

Impact of LR on post test probability High LR’s Low LR’s Effect on post-test probability >10 <0.1 Large 5-1 0.1-0.2 Moderate 2-5 0.2-0.5 Small 1 No Change

Effect of LR’s 10 and 0.1 on qualitative ranges of pretest probability Posttest probability (%) 10 10-30 (low) 53-80 (moderate to high) 30-60 (intermediate) 80-95 (high) 0.1 3-12 (low) 60-90 (high) 12-50 (low to intermediate)

Example 1 36 y/o female Sudden breathlessness Sharp pain in side Well’s Criteria for PE Alternative Diagnosis DVT Hemoptysis Recent surgery Cancer Hr > 100 36 y/o female Sudden breathlessness Sharp pain in side No h/o DVT/PE No OCP or smoking No hemoptysis HR 95 Score = 0 Low Probability (9.5%, 95% CI, 7.5% to 11.3%)

Post test probability = 17% Pre test probability 9.5% LR of + D-Dimer 2 (1.9-2.2) LR of – D-Dimer 0.02 (0.003-0.16) Post test probability = 17% Post test probability = 0.2% Well’s Ann Intern Med 1998;2001;135

Example 2 Obese 45 y/o female Breathlessness, bil leg swelling H/o CHF, COPD PE: tachypnea, bil rales CXR: Cardiomegally, unerpenetrated DD CHF COPD PE

Pre test probability of CHF 50% LR + BNP = 4.1 LR – BNP = 0.09 Post test probability = 78% Post test probability = 8% McCullough Acad Emerg Med 2003;10:275

Example 3 8 y/o boy with sore throat and fever Erythema and anterior adenopathy Clinical likelihood of + GABHS on culture 50% (pretest probability) Rapid Strep ELISA +LR=20 (large effect), -LR=0.2 (moderate effect) Likelihood of +GABHS given + Rapid Strep = 97% Clinical decision: treat with antibiotics

Example 3 – continuation Likelihood of +GABHS given negative Rapid Strep = 20% Clinical decision: send formal throat culture

Example 4 – PIOPED Study High probability V/Q scan Normal V/Q scan Sensitivity 41%, PPV 87%, LR = 17 (post test odds 17 times higher than pretest probability) If high pretest probability: treat Normal V/Q scan Sensitivity 2%, PPV 4%, specificity 96%, NPV 19% LR = 0.1 (post test odds 10 times lower than pretest probability) If low pretest probability: do not treat or conduct further testing

Example 4 - continuation Intermediate probability V/Q scan Sensitivity 41%, PPV 30% LR=1 (post test probability the same as pretest) Does not change pretest probability Decision: useless result, conduct further testing Low probability V/Q scan Sensitivity 16%, PPV 14% LR = 0.4 (likelihood of PE drops by 60%) Decision: if pretest probability high, further testing

Example 5 BNP is a new diagnostic test for CHF Sensitivity 90% Specificity 76% Assume patient with SOB, smoker, no edema Pre-test probability 20% Post-test probability if BNP elevated? http://araw.mede.uic.edu/cgi-bin/testcalc.pl Maisel et al. N Engl J Med 2002;347:161

Example 6 34 y/o male, acute flank pain Diagnostic characteristics of hematuria Sensitivity 84% Specificity 74% Probability of kidney stone No RBC’s Hematuria http://araw.mede.uic.edu/cgi-bin/testcalc.pl Luchs et al. Urology 2002;59:839

http://www.lrdatabase.com/index.php