1. Diagnostic Test Characteristics: What does this result mean
Diagnostic Test Characteristics: What does this result mean? Basic Research Lecture Series
Adam J. Singer, MD
Professor and Vice Chairman for Research
Department of Emergency Medicine
Stony Brook University

2. Why get diagnostic tests?
Rarely confirm or exclude disease with certainty
Results strengthen clinical estimate that disease likely or unlikely in particular patient
Test results often labeled as “positive” “negative”, “high” or “low probability”
Do not guarantee magnitude by which test results strengthen clinical assessment

3. Traditional measures of the diagnostic value of a test
Sensitivity
Specificity
These measure a test’s diagnostic discrimination compared to that of a criterion standard (that has 100% sensitivity and specificity)
Inherent characteristics of test unaffected by disease prevalence
Positive Predictive Value
Negative Predictive Value

4. Sensitivity
Measures the proportion of those with disease correctly identified by test
If my patient has disease, what is the chance that my test will detect it?
True Positives/all people with disease
= TP/(TP+FN)

5. Specificity
Measures the proportion of those without disease correctly identified as disease free
If my patient is healthy, what is the chance the test will be negative?
True Negatives/all healthy people
TN/(TN+FP)

6. Positive Predictive Value
The proportion of those with positive test who have disease
If the test is positive, how likely is it that my patient has the disease?
PPV=TP/all positives
PPV=TP/(TP+FP)

7. Negative Predictive Value
The proportion of those with a negative test who do not have the disease
If the test is negative, how likely is is that my patient is healthy?
NPV=TN/all negatives
NPV=TN/(TN+FN)

8. Two by Two Table Gold Standard Diseased Gold Standard Disease Free
Test Positive
A=number of diseased and + TP
B=number disease free and + FP
Test Negative
C=number diseased and – FN
D=number disease free and – TN
A+C=number with disease
B+D=number disease free

9. Example: Sensitivity=80%, Specificity=90%, Prevalence=10%
Disease
PPP=TP/(TP+FP)
PPP=80/(170)=47%
NPV=TN/(TN+FN)
NPV=810/830=98% + -
Test 80 90 20
810 +
170 -
830
100
900

10. Example: Sensitivity=80%, Specificity=90%, Prevalence=1%
Disease
PPP=TP/(TP+FP)
PPP=8/(107)=7%
NPV=TN/(TN+FN)
NPV=891/893=99.8% + -
Test 8 99 2
891 +
107 -
893 10
990

11. Problems with traditional measures of tests
Sensitivity and specificity
Refer to characteristics of test
Do not help determine likelihood of disease in any given patient
PPV and NPV
Highly dependent on prevalence of disease in
given population
Little interest in test quality in patients with known disease

12. Baye’s Theorem
Expresses the results of test in terms of how much it increases or decreases the existing prior clinical probability of disease
The likelihood that a positive or negative test is a true positive or negative depends on sensitivity and specificity of test as well as pretest probability that patients has the disease
Calculates probability of event given another event

13. Calculation of Baye’s Theorem
P (D+)P(T+D+)
P(D+)P(T+D+) + [1-P(D+)][1-P(T-D-)
P(D+T+)=
where:
P(D+T+) = probability of disease given a positive test
P(D+) = probability of disease
P(T+D+) = probability of a positive test given presence of disease
P(T-D-) =probability of a negative test given absence of disease

14. Likelihood Ratios or Playing the Odds
The likelihood of disease BEFORE testing (pre-test probability) is its prevalence in that particular population (local or published data)
The likelihood of disease AFTER knowing the test result is the post-test probability
LR measures the MAGNITUDE of change from initial assessment to post test assessment of disease probability
How will results of test change likelihood of disease

15. Likelihood Ratios
Diagnostic tests only useful if results substantially alter pre=test probability
Treatment threshold
Level of disease probability requiring no further testing and prompts treatment
Test threshold
Level of disease probability that effectively rules out disease requiring no further testing

17. Likelihood Ratios Measure accuracy of test
Ratio of a given test result in patients with disease to probability of the same test result in patients without disease
Indicates how much a given test result will increase or decrease probability of disease

18. Calculating Likelihood Ratios: Sensitivity 80%, Specificity 90%
+LR=
+LR= 0.8/0.1=8
1-Sensitivity
Specificity
-LR=
-LR= 0.2/0.9=0.2

19. Application of LR to clinical decision making
A useful diagnostic test has a very high or very low likelihood ratio
As LR approaches 1, utility of test decreases (probability remains the same)
Use of nomogram easiest way for clinician to calculate post test probability

20. Calculating post test probability from pretest probability and the LR
Pretest probability 5%, LR = 3.7
Convert probability to odds
Calculate posttest odds from pretest odds and LR
Convert odds back to probability
Probability
1-Probability
Pretest odds = = = 1 19
Posttest odds=Pretest odds x LR =
X 3.7 = 0.19
Odds
1+odds
Post test probability = =
= 0.16

22. Impact of LR on post test probability
High LR’s
Low LR’s
Effect on post-test probability
>10
<0.1
Large
5-1
Moderate
2-5
Small 1
No Change

23. Effect of LR’s 10 and 0.1 on qualitative ranges of pretest probability
Posttest probability (%) 10
10-30 (low)
53-80 (moderate to high)
30-60 (intermediate)
80-95 (high)
0.1
3-12 (low)
60-90 (high)
12-50 (low to intermediate)

24. Example 1 36 y/o female Sudden breathlessness Sharp pain in side
Well’s Criteria for PE
Alternative Diagnosis
DVT
Hemoptysis
Recent surgery
Cancer
Hr > 100
36 y/o female
Sudden breathlessness
Sharp pain in side
No h/o DVT/PE
No OCP or smoking
No hemoptysis
HR 95
Score = 0 Low Probability
(9.5%, 95% CI, 7.5% to 11.3%)

25. Post test probability = 17%
Pre test probability 9.5%
LR of + D-Dimer 2 ( )
LR of – D-Dimer 0.02 ( )
Post test probability = 17%
Post test probability = 0.2%
Well’s Ann Intern Med 1998;2001;135

26. Example 2 Obese 45 y/o female Breathlessness, bil leg swelling
H/o CHF, COPD
PE: tachypnea, bil rales
CXR: Cardiomegally, unerpenetrated DD
CHF
COPD PE

27. Pre test probability of CHF 50%
LR + BNP = 4.1
LR – BNP = 0.09
Post test probability = 78%
Post test probability = 8%
McCullough Acad Emerg Med 2003;10:275

28. Example 3 8 y/o boy with sore throat and fever
Erythema and anterior adenopathy
Clinical likelihood of + GABHS on culture 50% (pretest probability)
Rapid Strep ELISA +LR=20 (large effect), -LR=0.2 (moderate effect)
Likelihood of +GABHS given + Rapid Strep = 97%
Clinical decision: treat with antibiotics

29. Example 3 – continuation
Likelihood of +GABHS given negative Rapid Strep = 20%
Clinical decision: send formal throat culture

30. Example 4 – PIOPED Study High probability V/Q scan Normal V/Q scan
Sensitivity 41%, PPV 87%,
LR = 17 (post test odds 17 times higher than pretest probability)
If high pretest probability: treat
Normal V/Q scan
Sensitivity 2%, PPV 4%, specificity 96%, NPV 19%
LR = 0.1 (post test odds 10 times lower than pretest probability)
If low pretest probability: do not treat or conduct further testing

31. Example 4 - continuation
Intermediate probability V/Q scan
Sensitivity 41%, PPV 30%
LR=1 (post test probability the same as pretest)
Does not change pretest probability
Decision: useless result, conduct further testing
Low probability V/Q scan
Sensitivity 16%, PPV 14%
LR = 0.4 (likelihood of PE drops by 60%)
Decision: if pretest probability high, further testing

32. Example 5 BNP is a new diagnostic test for CHF Sensitivity 90%
Specificity 76%
Assume patient with SOB, smoker, no edema
Pre-test probability 20%
Post-test probability if BNP elevated?
Maisel et al. N Engl J Med 2002;347:161

33. Example 6 34 y/o male, acute flank pain
Diagnostic characteristics of hematuria
Sensitivity 84%
Specificity 74%
Probability of kidney stone
No RBC’s
Hematuria
Luchs et al. Urology 2002;59:839