1. Chapter 10 Screening for Disease
4/24/2017
Chapter 10
Screening for Disease
Screening for Disease

2. Terminology
Reliability ≡ agreement of ratings/diagnoses, “reproducibility”
Inter-rater reliability ≡ agreement between two independent raters
Intra-rater reliability ≡ agreement of the same rater with him/herself
Validity ≡ ability to discriminate without error
Accuracy ≡ a combination of reliability and validity

3. Inter-Rater Reliability
Two independent raters
Cross-tabulate
Observed proportion in agreement NOT adequate because a certain amount of agreement is due to chance
Rater B
Rater A + −
Total a b g1 c d g2 f1 f2 N

4. Kappa (κ) [Agreement corrected for chance] Rater B Rater A + − Total a

5. κ Benchmarks

6. Example 1: Flip two coins
To what extent are results reproducible?
Toss B
Toss A
Heads
Tails
Total 25 50
100

7. To what extent are these diagnoses reproducible?
Example 2
To what extent are these diagnoses reproducible?
Rater B
Rater A + −
Total 20 4 24 5 71 76 25 75
100
“substantial” agreement

8. §10.3 Validity
Compare screening test results to a gold standard (“definitive diagnosis”)
Each patient is classified as either true positive (TP), true negative (TN), false positive (FP), or false negative (FN)
Test D+ D−
Total T+ TP FP
TP+FP T− FN TN
FN+TN
TP+FN
FP+TN N

9. SEN ≡ proportion of cases that test positive
Sensitivity
Test D+ D−
Total T+ TP FP
TP+FP T− FN TN
FN+TN
TP+FN
FP+TN N
SEN ≡ proportion of cases that test positive

10. SPEC ≡ proportion of noncases that test negative
Specificity
Test D+ D−
Total T+ TP FP
TP+FP T− FN TN
FN+TN
TP+FN
FP+TN N
SPEC ≡ proportion of noncases that test negative

11. Predictive Value Positive
Test D+ D−
Total T+ TP FP
TP+FP T− FN TN
FN+TN
TP+FN
FP+TN N
PVP ≡ proportion of positive tests that are true cases

12. Predictive Value Negative
Test D+ D−
Total T+ TP FP
TP+FP T− FN TN
FN+TN
TP+FN
FP+TN N
PVN ≡ proportion of negative tests that are true non-cases

13. Prevalence [True] prevalence = (TP + FN) / N
Test D+ D−
Total T+ TP FP
TP+FP T− FN TN
FN+TN
TP+FN
FP+TN N
[True] prevalence = (TP + FN) / N
Apparent prevalence = (TP + FP) / N

14. Conditional Probability Notation
Pr(A|B) ≡ “the probability of A given B”
For example Pr(T+|D+) ≡ “probability test positive given disease positive” = SENsitivity
SPEC ≡ Pr(T−|D−)
PVP = Pr(D+|T+)
PVN= Pr(D−|T−)

15. Example Low Prevalence Population
Conditions: N = 1,000,000; Prevalence = .001 D+ D−
Total T+ T−
1000
1,000,000
Prevalence = (those with disease) / N
Therefore:
(Those with disease) = Prevalence × N
= .001× 1,000,000 = 1000

16. Example: Low Prevalence Population
Number of non-cases, i.e., TN + FP D+ D−
Total T+ T−
1000
999,000
1,000,000
1,000,000 – 1,000 = 999,000

17. Example: Low Prevalence Population
Assume test SENsitivity = .99, i.e.,
Test will pick up 99% of those with disease D+ D−
Total T+
990 T−
1000
TP = SEN × (those with disease)
= 0.99 × = 990

18. Example: Low Prevalence Population
It follows that: D+ D−
Total T+
990 T− 10
1000
FN = 1000 – 990 = 10

19. Example: Low Prevalence Population
Suppose test SPECificity = .99
i.e., it will correctly identify 99% of the noncases D+ D−
Total T+ T−
989,010
999,000
TN = SPEC × (those without disease)
= 0.99 × 999,000
= 989,010

20. Example: Low Prevalence Population
It follows that: D+ D−
Total T+
9,990 T−
989,010
999,000
FPs = 999,000 – 989,010 = 9,900

21. Example: Low Prevalence Population
It follows that the Predictive Value Positive is : D+ D−
Total T+
990
9,990
10,980 T− 10
989,010
989,020
1000
999,000
1,000,000
PVPT = TP / (TP + FP) = 990 / 10,980 = 0.090
Strikingly low PVP!

22. Example: Low Prevalence Population
It follows that the Predictive Value Negative is: D+ D−
Total T+
990
9,990
10,980 T− 10
989,010
989,020
1000
999,000
1,000,000
PVNT= TN / (all those who test negative)
= / = .9999

23. Example: High prevalence population
Same test parameters but used in population with true prevalence of .10 D+ D−
Total T+
99,000
9,000
108,000 T−
1,000
891,000
892,000
100,000
900,000
1,000,000
Prev = / 1,000,000 = 0.10
SEN = / 100,000 = 0.99
SPEC = 891,000 / 900,000 = 0.99

24. Example: High prevalence population
An HIV screening test is used in one million people. Prevalence in population is now 10%. SEN and SPEC are again 99%. D+ D−
Total T+
99,000
9,000
108,000 T−
1,000
891,000
892,000
100,000
900,000
1,000,000
Prevalence = / 1,000,000 = 0.10
PVP = 99,000 / 108,000 = 0.92
PVN = 891,000 / 892,000 =

25. PVPT and Prevalence As PREValence goes down, PVPT is affected
Figure shows relation between PVP, PREV, & SPEC (test SEN = constant .99)

26. Screening Strategy First stage  high SENS (don’t want to miss cases)
Second stage  high SPEC (sort out false positives from true positives)

27. Selecting a Cutoff Point
There is often an overlap in test results for diseased and non-diseased population
Sensitivity and specificity are influenced by the chosen cutoff point used to determine positive results
Example: Immunofluorescence test for HIV based on optical density ratio (next slide)

28. High sensitivity and low specificity
Low Cutoff
High sensitivity and low specificity

29. Low sensitivity and high specificity
High Cutoff
Low sensitivity and high specificity

30. moderate sensitivity & moderate specificity
Intermediate Cutoff
moderate sensitivity & moderate specificity