Chapter 10 Screening for Disease 4/24/2017 Chapter 10 Screening for Disease Screening for Disease

Terminology Reliability ≡ agreement of ratings/diagnoses, “reproducibility” Inter-rater reliability ≡ agreement between two independent raters Intra-rater reliability ≡ agreement of the same rater with him/herself Validity ≡ ability to discriminate without error Accuracy ≡ a combination of reliability and validity

Inter-Rater Reliability Two independent raters Cross-tabulate Observed proportion in agreement NOT adequate because a certain amount of agreement is due to chance Rater B Rater A + − Total a b g1 c d g2 f1 f2 N

Kappa (κ) [Agreement corrected for chance] Rater B Rater A + − Total a

κ Benchmarks

Example 1: Flip two coins To what extent are results reproducible? Toss B Toss A Heads Tails Total 25 50 100

To what extent are these diagnoses reproducible? Example 2 To what extent are these diagnoses reproducible? Rater B Rater A + − Total 20 4 24 5 71 76 25 75 100 “substantial” agreement

§10.3 Validity Compare screening test results to a gold standard (“definitive diagnosis”) Each patient is classified as either true positive (TP), true negative (TN), false positive (FP), or false negative (FN) Test D+ D− Total T+ TP FP TP+FP T− FN TN FN+TN TP+FN FP+TN N

SEN ≡ proportion of cases that test positive Sensitivity Test D+ D− Total T+ TP FP TP+FP T− FN TN FN+TN TP+FN FP+TN N SEN ≡ proportion of cases that test positive

SPEC ≡ proportion of noncases that test negative Specificity Test D+ D− Total T+ TP FP TP+FP T− FN TN FN+TN TP+FN FP+TN N SPEC ≡ proportion of noncases that test negative

Predictive Value Positive Test D+ D− Total T+ TP FP TP+FP T− FN TN FN+TN TP+FN FP+TN N PVP ≡ proportion of positive tests that are true cases

Predictive Value Negative Test D+ D− Total T+ TP FP TP+FP T− FN TN FN+TN TP+FN FP+TN N PVN ≡ proportion of negative tests that are true non-cases

Prevalence [True] prevalence = (TP + FN) / N Test D+ D− Total T+ TP FP TP+FP T− FN TN FN+TN TP+FN FP+TN N [True] prevalence = (TP + FN) / N Apparent prevalence = (TP + FP) / N

Conditional Probability Notation Pr(A|B) ≡ “the probability of A given B” For example Pr(T+|D+) ≡ “probability test positive given disease positive” = SENsitivity SPEC ≡ Pr(T−|D−) PVP = Pr(D+|T+) PVN= Pr(D−|T−)

Example Low Prevalence Population Conditions: N = 1,000,000; Prevalence = .001 D+ D− Total T+ T− 1000 1,000,000 Prevalence = (those with disease) / N Therefore: (Those with disease) = Prevalence × N = .001× 1,000,000 = 1000

Example: Low Prevalence Population Number of non-cases, i.e., TN + FP D+ D− Total T+ T− 1000 999,000 1,000,000 1,000,000 – 1,000 = 999,000

Example: Low Prevalence Population Assume test SENsitivity = .99, i.e., Test will pick up 99% of those with disease D+ D− Total T+ 990 T− 1000 TP = SEN × (those with disease) = 0.99 × 1000 = 990

Example: Low Prevalence Population It follows that: D+ D− Total T+ 990 T− 10 1000 FN = 1000 – 990 = 10

Example: Low Prevalence Population Suppose test SPECificity = .99 i.e., it will correctly identify 99% of the noncases D+ D− Total T+ T− 989,010 999,000 TN = SPEC × (those without disease) = 0.99 × 999,000 = 989,010

Example: Low Prevalence Population It follows that: D+ D− Total T+ 9,990 T− 989,010 999,000 FPs = 999,000 – 989,010 = 9,900

Example: Low Prevalence Population It follows that the Predictive Value Positive is : D+ D− Total T+ 990 9,990 10,980 T− 10 989,010 989,020 1000 999,000 1,000,000 PVPT = TP / (TP + FP) = 990 / 10,980 = 0.090 Strikingly low PVP!

Example: Low Prevalence Population It follows that the Predictive Value Negative is: D+ D− Total T+ 990 9,990 10,980 T− 10 989,010 989,020 1000 999,000 1,000,000 PVNT= TN / (all those who test negative) = 989010 / 989020 = .9999

Example: High prevalence population Same test parameters but used in population with true prevalence of .10 D+ D− Total T+ 99,000 9,000 108,000 T− 1,000 891,000 892,000 100,000 900,000 1,000,000 Prev = 100000 / 1,000,000 = 0.10 SEN = 99000 / 100,000 = 0.99 SPEC = 891,000 / 900,000 = 0.99

Example: High prevalence population An HIV screening test is used in one million people. Prevalence in population is now 10%. SEN and SPEC are again 99%. D+ D− Total T+ 99,000 9,000 108,000 T− 1,000 891,000 892,000 100,000 900,000 1,000,000 Prevalence = 100000 / 1,000,000 = 0.10 PVP = 99,000 / 108,000 = 0.92 PVN = 891,000 / 892,000 = 0.9989

PVPT and Prevalence As PREValence goes down, PVPT is affected Figure shows relation between PVP, PREV, & SPEC (test SEN = constant .99)

Screening Strategy First stage  high SENS (don’t want to miss cases) Second stage  high SPEC (sort out false positives from true positives)

Selecting a Cutoff Point There is often an overlap in test results for diseased and non-diseased population Sensitivity and specificity are influenced by the chosen cutoff point used to determine positive results Example: Immunofluorescence test for HIV based on optical density ratio (next slide)

High sensitivity and low specificity Low Cutoff High sensitivity and low specificity

Low sensitivity and high specificity High Cutoff Low sensitivity and high specificity

moderate sensitivity & moderate specificity Intermediate Cutoff moderate sensitivity & moderate specificity