1 Epidemiological Measures I Screening for Disease

2 Terminology ReliabilityagreementReliability ≡ agreement of ratings/diagnoses, “reproducibility” –Inter-rater reliability –Inter-rater reliability ≡ agreement between two independent raters –Intra-rater reliability –Intra-rater reliability ≡ agreement of the same rater with him/herself ValidityValidity ≡ ability to discriminate without error AccuracyAccuracy ≡ a combination of reliability and validity

3 Validity Compare screening test results to a gold standard (“definitive diagnosis”) Each patient is classified as either true positive (TP), true negative (TN), false positive (FP), or false negative (FN) Test D+D−D−Total T+TPFPTP+FP T−FNTNFN+TN TotalTP+FNFP+TNN

4 Sensitivity Test D+D−D−Total T+TPFPTP+FP T−FNTNFN+TN TotalTP+FNFP+TNN SEN ≡ proportion of cases that test positive or the proportion of those with the disease of interest that our test is able to detect.

Sensitivity Sensitivity = TP/TP+FN Sensitivity =Test positive/Those with disease SnOUT; A very sensitivity test is useful because it allows us to rule out diseases. If you test negative on a very sensitive test then you are probably disease free.

6 Specificity SPEC ≡ proportion of noncases that test negative Test D+D−D−Total T+TPFPTP+FP T−FNTNFN+TN TotalTP+FNFP+TNN

Specificity sPIN ;a very specific test enables us to rule in those with a disease. A positive outcome on a specific test is most likely indicative of disease.

Attributes Both sensitivity and specificity are intrinsic attributes of a test. They do not change with the disease prevalence.

PREDICTIVE VALUES

10 Predictive Value Positive Test D+D−D−Total T+TPFPTP+FP T−FNTNFN+TN TotalTP+FNFP+TNN PVP ≡ proportion of positive tests that are true cases

11 Predictive Value Negative Test D+D−D−Total T+TPFPTP+FP T−FNTNFN+TN TotalTP+FNFP+TNN PVN ≡ proportion of negative tests that are true non-cases

12 Prevalence [True] prevalence = (TP + FN) / N Apparent prevalence = (TP + FP) / N Test D+D−D−Total T+TPFPTP+FP T−FNTNFN+TN TotalTP+FNFP+TNN

Predictive value attributes Both the positive and negative predictive values are affected by prevalence.

14 Conditional Probability Notation Pr(A|B) ≡ “the probability of A given B” For example Pr(T+|D+) ≡ “probability test positive given disease positive” = SENsitivity SPEC ≡ Pr(T−|D−) PVP = Pr(D+|T+) PVN= Pr(D−|T−)

15 Example Low Prevalence Population D+D−Total T+ T− Total ,000,000 Conditions: N = 1,000,000; Prevalence =.001 Prevalence = (those with disease) / N Therefore: (Those with disease) = Prevalence × N =.001× 1,000,000 = 1000

16 Example: Low Prevalence Population D+D−Total T+ T− Total ,000 1,000,000 Number of non-cases, i.e., TN + FP 1,000,000 – 1,000 = 999,000

17 Example: Low Prevalence Population D+D−Total T+990 T− Total1000 TP = SEN × (those with disease) = 0.99 × 1000 = 990 Assume test SENsitivity = 99% i.e., Test will pick up 99% of those with disease

18 Example: Low Prevalence Population D+D−Total T+990 T− 10 Total1000 FN = 1000 – 990 = 10 It follows that:

19 Example: Low Prevalence Population D+D−Total T+ T− 989,010 Total999,000 TN = SPEC × (those without disease) = 0.99 × 999,000 = 989,010 Suppose test SPECificity =.99 i.e., it will correctly identify 99% of the noncases

20 Example: Low Prevalence Population D+D−Total T+ 9,990 T−989,010 Total999,000 FPs = 999,000 – 989,010 = 9,900 It follows that:

21 Example: Low Prevalence Population D+D−Total T+9909,99010,980 T−10989,010989,020 Total ,0001,000,000 PVPT = TP / (TP + FP) = 990 / 10,980 = Low PVP It follows that the Predictive Value Positive is :

22 Example: Low Prevalence Population D+D−Total T+9909,99010,980 T−10989,010989,020 Total ,0001,000,000 PVNT= TN / (all those who test negative) = / =.9999 It follows that the Predictive Value Negative is:

23 Example: High prevalence population D+D−Total T+99,0009,000108,000 T−1,000891,000892,000 Total100,000900,0001,000,000 SEN = / 100,000 = 0.99 SPEC = 891,000 / 900,000 = 0.99 Prev = / 1,000,000 = 0.10 Same test parameters but used in population with true prevalence of.10

24 Example: High prevalence population D+D−Total T+99,0009,000108,000 T−1,000891,000892,000 Total100,000900,0001,000,000 PVP = 99,000 / 108,000 = 0.92 PVN = 891,000 / 892,000 = Prevalence = / 1,000,000 = 0.10 An HIV screening test is used in one million people. Prevalence in population is now 10%. SEN and SPEC are again 99%.

25 PVPT and Prevalence As PREValence increases PVPT also increases

26 Screening Strategy First stage  high SENS (don’t want to miss cases) Second stage  high SPEC (sort out false positives from true positives)

27 Selecting a Cutoff Point There is often an overlap in test results for diseased and non-diseased population Sensitivity and specificity are influenced by the chosen cutoff point used to determine positive results Example: Immunofluorescence test for HIV based on optical density ratio (next slide)

28 Low Cutoff High sensitivity and low specificity

29 High Cutoff Low sensitivity and high specificity

30 Intermediate Cutoff moderate sensitivity & moderate specificity

31 Again,Snout and Spin Best strategy is to start with a very sensitive screening test. A negative test on a sensitive test rules out the disease You then proceed to rule in or confirm the disease by doing a test with high specificity. A positive test on a specific test rules in or confirms the diagnosis.