Class session 7 Screening, validity, reliability Epidemiology 503, Section 2

Announcements Next class is in SPHII-1020 at 10 AM Following class (next Monday) will be review problems – no group work Please complete them BEFORE coming to class Additional problems and a practice exam will be available this week (Wednesday)

Last Class Looked at creating a summary mortality rate for populations with different age distributions Key concept: Mortality rates differ by age group so the number of deaths observed in a population is dependent on the age distribution

Rates from Population A Rates from Population B Because using age-specific death rates from populations typically only used in large groups For each population: Calculate age-specific mortality rates Multiply age-specific rates by the # of people in corresponding age range in standard population Sum expected # of deaths across age groups Divide total # of expected deaths by total standard population Result: Age-adjusted mortality rate for each population of interest Direct Method Rates from Population A Rates from Population B Applied to A standard population e.g. US population in 2000 Choice of standard is somewhat ARBITRARY.

Applied to the age distribution of the study population Useful when I don’t have or trust the group- specific rates (i.e. population is too small) Acquire age-specific mortality rates for standard population Multiply standard population’s age- specific rates by # of people in age range in population of interest Sum expected # of deaths across age groups in study population Divide observed # of deaths by expected # of deaths in population of interest SMR: observed # deaths per year expected # deaths per year >1 more deaths than expected =1 as expected <1 less deaths than expected Indirect Method Rates from the standard population Applied to the age distribution of the study population

This Class Screening: the use of testing to sort out apparently well persons (asymptomatic) who probably have disease from those who probably do not

The Natural History of Disease Lead Time: Interval by which the time of diagnosis is advanced by screening and early detection compared to usual time of diagnosis Critical Point: Point in the natural history of disease before which treatment is effective and/or less difficult to administer Herman CR et al. Screening for Preclinical Disease: Test and Diagnosis. American Journal of Roentgenology. 2002;179:825-831.

Valid Valid Reliable Reliable Valid Valid Reliable Reliable Validity: Ability of a test to distinguish between who has disease and who does not ACCURACY Reliability: Ability to replicate results on same sample if test is repeated PRECISION Valid Reliable Valid Reliable Citation: Hal Morgenstern, Epid 601 Coursepack 2011, Part 3 page 2.

Assessing Validity: Sensitivity Ability of a test to correctly identify those who have the disease Proportion of those who test positive for a disease among those that have disease Specificity Ability of a test to correctly identify those who do not have the disease Proportion of those who test negative among those who do not have disease

Calculating Sensitivity & Specificity Disease Status Screening Test Disease No Disease Positive TP (a) FP (b) Negative FN (c) TN (d) diseased who screen positive a all diseased (a+c) Sensitivity = = non-diseased who screen negative d all non-diseased (b+d) Specificity = =

Example Disease Status Screening Test Disease No Disease Positive 60 TP (a) FP (b) Negative FN (c) TN (d) Disease Status Screening Test Disease No Disease Positive 60 240 Negative 10 290 Sensitivity: 𝑇𝑃 𝑇𝑃+𝐹𝑁 = 𝑎 𝑎+𝑐 = 60 60+10 = 60 70 =0.857 or 85.7% Specificity: 𝑇𝑁 𝐹𝑃+𝑇𝑁 = 𝑑 𝑏+𝑑 = 290 240+290 = 290 530 =0.547 or 54.7%

Trade-Off of Sensitivity and Specificity Ideally we want a test that is 100% sensitive and specific Generally there is a trade-off between sensitivity and specificity As you move the cut point, the sensitivity and specificity will change (no disease) (disease)

Positive predictive value (PPV) Negative predictive value (NPV) Disease Status Screening Test Disease No Disease Positive TP (a) FP (b) Negative FN (c) TN (d) Positive predictive value (PPV) Proportion of people who have the disease among those that tested positive Negative predictive value (NPV) Proportion of people who do not have disease among those that tested negative NPV: 𝑇𝑁 𝐹𝑁+𝑇𝑁 = 𝑑 𝑐+𝑑 PPV: 𝑇𝑃 𝑇𝑃+𝐹𝑃 = 𝑎 𝑎+𝑏

Relationship between PPV and NPV and Prevalence

Why? What happens when the prevalence of a disease goes up? No Disease (-) Totals Screening Test Says Disease (+) A (True Positives) B (False Positives) A+B Screening Test Says No Disease (-) C (False Negatives) D (True Negatives) C+D A + C B + D A+C+B+D What happens when the prevalence of a disease goes up?

Why? PPV: A/(A+B) NPV = D/(C+D) Disease (+) No Disease (-) Totals Screening Test Says Disease (+) A (True Positives) B (False Positives) A+B Screening Test Says No Disease (-) C (False Negatives) D (True Negatives) C+D A + C B + D A+C+B+D PPV: A/(A+B) NPV = D/(C+D)

Implications Because of differences in disease prevalence the same test can have very different predictive values when administered to a high-risk vs. a low-risk group Test interpretation should be done taking into consideration the population In some cases it is appropriate to use different cut- points for different populations or risk groups

Assessing Reliability: Reliability (precision) is influenced by: Intrasubject variation Intraobserver variation Interobserver variation % Agreement: # cells that Agree Total # Observations Kappa Statistic: How much better is the agreement between observers than would be expected by chance alone? Observer 1 + - Total 48 10 58 7 35 42 55 45 100 Observer 2 x100 Example: (48+35)/100= 0.83*100 = 83% agreement between Observer 1 and 2