1. Class session 7 Screening, validity, reliability
Epidemiology 503, Section 2

2. Announcements Next class is in SPHII-1020 at 10 AM
Following class (next Monday) will be review problems – no group work
Please complete them BEFORE coming to class
Additional problems and a practice exam will be available this week (Wednesday)

3. Last Class
Looked at creating a summary mortality rate for populations with different age distributions
Key concept: Mortality rates differ by age group so the number of deaths observed in a population is dependent on the age distribution

4. Rates from Population A Rates from Population B
Because using age-specific death rates from populations typically only used in large groups
For each population:
Calculate age-specific mortality rates
Multiply age-specific rates by the # of people in corresponding age range in standard population
Sum expected # of deaths across age groups
Divide total # of expected deaths by total standard population
Result: Age-adjusted mortality rate for each population of interest
Direct Method
Rates from Population A
Rates from Population B
Applied to
A standard population e.g. US population in 2000
Choice of standard is somewhat ARBITRARY.

5. Applied to the age distribution of the study population
Useful when I don’t have or trust the group- specific rates (i.e. population is too small)
Acquire age-specific mortality rates for standard population
Multiply standard population’s age- specific rates by # of people in age range in population of interest
Sum expected # of deaths across age groups in study population
Divide observed # of deaths by expected # of deaths in population of interest
SMR: observed # deaths per year
expected # deaths per year >1 more deaths than expected
=1 as expected <1 less deaths than expected
Indirect Method
Rates from the
standard population
Applied to the age distribution of the study population

6. This Class
Screening:
the use of testing to sort out apparently well persons (asymptomatic) who probably have disease from those who probably do not

7. The Natural History of Disease
Lead Time:
Interval by which the time of diagnosis is advanced by screening and early detection compared to usual time of diagnosis
Critical Point:
Point in the natural history of disease before which treatment is effective and/or less difficult to administer
Herman CR et al. Screening for Preclinical Disease: Test and Diagnosis. American Journal of Roentgenology. 2002;179:

8. Valid Valid Reliable Reliable Valid Valid Reliable Reliable
Validity: Ability of a test to distinguish between who has disease and who does not
ACCURACY
Reliability: Ability to replicate results on same sample if test is repeated
PRECISION
Valid
Reliable
Valid
Reliable
Citation: Hal Morgenstern, Epid 601 Coursepack 2011, Part 3 page 2.

9. Assessing Validity: Sensitivity
Ability of a test to correctly identify those who have the disease
Proportion of those who test positive for a disease among those that have disease
Specificity
Ability of a test to correctly identify those who do not have the disease
Proportion of those who test negative among those who do not have disease

10. Calculating Sensitivity & Specificity
Disease Status
Screening Test
Disease
No Disease
Positive TP
(a) FP
(b)
Negative FN
(c) TN
(d)
diseased who screen positive a
all diseased (a+c)
Sensitivity = =
non-diseased who screen negative d
all non-diseased (b+d)
Specificity = =

11. Example Disease Status Screening Test Disease No Disease Positive 60TP
(a) FP
(b)
Negative FN
(c) TN
(d)
Disease Status
Screening Test
Disease
No Disease
Positive 60
240
Negative 10
290
Sensitivity: 𝑇𝑃 𝑇𝑃+𝐹𝑁 = 𝑎 𝑎+𝑐 = = = or 85.7%
Specificity: 𝑇𝑁 𝐹𝑃+𝑇𝑁 = 𝑑 𝑏+𝑑 = = = or 54.7%

12. Trade-Off of Sensitivity and Specificity
Ideally we want a test that is 100% sensitive and specific
Generally there is a trade-off between sensitivity and specificity
As you move the cut point, the sensitivity and specificity will change
(no disease)
(disease)

13. Positive predictive value (PPV) Negative predictive value (NPV)
Disease Status
Screening Test
Disease
No Disease
Positive TP
(a) FP
(b)
Negative FN
(c) TN
(d)
Positive predictive value (PPV)
Proportion of people who have the disease among those that tested positive
Negative predictive value (NPV)
Proportion of people who do not have disease among those that tested negative
NPV: 𝑇𝑁 𝐹𝑁+𝑇𝑁 = 𝑑 𝑐+𝑑
PPV: 𝑇𝑃 𝑇𝑃+𝐹𝑃 = 𝑎 𝑎+𝑏

14. Relationship between PPV and NPV and Prevalence

15. Why? What happens when the prevalence of a disease goes up?
No Disease (-)
Totals
Screening Test Says Disease (+) A
(True Positives) B
(False Positives)
A+B
Screening Test Says No Disease (-) C
(False Negatives) D
(True Negatives)
C+D
A + C
B + D
A+C+B+D
What happens when the prevalence of a
disease goes up?

16. Why? PPV: A/(A+B) NPV = D/(C+D) Disease (+) No Disease (-) Totals
Screening Test Says Disease (+) A
(True Positives) B
(False Positives)
A+B
Screening Test Says No Disease (-) C
(False Negatives) D
(True Negatives)
C+D
A + C
B + D
A+C+B+D
PPV: A/(A+B) NPV = D/(C+D)

17. Implications
Because of differences in disease prevalence the same test can have very different predictive values when administered to a high-risk vs. a low-risk group
Test interpretation should be done taking into consideration the population
In some cases it is appropriate to use different cut- points for different populations or risk groups

18. Assessing Reliability:
Reliability (precision) is influenced by:
Intrasubject variation
Intraobserver variation
Interobserver variation
% Agreement:
# cells that Agree
Total # Observations
Kappa Statistic:
How much better is the agreement between observers than would be expected by chance alone?
Observer 1 + -
Total 48 10 58 7 35 42 55 45
100
Observer 2
x100
Example: (48+35)/100= 0.83*100
= 83% agreement between Observer 1 and 2