Analysis of matched data; plus, diagnostic testing

Correlated Observations Correlated data arise when pairs or clusters of observations are related and thus are more similar to each other than to other observations in the dataset. Ignoring correlations will: – overestimate p-values for within-person or within-cluster comparisons – underestimate p-values for between-person or between-cluster comparisons

Pair Matching: Why match? Pairing can control for extraneous sources of variability and increase the power of a statistical test. Match 1 control to 1 case based on potential confounders, such as age, gender, and smoking.

Example Johnson and Johnson (NEJM 287: , 1972) selected 85 Hodgkin’s patients who had a sibling of the same sex who was free of the disease and whose age was within 5 years of the patient’s…they presented the data as…. Hodgkin’s Sib control TonsillectomyNone From John A. Rice, “Mathematical Statistics and Data Analysis. OR=1.47; chi-square=1.53 (NS)

Example But several letters to the editor pointed out that those investigators had made an error by ignoring the pairings. These are not independent samples because the sibs are paired…better to analyze data like this: From John A. Rice, “Mathematical Statistics and Data Analysis. OR=2.14*; chi-square=2.91 (p=.09) Tonsillectomy None TonsillectomyNone Case Control

Pair Matching: example Match each MI case to an MI control based on age and gender. Ask about history of diabetes to find out if diabetes increases your risk for MI.

Pair Matching: example Which cells are informative? Just the discordant cells are informative! Diabetes No diabetes DiabetesNo Diabetes MI cases MI controls

Pair Matching Diabetes No diabetes DiabetesNo Diabetes MI cases MI controls OR estimate comes only from discordant pairs! The question is: among the discordant pairs, what proportion are discordant in the direction of the case vs. the direction of the control. If more discordant pairs “favor” the case, this indicates OR>1.

Diabetes No diabetes DiabetesNo Diabetes MI cases MI controls P(“favors” case/discordant pair) =

Diabetes No diabetes DiabetesNo Diabetes MI cases MI controls odds(“favors” case/discordant pair) =

Diabetes No diabetes DiabetesNo Diabetes MI cases MI controls OR estimate comes only from discordant pairs!! OR= 37/16 = 2.31 Makes Sense!

Diabetes No diabetes DiabetesNo Diabetes MI cases MI controls McNemar’s Test Null hypothesis: P(“favors” case / discordant pair) =.5 (note: equivalent to OR=1.0 or cell b=cell c)

Diabetes No diabetes DiabetesNo Diabetes MI cases MI controls McNemar’s Test Null hypothesis: P(“favors” case / discordant pair) =.5 (note: equivalent to OR=1.0 or cell b=cell c) By normal approximation to binomial:

McNemar’s Test: generally By normal approximation to binomial: Equivalently: exp No exp expNo exp ab c d cases controls

Diabetes No diabetes DiabetesNo Diabetes MI cases MI controls McNemar’s Test McNemar’s Test:

Example: McNemar’s EXACT test Split-face trial: – Researchers assigned 56 subjects to apply SPF 85 sunscreen to one side of their faces and SPF 50 to the other prior to engaging in 5 hours of outdoor sports during mid-day. The outcome is sunburn (yes/no). – Unit of observation = side of a face – Are the observations correlated? Yes. Russak JE et al. JAAD 2010; 62:

Results ignoring correlation: Table I -- Dermatologist grading of sunburn after an average of 5 hours of skiing/snowboarding (P =.03; Fisher’s exact test) Sun protection factorSunburnedNot sunburned Fisher’s exact test compares the following proportions: 1/56 versus 8/56. Note that individuals are being counted twice!

Correct analysis of data: Table 1. Correct presentation of the data (P =.016; McNemar’s exact test). SPF-50 side SPF-85 sideSunburnedNot sunburned Sunburned10 Not sunburned748 McNemar’s exact test: Null hypothesis: X~binomial (n=7, p=.5)

Standard error of the difference of two proportions= RECALL: 95% confidence interval for a difference in INDEPENDENT proportions Standard error can be estimated by:95% confidence interval for the difference between two proportions:

95% CI for difference in dependent proportions Variance of the difference of two random variables is the sum of their variances minus 2*covariance:

95% CI for difference in dependent proportions Diabetes No diabetes DiabetesNo Diabetes MI cases MI controls

The connection between McNemar and Cochran-Mantel-Haenszel Tests

View each pair is it’s own “age-gender” stratum Diabetes No diabetes Case (MI)Control Example: Concordant for exposure (cell “a” from before)

Diabetes No diabetes Case (MI)Control Diabetes No diabetes Case (MI)Control x 9 x 37 Diabetes No diabetes Case (MI)Control Diabetes No diabetes Case (MI)Control x 16 x 82

Mantel-Haenszel for pair- matched data We want to know the relationship between diabetes and MI controlling for age and gender (the matching variables). Mantel-Haenszel methods apply.

RECALL: The Mantel-Haenszel Summary Odds Ratio Exposed Not Exposed CaseControl ab c d

Diabetes No diabetes Case (MI)Control Diabetes No diabetes Case (MI)Control ad/T = 0 bc/T=0 ad/T=1/2 bc/T=0 Diabetes No diabetes Case (MI)Control Diabetes No diabetes Case (MI)Control ad/T=0 bc/T=1/2 ad/T=0 bc/T=0 x 9 x 37 x 16 x 82

Mantel-Haenszel Summary OR

Mantel-Haenszel Test Statistic (same as McNemar’s)

Concordant cells contribute nothing to Mantel- Haenszel statistic (observed=expected) Diabetes No diabetes Case (MI)Control Diabetes No diabetes Case (MI)Control

Discordant cells Diabetes No diabetes Case (MI)Control Diabetes No diabetes Case (MI)Control

From: “Large outbreak of Salmonella enterica serotype paratyphi B infection caused by a goats' milk cheese, France, 1993: a case finding and epidemiological study” BMJ 312: ; Jan Example: Salmonella Outbreak in France, 1996

Epidemic Curve

Matched Case Control Study Case = Salmonella gastroenteritis. Community controls (1:1) matched for:  age group ( = 65 years)  gender  city of residence

Results

In 2x2 table form: any goat’s cheese Goat’s cheese None 2930 Goat’ cheeseNone Cases Controls

In 2x2 table form: Brand A Goat’s cheese Goat’s cheese B None 1049 Goat’ cheese BNone Cases Controls

Brand A None Case (MI)Control Brand A None Case (MI)Control Brand A None Case (MI)Control Brand A None Case (MI)Control x8x8 x 24 x2x2 x 25

Summary: 8 concordant-exposed pairs (=strata) contribute nothing to the numerator (observed-expected=0) and nothing to the denominator (variance=0). Summary: 25 concordant-unexposed pairs contribute nothing to the numerator (observed-expected=0) and nothing to the denominator (variance=0). Using Agresti notation here!

Summary: 2 discordant “control-exposed” pairs contribute -.5 each to the numerator (observed-expected= -.5) and.25 each to the denominator (variance=.25). Summary: 24 discordant “case-exposed” pairs contribute +.5 each to the numerator (observed-expected= +.5) and.25 each to the denominator (variance=.25).

Diagnostic Testing and Screening Tests

Characteristics of a diagnostic test Sensitivity= Probability that, if you truly have the disease, the diagnostic test will catch it. Specificity=Probability that, if you truly do not have the disease, the test will register negative.

Calculating sensitivity and specificity from a 2x2 table +- +ab -cd Screening Test Truly have disease Sensitivity Specificity Among those with true disease, how many test positive? Among those without the disease, how many test negative? a+b c+d

Hypothetical Example Mammography Breast cancer ( on biopsy) Sensitivity=9/10= Specificity= 881/990 =.89 1 false negatives out of 10 cases 109 false positives out of 990

What factors determine the effectiveness of screening? The prevalence (risk) of disease. The effectiveness of screening in preventing illness or death. – Is the test any good at detecting disease/precursor (sensitivity of the test)? – Is the test detecting a clinically relevant condition? – Is there anything we can do if disease (or pre-disease) is detected (cures, treatments)? – Does detecting and treating disease at an earlier stage really result in a better outcome? The risks of screening, such as false positives and radiation.

Positive predictive value The probability that if you test positive for the disease, you actually have the disease. Depends on the characteristics of the test (sensitivity, specificity) and the prevalence of disease.

Example: Mammography Mammography utilizes ionizing radiation to image breast tissue. The examination is performed by compressing the breast firmly between a plastic plate and an x-ray cassette that contains special x-ray film. Mammography can identify breast cancers too small to detect on physical examination. Early detection and treatment of breast cancer (before metastasis) can improve a woman ’ s chances of survival. Studies show that, among year-old women, screening results in 20-35% reductions in mortality from breast cancer.

Mammography Controversy exists over the efficacy of mammography in reducing mortality from breast cancer in year old women. Mammography has a high rate of false positive tests that cause anxiety and necessitate further costly diagnostic procedures. Mammography exposes a woman to some radiation, which may slightly increase the risk of mutations in breast tissue.

Example A 60-year old woman has an abnormal mammogram; what is the chance that she has breast cancer? E.g., what is the positive predictive value?

Calculating PPV and NPV from a 2x2 table +- +ab -cd Screening Test Truly have disease PPV NPV Among those who test positive, how many truly have the disease? Among those who test negative, how many truly do not have the disease? a+cb+d

Hypothetical Example Mammography Breast cancer ( on biopsy) PPV=9/118=7.6% Prevalence of disease = 10/1000 =1% NPV=881/882=99.9%

What if disease was twice as prevalent in the population? Mammography Breast cancer ( on biopsy) sensitivity=18/20= specificity=872/980=.89 Sensitivity and specificity are characteristics of the test, so they don’t change!

What if disease was more prevalent? PPV=18/126=14.3% Prevalence of disease = 20/1000 =2% NPV=872/874=99.8% Mammography Breast cancer ( on biopsy)

Conclusions Positive predictive value increases with increasing prevalence of disease Or if you change the diagnostic tests to improve their accuracy.