2009 Joint Statistical Meetings Washington, DC August 1-6, 2009 Interpreting Differential Effects in Light of Fundamental Statistical Tendencies James P. Scanlan Attorney at Law Washington, DC, USA Oral at

Summary 1. Factors that similarly affect two groups with different baseline rates of an outcome will tend to show a larger proportionate effect on the outcome for the group with the lower base rate but a larger proportionate effect on the opposite outcome for the other group 2. True subgroup effects can only be identified by determining the degree to which a factor shifts each group’s risk distribution

References Measuring Health Disparities page on jpscanlan.com – especially the Solutions tab Scanlan’s Rule page on jpscanlan.com – especially the Subgroup Effects tab Can We Actually Measure Health Disparities? (Chance 2006) Race and Mortality (Society 2000) Divining Difference (Chance 1994)

An illogical expectation There exists a tendency to regard it as somehow normal that a factor that similarly affects two groups’ susceptibilities to an outcome will cause the same proportionate change in the outcome rates for each group and to regard anything else as a differential effect (subgroup effect, interaction)

One reason the expectation is illogical Where two groups have different base rates of experiencing an outcome, a factor that has the same proportionate effect on each group’s rate of experiencing the outcome necessarily has a different proportionate effect on each group’s rate of avoiding the outcome E.g., Group A (10/90); Group B (20/80)

A logical expectation A factor that similarly affects two groups with different baseline rates of an outcome will tend to show a larger proportionate effect on the outcome for the group with the lower base rate but a larger proportionate effect on the opposite outcome for the other group Why is that logical?

Interpretive Rule 1/Scanlan’s Rule 1 When two groups differ in their susceptibility to an outcome, the rarer the outcome, the greater tends to be the relative difference in experiencing it and the smaller tends to be the relative difference in avoiding it.

Fig 1. Ratio of (1) Disadvantaged Group (DG) Fail Rate to Advantaged Group (AG) Fail Rate at Various Cutoff Points Defined by AG Fail Rate

Fig. 2. Ratios of (1) DG Fail Rate to AG Fail Rate and (2) AG Pass Rate to DG Pass Rate at Various Cutoff Points Defined by AG Fail Rate

Fig 3. Ratios of (1) Bl Rate of Falling below Various Income Levels to Wh Rate of Falling below Level and (2) Wh Rate of Falling above Level to Bl Rate of Falling above Level

Fig. 4. Ratio of (1) Bl to Wh Rate of Falling above Various SBP Levels and (2) Wh to Bl Rate of Falling below the Level (NHANES , , Men 45-64)

Implications of SR1 As an adverse becomes less prevalent relative difference in experiencing it tend to increase while relative differences in avoiding it tend to decline (e.g., test failure, poverty, hypertension, mortality). Relative differences in adverse outcomes tend to be larger among advantaged subpopulations while relative differences in favorable outcomes tend to be smaller among such subpopulations (e.g. race differences in health outcomes among the college-educated, race and SES differences in health outcomes among the young, SES differences in health outcomes among British Civil servants) Factors that decrease/increase the prevalence of an outcome will tend to have greater proportionate effect on group with lower base rate but greater proportionate effect on the opposite outcome for the other group

Problems Arising from SR1 When we observe standard patterns of changes in differences between group rates as prevalence of an outcome changes, how do we determine whether the difference between the groups’ situations actually changed in a meaningful way and how do we quantify the difference at each point in time (in a way that is unaffected by overall prevalence)? How do we identify genuine differential effects?

Estimated Effect Size Estimated effect size (EES) = estimated difference, in terms of percentage of a standard deviation, between means of hypothesized underlying, continuously- scaled normal distributions of factors associated with experiencing an outcome, derived from outcome rates of each group (see JSM 2008 and Solutions and Solutions Database tabs on jpscanlan.com)

Table 2 Illustration of the Solution (in terms of an adverse outcome decreasing in overall prevalence) Period Yr 1 Yr 2 AG Rate60% 42% DG Rate77% 60% Measures of Difference Change Direction AdvRatio Increase FavRatio Decrease EES (z) Decrease

Illustrations from Two Perspectives Perspective 1: Identifying/evaluating differential effects of factor on two groups Perspective 2: Comparing the size of the difference between the rates of two groups according to the presence or absence of a factor

Table 3. Comparison of Effects of Hypertension Control on Heart Attack Risk of Women and Men with Similar Risk Factor Profiles (A65,TC300,HDL50,NS, NM) (Perspective 1) Gender SBP 120 Risk SPB150 Risk Adverse Reduction Favorable IncreaseEES F4.0%7.0%42.9%3.2%0.28 M14.0%19.0%26.3%6.2%0.21

Table 4. Comparison of Gender Differences in Heart Attack Risk for Women and Men with Similar Risk Factor Profiles, by with and without Hypertension Control (A65,TC300,HDL50,NS, NM) (Perspective 2) SBP Fem Rate Male Rate Adverse Ratio Favorable RatioEES %14.0% %19.0%

Table 5: Comparison of Black-White Difference in Lung Cancer among 29 Cig Per Day (Haimon NEJM 2006) (Perspective 2) Cig/DayWhite RateBlack Rate Adverse Ratio Favorable RatioEES <110.7%2.2% >293.1%5.4%

Table 6. Comparison of Effects of Decreasing Smoking from >29 to <11 Cig Per Day on Whites and Blacks (Haimon NEJM 2006) (Perspective 1) (cor. 9/20/14) Race<11 CPD>29 CPD Adverse Reduction Favorable IncreaseEES W0.7%3.1%77.2%2.5%0.60 B2.2%5.9%63.4%4.0%0.42

Table 7. Comparison of Effects of Beta Blockers on Mortality Among Heart Patients at Different Ages (Gottlieb NEJM 1998) (Perspective 1) AgeBeta RateNo Beta Rate Adverse Reduction Favorable IncreaseEES <7011.3%18.7%39.6%9.1%0.34 >8022.6%33.1%31.7%15.7%0.32

Table 8. Comparison of Age Differences in Mortality Among Patients Treated and Not Treated with Beta Blockers (Gottlieb NEJM 1998) (Perspective 2) Treatment Status<70 Rate>80 Rate Adverse Ratio Favorable RatioEES Beta11.3%22.6% NoBeta18.7%33.1%

Issues Contrary literature regarding the patterns Clinical significance issue Statistical significance issues Re spurious patterns Re solutions approach Absolute minimum issue