1. 2008 Joint Statistical Meetings Denver, Colorado, August 2-7, 2008 Evaluating the Size of Differences Between Group Rates in Settings of Different Overall Prevalences James P. Scanlan Attorney at Law Washington, DC, USA jps@jpscanlan.com

2. Subjects 1. The problem with standard binary measures of differences between rates (relative differences, absolute differences, odds ratios): that all exhibit patterns of correlation with overall prevalence (i.e., among other things, they tend to change as overall prevalence changes) 2. A plausible alternative approach that avoids the problem with standard measures: a measure that does not change as overall prevalence changes

3. References Measuring Health Disparities page on jpscanlan.com – especially the Solutions tab Can We Actually Measure Health Disparities? (Chance 2006) (A12) Race and Mortality (Society 2000) (A10) The Misinterpretation of Health Inequalities in the United Kingdom (BSPS 2006) (B6)

4. Patterns by which Binary Measures Tend to Change as the Overall Prevalence of an Outcome Changes As an outcome increases from being very rare to being almost universal: 1. Relative differences in experiencing the outcome tend to decrease. 2. Relative differences in failing to experience the outcome tend to increase. 3. Odds ratios tend to change in the same direction as the larger of the two relative differences. 4. Absolute differences tend to change in the opposite direction of odds ratios.

5. Fig 1. Ratio of (1) Advantaged Group (AG) Success Rate to Disadvantaged Group (DG) Success Rate (Ratio 1) at Various Cutoffs Defined by AG Success Rate

6. Fig 2. Ratios of (1) AG Success Rate to DG Success Rate (Ratio 1) and (2) DG Fail Rate to AG Fail Rate (Ratio 2)

7. Fig 3. Ratios of (1) AG Success Rate to DG Success Rate (Ratio 1), (2) DG Fail Rate to AG Fail Rate (Ratio 2), and (3) DG Fail Odds to AG Fails Odds

8. Fig 4. Ratios of (1) AG Success Rate to DG Success Rate, (2) DG Fail Rate to AG Fail Rate, and (3) DG Fail Odds to AG Fails Odds; and Absolute Diff Between Rates

9. Table 1 Illustration of the Problem and Intimation of the Solution (in terms of a favorable outcome increasing in overall prevalence) Period Yr 1 Yr 2 Yr 3 Yr 4 AG Rate40% 58% 76% 94% DG Rate23% 39% 58% 85% Measures of Difference (Blue=decrease; Red=increase) Ratio 11.74 1.45 1.31 1.11 Ratio 21.28 1.49 1.75 2.50 Odds Ratio2.23 2.16 2.29 2.76 Absol Diff.17.19.18.09 EES (z).50.50.50.50

10. Estimated Effect Size Estimated effect size (EES) = estimated difference, in terms of a percentage of a standard deviation, between means of hypothesized underlying,continuously- scaled normal distributions of factors associated with experiencing an outcome, derived from adverse outcome rates of AG and DG

11. Table 2 Simplified Illustration of the Solution (in terms of a favorable outcome increasing in overall prevalence) Period Yr 1 Yr 2 AG Rate40% 58% DG Rate23% 40% Measures of Difference Change Direction Ratio 11.74 1.43 Decrease Ratio 21.28 1.45 Increase Odds Ratio2.23 2.07 Decrease Absol Diff.17.18 Increase EES (z).50.47 Decrease

12. Table 3 Illustration Based on Morita et. al. (Pediatrics 2008) Data on Black and White Hepatitis B Vaccination Rates Pre and Post School-Entry Vaccination Requirement (see D52) PeriodGradeYearWhRtBlRt Fav Ratio Adv RatioAbsDfEES Pre Requ5 (Y1) 19968%3%2.671.050.050.47 Post Requ5 (Y2) 199746%33%1.391.240.130.34 Pre Requ9 (Y1) 199646%32%1.441.260.140.37 Post Requ9 (Y2) 199789%84%1.061.450.050.24

13. Table 4 Illustrations Based on Escarce and McGuire (AJPH 2004) Data on White and Black Coronary Procedure Rates, 1986 and 1997 (see D48) ProcYearWh RtBl Rt Fav Ratio Adv RatioAbsDfEES Angrm (Y1) 19868.56%4.31%1.991.050.040.25 Angrm (Y2) 199722.83%16.10%1.421.090.070.14 Angpls (Y1) 19860.99%0.32%3.091.010.010.32 Angpls (Y2) 19972.57%1.60%1.611.010.010.15 ArtByp (Y1) 19863.06%0.81%3.781.020.020.41 ArtByp (Y2) 19975.86%2.60%2.251.030.030.27

14. Table 5: Illustration Based on Hetemaa et al. (JECH 2003) Data on Finnish Revascularization Rates, 1988 and 1996, by Income Group (see D21, D58) GenderYear AG RevRt LowInc RevRt Fav Ratio Adv RatioAbsDfEES M (Y1) 198817.9%8.3%2.161.12.0960.48 M (Y2) 199641.2%25.4%1.631.27.1590.44 F (Y1) 198810.0%3.7%2.701.07.0630.51 F (Y2) 199630.8%17.1%1.801.20.1370.45

15. Problems with the Solution Always practical issues (we do not really know the shape of the underlying distributions) Sometimes fundamental issues (e.g., where we know distributions are not normal because they are truncated portions of larger distributions, see D43) Absolute minimum issue (D43,B6)

16. Conclusion If we are mindful of the problems, the approach provides a framework for cautiously appraising the size of differences between rates in different settings. Regardless of problems, the approach is superior to reliance on standard binary measures of differences between rates without regard to the way those measures tend to be correlated with overall prevalence of an outcome.