Perverse Perceptions of the Impact of Pay for Performance on Healthcare Disparities 9th International Conference on Health Policy Statistics Cleveland, Ohio, October 5-7, 2011 James P. Scanlan Attorney at Law Washington, DC jps@jpscanlan.com

Four Main Points Standard measures of differences between outcome rates (proportions) are problematic for appraising the size of health and healthcare disparities because each is affected by the overall prevalence of an outcome. Healthcare disparities research is in disarray because of observers’ reliance on various measures without recognition of the way each measure is affected by the overall prevalence of an outcome. There exists only one answer to the question of whether a disparity has increased or decreased over time or is otherwise larger in one setting than another. That answer can be divined, albeit imperfectly, by deriving from each pair of outcome rates the difference between means of the underlying risk distributions.

References Measuring Health Disparities page (MHD) of jpscanlan.com (especially the Pay for Performance , Relative versus Absolute, and Solutions sub-pages and Section E.7) Scanlan’s Rule page of jpscanlan.com and its twenty sub-pages (especially the Immunization Disparities sub-page) Measurement Problems in the National Healthcare Disparities Report (APHA 2007) “Can We actually Measure Health Disparities?,” Chance 2006 “Race and Mortality,” Society 2000 “Divining Difference,” Chance 1994

Patterns of Distributionally-Driven Changes in Standard Measures of Differences Between Rates as an Outcome Increases in Overall Prevalence Relative differences in experiencing the outcome tend to decrease. Relative differences in failing to experience the outcome tend to increase. Absolute differences between rates tend to increase to the point where the first group’s rate reaches 50%; behave inconsistently until the second group’s rate reaches 50%; then decline. Absolute differences tend also to move in the same direction as the smaller relative difference. See Introduction to Scanlan’s Rule page for nuances. Differences measured by odds ratios tend to change in the opposite direction of absolute differences (hence to track the larger relative difference).

Fig 1: Ratios of (1) Advantaged Group (AG) Success Rate to Disadvantaged Group (DG) Success Rate, (2) DG Fail Rate to AG Fail Rate, and (3) DG Fail Odds to AG Fails Odds; and (4) Absolute Difference Between Rates

Fig 2: Absolute Difference Between Success (or Failure) Rates of AG and DG at Various Cutoffs

Other Illustrative Data Income data (Chance 2006) NHANES Illustrations Framingham Illustrations Life Table Illustration Other types of data: test scores of any sort, foot race results, propensity score data, mortgage eligibility ratings, etc.

Reminder One It does not matter that one observes departures from the described prevalence-related (distributionally-driven) patterns. Actual patterns are functions of both (a) the prevalence-related forces and (b) the differences between the underlying distributions in the settings being compared.

Reminder Two That the prevalence-related forces may depart from those I describe (e.g., distributions may be irregular) may indeed complicate efforts to appraise the size of disparities. But such possibility cannot justify reliance on standard measures of differences between outcome rates without consideration of the prevalence-related forces.

Reminder Three Do not find the points made here “interesting” then go on to do research using standard measures. If the points made here are valid, interpretations of patterns of changes using standard measures of differences between rates are invalid. They do not provide satisfactory results; they provide misleading results.

Key Government Approaches to Disparities Measurement National Center for Health Statistics (Health People 2010, 2020 etc.) relative differences in adverse outcomes Agency for Healthcare Research and Quality HRQ (National Healthcare Disparities Report) whichever relative difference (favorable or adverse) is larger Centers for Disease Control and Prevention (Jan. 2011 Health Disparities and Inequalities Report) absolute differences between rates

Table 1: Illustration from Werner (Circulation 2005) Data on White and Black CABG Rates Before and After Implementation of CABG Report Card (see Comment on Werner) (1) Period 2 Wh Rt (3) Bl Rt (4) Fav Ratio (5) Adv Ratio (6) Abs Df (PP) (7) OR (8) EES 1 3.60% 0.90% 4.00 1.03 2.70 4.11 0.58 8% 3% 2.67 1.05 5.00 2.81 0.48

Table 2: Illustrations from Escarce and McGuire (AJPH 2004) based on White and Black Coronary Procedure Rates, 1986 and 1997 (see Comment on Escarce and McGuire) (1) Proc (2) Year (3) Wh Rt (4) Bl Rt (5)Fav Ratio (6) Adv (7) AbsDf (8) EES Angrm 1986 8.56% 4.31% 1.99 1.05 4.25 0.25 1997 22.83% 16.10% 1.42 1.09 6.73 0.14 Angplst 0.99% 0.32% 3.09 1.01 0.67 0.32 2.57% 1.60% 1.61 0.97 0.15 ArtByp 3.06% 0.81% 3.78 1.02 2.25 0.41 5.86% 2.60% 1.03 3.26 0.27

Table 3: Illustration Based on Morita et. al Table 3: Illustration Based on Morita et. al. (Pediatrics 2008) Data on Black and White Hepatitis Vaccination Rates Pre and Post School-Entry Vaccination Requirement (see Comment on Morita) (1) Period (2) Grade (3) Year (4) Wh Rt (5) Bl Rt (6) Fav Ratio (7) Adv (8) AbsDf (9) EES PreRq 5 1996 8% 3% 2.67 1.05 5.0 0.47 Post 1 1997 46% 33% 1.39 1.24 13.0 0.34 Post 2 1998 50% 39% 1.28 1.22 11.0 0.29 9 32% 1.44 1.26 14.0 0.37 89% 84% 1.06 1.45 0.24 93% 1.04 1.57 4.0 0.26

Table 4: Illustration Based on Sehgal (JAMA 2003) Data on White and Black Rates of Adequate Hemodialysis (see Comment on Sehgal and Comment on Aaron) (1) Year (2) Wh Rt (3) Bl Rt (4) Fav Ratio (5) Adv Ratio (6)Abs Df (7) EES 1993 43% 36% 1.19 1.12 7.0 19 1994 53% 1.23 1.21 10.0 27 1995 62% 54% 1.15 8.0 21 1996 70% 63% 1.11 20 1997 73% 69% 1.06 4.0 13 1998 76% 1.09 1.25 6.0 1999 85% 83% 1.02 1.13 2.0 9 2000 87% 84% 1.04 3.0 15

Table 5: Illustration of Appraisals of the Comparative Degree of Employer Bias Using Different Measures of Disparities in Selection/Rejection (see , Relative versus Absolute sub-page of MHD) Employer AG Sel Rate DG Sel Rate RR Selection RR Rejection AbsDf OR A 20.0% 9.0% 2.22 (1) 1.14 (4) 0.11 (4) 2.53 (1) B 40.1% 22.7% 1.77 (2) 1.29 (3) 0.17(2) 2.29 (3) C 59.9% 40.5% 1.48 (3) 1.48 (2) 0.19 (1) 2.19 (4) D 90.0% 78.2% 1.15 (4) 2.18 (1) 0.12 (3) 2.50 (2) parenthetical numbers reflect the rankings of most to least discriminatory employer using the particular measure.

Conclusion Researchers and governmental bodies need generally to rethink the way they measure health and healthcare disparities. Certainly we do not want to start paying providers on the basis of perceived effects on healthcare disparities until measurement issues are resolved.