1. D EPARTMENT of F AMILY M EDICINE Comparing Myocardial Infarction Mortality Rates in Rural and Urban Hospitals : Rethinking Measures of Quality of Care Presenters: Paul James MD and Pengxiang (Alex) Li Co-author Marcia M. Ward, PhD Support grant: Agency for Healthcare Research and Quality Grant # HS015009

2. D EPARTMENT of F AMILY M EDICINE Introduction  Observational studies find that the quality of care for myocardial infarction (MI) patients admitted to rural hospitals is substandard (Sheikh 2001, Baldwin 2004)  Lower volumes of MI patients in rural hospitals  Lacking cardiologists  Lacking support services

3. D EPARTMENT of F AMILY M EDICINE Introduction  Validity of these observational studies has been questioned  Unbalanced comparison groups  Patients admitted to rural hospitals tend to be older, poorer, in poorer health, and have greater number of comorbidities (Baldwin 2004, Chen 2000, Frances 2000)  Referral patterns of rural provider  Empirical study showed that less severe patients were referred to urban hospitals (Metha 1999)  Unmeasured confounding may account for differences in patient outcomes

4. D EPARTMENT of F AMILY M EDICINE Objectives of the study  To compare characteristics of MI patients admitted to rural and urban hospitals  To examine in-hospital mortality between rural and urban hospitals among MI patients  Using traditional risk adjustment techniques (Logistic regression)  Using instrumental variable methods (IV)

5. D EPARTMENT of F AMILY M EDICINE Introduction: Objectives of Presentation Instrumental Variable Technique is a strategy to control for unmeasured confounding An example: Wehby (2006) found that using the logistic regression model, early initiation of prenatal care is associated with a higher probability of low birth weight (LBW) Unmeasured confounders: women at a higher risk demand more (or early) prenatal care compared to those at lower risk. IV estimations showed that early time to prenatal care initiation is associated with a lower probability of LBW.

6. D EPARTMENT of F AMILY M EDICINE Introduction: Objective of Presentation Instrumental Variable Technique is a strategy to control for unmeasured confounding  RCT: Insert variable to control for unmeasured confounders by Randomization (e.g. to Coin Flip)  Observational Study: Insert an Instrumental Variable (like Coin Flip) that affects the intervention but not the outcome under study. (Pseudo-Randomization)

7. D EPARTMENT of F AMILY M EDICINE Methods: Data  Discharge data from Iowa State Inpatient Dataset (2002 & 2003)  Inclusion criteria  A principal diagnosis of MI (ICD-9-CM: 410.01-410.91)  Eighteen years or older  Exclusion criteria  The hospital identification number was missing (n=9)  Patient’s whose home county was not in Iowa (n=1,248)  Patients’ zip code was missing (n=14)  Patients’ sex was missing (n=1)  Our primary analyses also excluded patients discharged or transferred to another short term general hospital for inpatient care (n=1,618)  Most of our analyses are based on 12,191 MI patients

8. D EPARTMENT of F AMILY M EDICINE Methods: Variables  Dependent variable  In-hospital mortality  Independent variables  Urban vs Rural hospitals that patients admitted to  Urban: 27 hospitals  Rural: 89 hospitals  Payer: e.g. Medicare, private insurance, self-pay  Admission type: e.g. emergency  Race  Risk adjustment index  Charlson comorbidity index  All Patient Refined DRGs (APR-DRGs) risk index

9. D EPARTMENT of F AMILY M EDICINE Methods: Traditional Analytic Approach (Logistic Regression)  Univariate analyses of group comparisons  Chi-square tests for dichotomous data  ANOVAs for continuous data  Logistic regressions for multiple regression analyses

10. D EPARTMENT of F AMILY M EDICINE Methods: Pitfalls with Logistic Regression  Using administrative inpatient data, one cannot control all patients’ risk factors (e.g. severity of illness)  If unmeasured variables are related to selection of the hospital, the estimates of the hospital-specific contribution to mortality will be biased.  For example, elderly MI patients with severe comorbid conditions, which are unmeasured in administrative data, might prefer to remain in the rural hospitals.  As a result, a higher risk-adjusted mortality rate in rural hospitals might simply be due to more severe patients in rural hospitals.

11. D EPARTMENT of F AMILY M EDICINE Approaches to Minimize Bias  Collect all the relevant patient-level variables: very costly  Randomized controlled trial  Not feasible  Instrumental variable (IV) estimation  An econometric technique which enables us to obtain unbiased estimates of treatment effects in observational studies

12. D EPARTMENT of F AMILY M EDICINE The Instrumental Variable (IV) estimation  IVs are used to achieve a “pseudo-randomization”  The instrumental variable technique can extract variation in the focal variable (rural hospital selection) that is unrelated to unmeasured confounders, and employ this variation to estimate the causal effect on an outcome  Assumptions for IV(s) 1. IV(s) should correlate with treatment variable (choice of urban/rural hospital) 2. IV(s) should not be correlated with the unmeasured confounders (underlying pathology related to MI)

13. D EPARTMENT of F AMILY M EDICINE Methods: Instrumental Variable Technique  Instrumental Variable = Patients’ distance to the nearest urban hospital  The distances between each patient’s home and all urban hospitals in Iowa were obtained by calculating the distances between the centroids of each patient’s resident zip code and all urban hospitals’ zip codes.  Similar to Brooks (2003) approach, instrumental variables in the study are binary variables that group patients based on the their distance to the nearest urban hospital.

14. D EPARTMENT of F AMILY M EDICINE Methods: IV Technique: First assumption  Patients who live closer to an urban hospital are more likely to choose an urban hospital than those who live farther away.  Partial F-statistics for the IVs in the first stage regression  Small values of first-stage F-statistics imply failure of assumption 1  Rule of thumb: F>10 indicates good association (Staiger 1997)

15. D EPARTMENT of F AMILY M EDICINE Methods: IV Technique Second Assumption:  Distance to the nearest urban hospital is not associated with the severity or pre-morbid risks of patients with MI  Descriptive comparison between two groups of patients classified by IV  If the instrument is independent of the unmeasured confounders, it should also be independent of observed risk factors (e.g. age, and comorbidity index).  Over-identifying restrictions tests  The null hypothesis is that the IV is not correlated with unmeasured confounders

16. D EPARTMENT of F AMILY M EDICINE Methods: IV Technique  To examine the robustness of our findings:  We used a range of patients’ groups for the instrumental variable (2, 4, 8, and 12 groups).  We varied the independent variables.  The syslin two-stage least squares (2SLS) procedure in SAS 9.1 was used to do IV estimation.

17. D EPARTMENT of F AMILY M EDICINE Results: Table 1: Baseline characteristics of MI patients* admitted to rural and urban hospitals VariablesRural (N= 1,426) Urban (N= 10,765) p-value Age82.3568.89<.0001 Male (%)45.0259.76<.0001 Black (%)0.141.130.0004 Number of secondary diagnoses5.665.610.43 Charlson comorbidity index0.960.69<.0001 APR-DRG risk index0.090.06<.0001 In-hospital Mortality0.140.06<.0001 * Excluding patients discharged or transferred to another short term general hospital for inpatient care.

18. D EPARTMENT of F AMILY M EDICINE Results: Table 2: Baseline characteristics of MI patients transferred out of rural hospitals or staying in rural hospitals VariablesStay in rural hospitals (N=1,426) Transfer out* of rural hospitals (N=730) p-value Age82.3571.46<.0001 Male (%)45.0256.71<.0001 Black (%)0.14 0.99 Number of secondary diagnoses5.664.24<.0001 Charlson comorbidity index0.960.67<.0001 APR-DRG risk index0.090.04<.0001 * Patients discharged or transferred to another acute care hospital for inpatient care

19. D EPARTMENT of F AMILY M EDICINE Results: Table 3: Odds ratios of in-hospital mortality* among MI patients admitted to urban hospitals or to rural hospitals, using logistic regression models (n=12,191) Model componentsOdds ratio (Urban vs Rural) 95% CI p-valuec-statistic Unadjusted0.420.36-0.50<.00010.56 Adjusted for demographic variables (age, sex, race, admission type and source of payment) 0.700.59-0.84<.00010.71 Adjusted for demographic variables and Charlson comorbidity index 0.700.59-0.840.00010.71 Adjusted for demographic variables and APR-DRG risk index 0.680.56-0.82<.00010.86 * Excluding patients discharged or transferred to another short term general hospital for inpatient care

20. D EPARTMENT of F AMILY M EDICINE Results: Table 4: Goodness of fit of APR-DRG models in rural hospital sample and urban hospital sample Goodness-of-fit statistics Rural (N=1,426) Urban (N=10,779) c-statistic0.690.88 Hosmer-Lemishow chi-square test (8 df) 22.2210.03 P value for Hosmer-Lemishow chi-square test 0.00280.1994

21. D EPARTMENT of F AMILY M EDICINE Results: Table 5: Baseline characteristics among MI patients grouped by distance to the nearest urban hospital VariablesDistance to nearest urban hospital <=14.08 miles* (N= 6,097) Distance to nearest urban hospital >14.08 miles (N= 6,104) p-value Mean Distance to the nearest urban hospital (miles) 4.9434.20<0.0001 Percent of patients admitted to urban hospitals (%) 99.5477.07<0.0001 Age 68.8972.02<0.0001 Male (%) 58.6557.450.18 Black (%) 1.950.08<0.0001 Number of secondary diagnoses 5.725.53<0.0001 Charlson comorbidity index 0.72 0.67 APR-DRG risk index 0.07 0.48 In-hospital mortality rate (%) 7.077.520.34 *14.08 miles is the median distance from patient’s home to the nearest urban hospital

22. D EPARTMENT of F AMILY M EDICINE Results: Table 6: Instrumental variable estimates of the difference of in-patient mortality between urban and rural hospitals * If F-statistic is less than 10, the instrumental variables are weak. ** If p-value is less than 0.05, one of the instrumental variables is correlated with unmeasured confounders IV models (n=12,191) Number of groups for instrument al variable Tests for instrumental variables IV estimates of mortality difference Instrument P-value for overidentifying restrictions tests**CoefficientsP-value F-statistic* Unadjusted 21540.16--0.01990.34 4642.650.65-0.02690.16 12184.310.13-0.02880.13 Adjusted for demographic variables 21568.24-0.01270.58 4652.860.800.00810.69 12187.140.100.00650.75 Adjusted for demographic variables and Charlson comorbidity index 21539.9-0.00900.69 4642.510.920.00530.80 12184.290.120.00400.84 Adjusted for demographic variables and APR-DRG risk index 21694.27--0.00340.87 4640.610.92-0.00690.72 12202.500.01-0.00630.74

23. D EPARTMENT of F AMILY M EDICINE Results: Sensitivity analyses  Repeat analyses in different samples  Non-transferred MI patients  Three-year state inpatient datasets (2001 to 2003)  Different IV estimation method  Bivariate Probit model (using Stata 9.0)  The results are consistent with IV estimation in Table 6

24. D EPARTMENT of F AMILY M EDICINE Discussion  This study confirms earlier studies  MI patients admitted to rural hospitals were older and sicker than their urban counterparts  Traditional models all indicate significantly higher in-hospital mortality for those admitted to rural hospitals

25. D EPARTMENT of F AMILY M EDICINE Discussion  Logistic Regression model appears deficient  Significant difference in patient characteristics between two groups  Poor goodness-of-fit for rural-hospital logistic model

26. D EPARTMENT of F AMILY M EDICINE Discussion  Our findings suggest that the traditional logistic regression models are biased  Admissions to rural or urban hospitals are likely to be confounded by unmeasured patient variables  Referral patterns in rural hospitals  Younger and less sick patients are transferred to urban hospitals  The clinical judgment about transfer of rural senior patients with MI may rely on different criteria

27. D EPARTMENT of F AMILY M EDICINE Discussion  Patient preferences are likely to play a significant role in transfer decisions for older MI patients  May reflect personal choice or existing serious comorbidities  Serious cases may choose to remain close to home  The transfer patterns may reflect rural doctors respecting their patients’ wishes  Our data do not suggest that substandard quality of care is delivered at rural hospitals based on mortality.

28. D EPARTMENT of F AMILY M EDICINE Limitations of the study  The results of the IV estimation can only be generalized to patients for whom distance affects their choice  The conclusion cannot be applied to MI patients bypassing rural hospitals and seeking care in urban hospitals  The findings for hospitals in one state may not generalize to other states.  Analyses of in-hospital mortality rates may not generalize to mortality rates after hospitalization.

29. D EPARTMENT of F AMILY M EDICINE Conclusions  Mortality from MI in rural Iowa hospitals is not higher when controlled for unmeasured confounders.  Current risk-adjustment models may not be sufficient when assessing hospitals that perform different functions within the healthcare system.  Unmeasured confounding is a significant concern when comparing heterogeneous and undifferentiated populations.

30. D EPARTMENT of F AMILY M EDICINE Questions