1. Comparative Rankings of Hospital Quality – Does the Data Source Matter? Anne Elixhauser, Ph.D. Bernard Friedman, Ph.D. June 26, 2005 AcademyHealth Research Meeting

2. Background Hospitals are being compared based on readily available data Hospitals are being compared based on readily available data Data on Medicare patients are available from virtually all U.S. hospitals Data on Medicare patients are available from virtually all U.S. hospitals Convenient – but do we know how the Medicare experience reflects hospital quality overall? Convenient – but do we know how the Medicare experience reflects hospital quality overall?

3. Purpose of Study Examine the extent to which information on Medicare patients can be extrapolated to the general population when comparing hospital quality Examine the extent to which information on Medicare patients can be extrapolated to the general population when comparing hospital quality

4. Methods – Data Source 2001 Healthcare Cost and Utilization Project (HCUP) Nationwide Inpatient Sample (NIS) 2001 Healthcare Cost and Utilization Project (HCUP) Nationwide Inpatient Sample (NIS) Sample of 986 hospitals from 33 states Sample of 986 hospitals from 33 states – All discharges from each hospital are included No weighting for this study – Used NIS as a convenience sample of hospitals No weighting for this study – Used NIS as a convenience sample of hospitals Study population: hospital inpatients from short-term, non-Federal, acute care hospitals Study population: hospital inpatients from short-term, non-Federal, acute care hospitals – Limited to hospitals with Medicare patients

5. Methods – Quality Measures AHRQ Quality Indicators AHRQ Quality Indicators – Based on hospital administrative data – 15 Patient Safety Indicators (PSIs) Risk adjusted using gender, age, comorbidities, and collapsed DRGs Risk adjusted using gender, age, comorbidities, and collapsed DRGs – 12 in-hospital mortality measures from the Inpatient Quality Indicators (IQIs) Risk adjusted using APR-DRGs Risk adjusted using APR-DRGs

6. Methods – Rankings For each measure: – Dropped the 10% of hospitals with the fewest Medicare cases – Using Medicare discharges only Ranked hospitals and ordered into deciles Ranked hospitals and ordered into deciles – Using all patients (including Medicare) Ranked hospitals and ordered into deciles Ranked hospitals and ordered into deciles

7. Comparison of Rankings How many hospitals changed from the highest or lowest rank by at least two deciles? How many hospitals changed from the highest or lowest rank by at least two deciles? Compared rank using Medicare-only data to rank using all-patient data Compared rank using Medicare-only data to rank using all-patient data What % of poorest performing hospitals increased their ranking? What % of poorest performing hospitals increased their ranking? What % of best-performing hospitals fell in their ranking? What % of best-performing hospitals fell in their ranking?

8. Percent of hospitals in lowest decile that increased rank by at least 2 deciles

9. Percent of hospitals in highest decile that fell in rank by at least 2 deciles

10. Percent of hospitals in lowest decile that increased rank by at least 2 deciles

11. Percent of hospitals in highest decile that fell in rank by at least 2 deciles

12. Summary of Findings: Patient Safety Indicators Among the top 10% of hospitals (best, or lowest PSI rates): At least 40% of hospitals fell to 3 rd decile or lower for: Postop hip fracture Postop physiologic and metabolic derangement Postop respiratory failure Wound dehiscence 1/3 of hospitals fell to 3 rd decile or lower for: Anesthesia complications Death in low mortality DRGs Foreign body after procedure 1/4 of hospitals fell to 3 rd decile or lower for: Iatrogenic pneumothorax Infection due to medical care Postop hemorrhage Postop PE and DVT Postop sepsis

13. Summary of Findings: In-hospital Mortality Indicators Among top 10% of hospitals (best, or lowest mortality rates): 38% of hospitals fell to 3 rd decile or lower for: Craniotomy 12-20% of hospitals fell to 3 rd decile or lower for: GI hemorrhage Carotid endarterectomy Abdominal aortic aneurysm repair Among bottom 10% of hospitals (worst, or highest mortality rates): 24% of hospitals rose to 8 th decile or higher CABG 21% of hospitals rose to 8 th decile or higher Craniotomy 14% of hospitals rose to 8 th decile or higher PTCA 10% of hospitals rose to 8 th decile or higher Abdominal aortic aneurysm repair

14. Limitations Findings may be unique to these specific indicators Findings may be unique to these specific indicators Findings may not hold for cutpoints other than deciles Findings may not hold for cutpoints other than deciles Convenience sample of hospitals – not nationally representative Convenience sample of hospitals – not nationally representative – But hospitals are drawn from a sampling frame that comprises 80% of U.S. discharges

15. Conclusions When comparing Medicare and all-payer analyses When comparing Medicare and all-payer analyses – Found loose overlap of the top-ranking and poorest-ranking hospitals – Saw largest shifts for PSIs among the top- ranking hospitals Using Medicare-only data may carry a greater risk of incorrectly labeling a hospital as a top-ranking performer Using Medicare-only data may carry a greater risk of incorrectly labeling a hospital as a top-ranking performer Pre-test ranking approaches Pre-test ranking approaches