1. Organizational characteristics and quality of care: Inside the black box Elizabeth H. Bradley, PhD Yale School of Public Health Health Policy and Administration

2. Background Substantial variation across hospitals in quality of care - Processes - Risk-adjusted outcomes Application of AMI care - Beta-blocker rates - Time to reperfusion therapy - Risk-adjusted mortality What contributes to this variation?

3. Previous work has shown: Variation by region Census regions (Krumholz, AHJ, 2003; Normand JAMA, 1996; Subramanian, J Gen Intern Med, 2002) Hospital referral regions (small areas) (Fisher, Ann Intern Med, 2003; Fisher, HSR 2000) Variation by types of hospitals Teaching (Allison, JAMA 2003; Rosenthal, JAMA 1997) Volume (Theimann, JAMA, 1999; Magid, JAMA 2000)

4. However… These traditional hospital attributes explain only 6%-12% of the hospital-level variation in processes and outcomes (Normand, JAMA, 1999; Schultz, Applied Nurs Rsch, 1999)

5. What distinguishes the top performing hospitals? Market factors (competition, managed care penetration, demand for services) Regulatory factors (Certificate of Need, public performance reporting) Organizational factors (Inside the black box)

6. We have examined top performing hospitals in time to reperfusion therapy Few hospitals are meeting guidelines, and hospitals’ average performance has not changed dramatically over time.

7. However, some hospitals have improved

8. How do top performers do it? Mixed methods studies - Qualitative in-depth interviews - Quantitative hospital survey of efforts - Chart review data on clinical practices

9. Approaches to measurement Start with open-ended, qualitative assessment to understand scope and create a taxonomy, or language for possible predictor variables Move to closed-ended, quantitative measures in order to increase sample sizes, enhance ability to make statistical inferences, and improve generalizability

10. Success Factors Systems design (structures & processes) --------------------------- Organizational goal Administrative support Clinical leadership Teams Organizational culture Sources: Bradley et al., JAMA 2001; Bradley et al., Med Care 2005

11. Measuring systems design Systems design has manifest attributes - Structures - Processes The structures are easiest and most objective internal features to measure The processes are more complex because they are fluid, vary by the people involved, and include multiple components

12. Measuring structures & processes Start with set of key components hypothesized from qualitative work Survey (self-report or external observations) for quantitative assessment of key components identified in qualitative work

13. Key components of systems design

14. Moving from qualitative to quantitative measurement Original data collection (self-reported or observations by others) Challenges - Instrument design, length, and validation - Pinpointing the time frame - Measurement can itself change processes - Issue of multiple respondents

15. Measuring organizational environment The organizational environment is most difficult because it is more hidden (i.e., culture), and it is a composite of diverse experiences and perceptions

16. Key components of organizational environment Clear, explicit goals in clinical quality Administrative support Clinician “champions” Collaborative interdepartmental teams Organization culture

17. Moving to quantitative assessment Measures can be noisy, making it harder to see their statistical effects (e.g., org culture) Multiple measures are good (but expensive) Multiple respondents per organization are good (but also expensive) Some black box features defy quantitative measurement; good to retain mixed methods

18. Take Home Messages The black box is complex; we like to keep it closed But recent efforts suggest that org context matters Research can give insight into the org structures, processes, and environments that foster higher quality, but we must use a mix of methods and disciplinary approaches to producing evidence