Measuring the Impact of Resident Work Hours Reform: Recent Findings and Next Steps Patrick S. Romano, MD MPH Professor of Medicine and Pediatrics Division of General Medicine and Center for Healthcare Policy and Research University of California, Davis May 21, 2009

Disclosure Financial support from NHLBI RO1 HL82637 (Kevin Volpp, University of Pennsylvania, PI) Have you (or your spouse/partner) had a personal financial relationship in the last 12 months with the manufacturer of the products or services that will be discussed in this CME activity? NOT APPLICABLE

Acknowledgments Investigative team Kevin Volpp, MD PhD, University of Pennsylvania Jeffrey H. Silber, MD PhD, Director, Center for Outcomes Research, Children’s Hospital of Philadelphia Amy K. Rosen, PhD, Bedford VA Center of Excellence in HSR&D Paul Rosenbaum, PhD (Statistician), Wharton School Lisa Bellini, MD, Program Director, University of Pennsylvania Staff Orit Even-Shoshan, MS, Anne Canamucio, MS, Tiffany Behringer, MS, Yanli Wang, Hong Zhou, Liyi Cen, Mike Halenar Other sources of slides John Welch, MD (former pediatric resident) Garth Utter, MD, Department of Surgery

Learning Objectives To Explain Recent and Proposed Policy Changes to Limit Resident Work Hours To Summarize Recent Evidence Regarding the Impact of ACGME Work Hour Rules Implemented in 2003 To Discuss Ideas and Methods for Future Research in this Area

The Birth of Residency 1889 with the opening of The Johns Hopkins Hospital Osler, Halsted, and Kelly (bedside teaching) Based on German model Room, board, and laundry provided; salary optional until 1965

Core Concepts and Practices Graded responsibility, especially for inpatients Variable and lengthy training period Pyramidal system of promotion Restrictive lifestyle ( hrs/week, continuous shifts up to 36 hours)

Libby Zion Case 18 year old college student, with known history of depression, taking Nardil® (MAOI), was brought into New York Hospital on October 4, 1984 Presented with fever, agitation and strange jerking motions of body, with occasional disorientation Admitted with diagnosis of “viral syndrome with hysterical symptoms”

Libby Zion Case Ordered Demerol® to control her shaking Later in evening Libby became more agitated The intern was contacted at least twice, ordered physical restraints and Haldol® Patient finally fell asleep At 6:30 a.m. her temperature was noted to be 107°F Emergency measures were attempted Patient suffered a cardiac arrest and died

Aftermath Sidney Zion’s efforts to change the system 1986 Grand Jury does not indict those involved but strongly criticizes system Bell Commission established to investigate and make recommendations July 1, 1989 Section of Title 10 of the New York Codes, Rules and Regulations of the Department of Health go into effect Work hours: Maximum 80 hrs/week averaged over 4 weeks, 24 consecutive hours (12 in ED), at least 1 scheduled 24 hour break per week On-site supervision 24 hrs/day, 7 days/week

The 80 hour work week “The specific "80-hour week" was actually determined by a colleague on my porch and was based on the following informal reasoning: (1) there are 168 hours in a week; (2) it is reasonable for residents to work a 10-hour day for 5 days a week; (3) it is humane for people to work every fourth night; (4) subtracting the 50-hour week (10 hours per day x 5 days) from 168 hours leaves 118 hours; (5) divide 118 by 4 (every fourth night) and add to the 50 hours and, eureka, that equals an 80-hour week.” Bell JAMA 2007: 298(24):

Timeline June New York State Department of Health found work hours often exceeded regulatory limits April OSHA is petitioned to regulate work hours nationwide November 2001 Representative John Conyers (D-Mich) introduced federal legislation to restrict resident work hours In response ACGME announced its guidelines effective July 1, 2003

ACGME Work Hour Restrictions Principles, Supervision, Fatigue, Duty Hours, On- Call activities, and Moonlighting Limited to 80 hours per week One day in seven free of all responsibilities 10 hour off-duty period between work periods In house call no more frequently than every 3 rd night In house call must not exceed 24 consecutive hours (with up to 6 extra hours for transition of care)

Resident Education and QOL Meta-analysis of studies that assessed a system change designed to counteract the effects of resident work hours, fatigue, or sleep deprivation; included an outcome directly related to residents; and were conducted in the United States. 54 articles met inclusion criteria (12 IM, 6 ob/gyn, 7 pediatrics, 25 surgery, 4 other) Interventions included night and day float teams, extra cross-coverage, and physician extenders. Fletcher, JAMA, 2005

Resident Education and QOL Interventions to reduce resident work hours resulted in mixed effects on both operative experience and on perceived educational quality but generally improved residents’ quality of life (i.e., more sleep, better mood, better family relationships, better satisfaction). Interpretation of the outcomes of these studies is hampered by suboptimal study design and the use of nonvalidated instruments. The long-term impact of reducing resident work hours on education remains unknown. Fletcher, JAMA, 2005

N Engl J Med 2004;351:

Effect of Reducing Interns’ Work Hours Investigated effect of reducing interns’ work hours on serious medical errors in ICU Randomized, prospective crossover trial Conducted in MICU and CCU, 20 interns on 3 week rotations Q3 day “traditional” call versus 4-day schedule without extended shifts >16 hours (7am-3pm on day 1, 7am-10pm "day call" on day 2, 9pm-1pm “night call" on days 3-4) Compared rates of serious medical errors (by masked direct observation) made by interns on traditional vs. intervention schedule

Effect of Reducing Interns’ Work Hours Traditional work week: hours (mean 85) Intervention work week: hours (mean 65) No change in staffing or other personnel Randomly assigned order, and spread throughout year Interns worked 19.5 hrs/week less (P<0.001), slept 5.8 hrs/week more (P<0.001), and had fewer attentional failures (EOG slow eye movements) during on-call nights (0.33/hr= 2.6/overnight versus 0.69/hr=5.5/overnight; P=0.02) on the intervention schedule

Relationship between work hours and sleep duration on two schedules

Most but not all interns slept more on intervention schedule

Results of RCT 2203 patient days, 5888 hours of observation During traditional schedule, interns made: 35.9% more serious medical errors 27.8% more intercepted serious medical errors 56.6% more nonintercepted serious medical errors 20.8% more serious medication errors 5.6 times as many serious diagnostic errors No difference in procedural errors No significant change in other staff errors No significant change in # of medications or procedures, tests interpreted, LOS, mortality

Landrigan C et al. N Engl J Med 2004;351: Incidence of Serious Medical Errors

Evaluations of 1988 Libby Zion Laws No relative improvements in mortality for AMI, CHF, pneumonia in teaching vs. non-teaching hospitals 1 Significant increases in proportion of patients with at least one complication (35% vs. 22%, p=.002) and in delays in diagnostic tests (17% vs. 1.9%, P<.0001) 2 Compliance poor (60% of surgical residents exceeded 95 hours per week – 1997 audit) 1 Howard, Silber, Jobes JGIM Laine JAMA 1993

Effect of work hours reform in NY teaching hospitals on smoothed rates of Patient Safety Indicators Poulose BK, et al., Ann Surg 2005;241:

Benefits versus Harms Benefits: More sleep, better sleep, better quality of life Better cognitive performance, fewer errors Caveat #1: while duty hour rules reduce total number of hours work per week, 30 hour shifts allowed Caveat #2: we don’t know how much more residents are actually sleeping Harms: Less opportunity to observe trajectory of illness More frequent hand-offs Studies have shown higher rates of significant adverse events when patients are “cross covered” (26% vs. 12%, OR=3.5) 1 1 Petersen et al, Annals of Int Med :

First study of impact on mortality: Inpatient only, different samples by year Shetty, Ann Intern Med, % absolute reduction; 3.75% relative reduction

Our Study Cohorts Volpp K, et al. JAMA 2007;298(9): All unique patients admitted between July 1, 2000 and June 30, 2005 (3 yrs pre-reform, 2 yrs post-reform) Principal diagnoses: AMI, CHF, GI bleed, or stroke DRG classification of general, orthopedic, or vascular surgery VA 320,685 patients, 131 hospitals Data from VA Patient Treatment File (PTF) and Beneficiary Identification Record Locator System (BIRLS), VA Office of Academic Affiliations Medicare 8,529,595 patients, 3321 hospitals Data from the Medicare Provider Analysis and Treatment File (MEDPAR), denominator files, Medicare Cost Reports

Effects measured by comparing pre- to post- reform changes in mortality in hospitals of differing teaching intensity ACGME Reform Logistic regression used to adjust for patient comorbidities, secular trends, hospital site where treated using “difference in differences” EFFECT = Diff A – Diff B (D-in-D) Diff B Diff A1. Diff A2

Medicare - No significant relative change in mortality according to teaching intensity Volpp KG et al. JAMA; 2007: 298 (9):

VA - Significant relative improvement in mortality among medical patients in post-reform year 2 Volpp KG et al. JAMA; 2007: 298 (9):

How big were these effects? Medical patients: Improvement in mortality from pre-1 to post-2 of 0.70 percentage points (11.1%) for hospitals in 75th compared to 25th percentile Volpp KG et al. JAMA; 2007: 298 (9):

VA hospitals much more teaching intensive VA Hospitals Medicare Volpp KG et al. JAMA; 2007: 298 (9): ; Volpp KG et al. JAMA 2007; 298(9):

Do effects of reform on mortality vary across hospitals (Medicare)?

Why no improvement in quality among Medicare patients? Design flaws 30 hour shifts allow acute sleep deprivation Current design does not respect circadian rhythms Sleep inertia at night when paged Implementation Compliance likely incomplete; may be worse than in VA hospitals, given higher work intensity Offsetting factors Worsened continuity Higher work intensity Sicker patients

Why improvement in some groups but not others? VA vs. Medicare VAs more teaching intensive (“dose response”) Better information systems may have mitigated some of the continuity of care (hand-off) problems Confounding due to other changes Medical vs. surgical Differences in balance between reduction in fatigue and continuity? Differences in compliance? Differences in effort to address discontinuity through structured sign-out, increased attending involvement?

Failure to Rescue: Death among surgical patients with potentially treatable complications

Resident/bed ratio  post-reform year 1 * top 10%/25% of severity Resident/bed ratio  post-reform year 2 * top 10%/25% of severity Odds ratio† (95% CI)P-valueOdds ratio† (95% CI)P-value Medicare Combined medical Highest 10% (vs. bottom 90%) 1.01 (0.90, 1.13) (0.80, 1.02)0.09 Highest 25% (vs. bottom 75%) 0.99 (0.98, 1.08) (0.85, 1.03)0.17 Combined surgical Highest 10% (vs. bottom 90%) 0.91 (0.80, 1.04) (0.88, 1.15)0.88 Highest 25% (vs. bottom 75%) 0.98 (0.86, 1.12) (0.95, 1.24)0.21 Failure to rescue Highest 10% (vs. bottom 90%) 0.94 (0.80, 1.09) (0.86, 1.18)0.92 Highest 25% (vs. bottom 75%) 0.90 (0.79, 1.02) (0.88, 1.14)0.98 Highest risk patients fared no differently than lower risk patients - Medicare Volpp KG et al. JGIM In Press.

Resident/bed ratio  post-reform year 1 * top 10%/25% of severity Resident/bed ratio  post-reform year 2 * top 10%/25% of severity Odds ratio† (95% CI)P-valueOdds ratio† (95% CI)P-value Medicare Combined medical Highest 10% (vs. bottom 90%) 1.63 (1.08, 2.46) (0.88, 2.07)0.17 Highest 25% (vs. bottom 75%) 1.44 (1.01, 2.05) (0.67, 1.43)0.93 Combined surgical Highest 10% (vs. bottom 90%) 0.68 (0.39, 1.20) (0.45, 1.43)0.45 Highest 25% (vs. bottom 75%) 0.52 (0.29, 0.96) (0.59, 2.17)0.71 Failure to rescue Highest 10% (vs. bottom 90%) 0.67 (0.35, 1.30) (0.33, 1.24)0.19 Highest 25% (vs. bottom 75%) 0.86 ( ) (0.46, 1.48)0.51 Highest risk patients fared no differently than lower risk patients - VA Volpp KG et al. JGIM In Press.

The concept of prolonged stays

Non-teaching (0)Very Minor/Minor (>0 & <.25) Major (>0.25& <0.6)Very Major (>0.6) The rate of prolonged stays varies little over time Silber et al Medical Care

Odds of prolonged stay change at similar rates in more vs. less teaching intensive hospitals Patient categories (Number of Cases Medicare/VA) RB ratio  post-reform year 1 OR (95%CI) RB ratio  post-reform year 2 OR (95% CI) Medical ConditionsMedicareVAMedicareVA Stroke (933,225/25,385)1.01 (0.92, 1.10)0.92 (0.66, 1.27)1.01 (0.92, 1.10)0.95 (0.69, 1.31) AMI (970,184/32,170)1.01 (0.93, 1.10)0.96 (0.72, 1.29)1.06 (0.97, 1.15)0.96 (0.72, 1.28) GI bleed (763,765/36,035)1.06 (0.97, 1.16)1.26 (1.00, 1.58) a 1.09 (1.00, 1.20)1.08 (0.86, 1.36) CHF (1,196,294/50,266)0.99 (0.92, 1.06)1.11 (0.92, 1.35)1.02 (0.95, 1.10)1.18 (0.97, 1.43) Combined Medical (3,863,468/143,856)1.01 (0.97, 1.05)1.07 (0.94, 1.20)1.04 (0.99, 1.08)1.05 (0.93, 1.19) Surgical Conditions General Surgery (651,515/22,482) 1.09 (0.99, 1.21)1.07 (0.79, 1.43)0.94 (0.85, 1.05)1.02 (0.76, 1.36) Orthopedic Surgery (1,364,559/32,719) 1.03 (0.96, 1.10)0.82 (0.61, 1.12)0.94 (0.88, 1.01)1.04 (0.77, 1.41) Vascular Surgery (179,473/11,219) 1.16 (1.00, 1.34)1.08 (0.66, 1.77)1.21 (1.04, 1.40) a 1.16 (0.71, 1.91) Combined Surgical (2,195,547/66,420)1.04 (0.98, 1.09)0.94 (0.78, 1.14)0.96 (0.91, 1.01)1.00 (0.83, 1.21) a p<0.05 b p<0.01 c p<0.001 Silber et al. Medical Care 2009.

AHRQ Patient Safety Indicators Technical composite iatrogenic pneumothorax foreign body left in during procedure postoperative wound dehiscence accidental puncture or laceration postoperative hemorrhage or hematoma Continuity of Care composite postoperative physiologic or metabolic derangement postoperative pulmonary embolism or deep vein thrombosis (PE/DVT) postoperative sepsis Collaborative Care composite postoperative hip fracture postoperative respiratory failure selected infections due to medical care

PSI composite rates change at similar rates in hospitals of different teaching intensity Rosen et al. Medical Care In Press.

Technical Care Composite Continuity of Care Composite Collaborative Care Composite VA Odds Ratio (95% CI) Medicare Odds Ratio (95% CI) VA Odds Ratio (95% CI) Medicare Odds Ratio (95% CI) VA Odds Ratio (95% CI) Medicare Odds Ratio (95% CI) Resident/bed ratio*post1 a 1.09 ( ) P= (1.04 – 1.27) P= ( ) P= ( ) P = ( ) P= ( ) P= 0.80 Resident/bed ratio*post2 a 1.05 ( ) P= (0.99 – 1.21) P= ( ) P= ( ) P = ( ) P= ( ) P= 0.97 Number of cases 795,30612,426,475339,5047,669,946653,27011,295,527 Odds of experiencing a PSI generally changed at similar rates in more vs. less teaching intensive hospitals Rosen et al. Medical Care In Press.

Results Summary Good news? No evidence of worsening of outcomes for a broad range of measures within either Medicare or VA No evidence of harm (or benefit) for high-risk patients Question about prolonged length of stay for vascular surgery patients Bad news? No evidence of significant relative improvements in outcomes except for medical patients in VA in post-reform year 2

Institute of Medicine 2008 report The Institute of Medicine formed a consensus committee to: 1) synthesize current evidence on medical resident schedules and healthcare safety. 2) develop strategies to enable optimization of work schedules to improve safety in the healthcare work environment. The strategies recommended will take into account the learning and experience that residents must achieve during their training. The recommendations will be structured to optimize both the quality of care and the educational objectives.

What do we not know? Hours per week residents are actually working How much more sleep residents are actually getting Impact on broader range of clinical outcomes Longer-term impact on clinical outcomes Impact on educational outcomes How residents are spending their time What approaches have helped programs successfully adapt Comparative effectiveness and cost effectiveness of different approaches Role of hospital finances IOM Report on Resident Work Hours

IOM Recommendations “Safe transportation options” Minimize work “that is of limited or no educational value, is extraneous to their program’s goals and objectives, and can be done well by others” “Adequate time to conduct thorough evaluations of patients and for reflective learning…” Specialty-specific, RRC-set limits on caseload Supervisory physician (resident) in house at all times Schedule overlap time and facilitate safe handoffs

IOM Recommendations 80 hour week 5 hour mandatory nap between 10pm-8am if overnight shifts used Averaging of days off not allowed; 1 day per week and 5 days per month All moonlighting counted against limits 10 hours off after day shift; 12 hours off after night shift; 14 hours off after extended shift

Design of R01 HL094593

New study focuses on educational and clinical impact (NHLBI) To describe the variety and frequency of program-level behavioral responses to duty hour reform and resident work conditions qualitative field work at a sample of IM and GS residency programs Mixed-method approach of direct observation and interviews 12 hospitals placed in a 2x2 matrix of large versus small size and good versus poor financial balance sheets in FY 2008 by assessing operating margins and fund balances over the previous 5 years Direct observation of resident involvement in provision of hospital care (especially rounds, hand-offs) Semi-structured 1 on 1 interviews with open ended questions involving residents, nurses, attending physicians, and administrators

New study focuses on educational and clinical impact (NHLBI) To describe the variety and frequency of program-level behavioral responses to duty hour reform and resident work conditions national surveys of program directors and residents in Internal Medicine (IM) and General Surgery (GS) Partner with the ABIM, ABS, APDIM, APDS, and ACP Surveys of residents to focus on resident-specific issues such as balance between service delivery and education, assessment of work intensity, use of free time, how handoffs are done, actual hours worked and days off, and hours slept. Surveys of program directors to focus on use of non-teaching services, hiring hospitalists or physician extenders, work intensity (admissions per resident, # patients covered, hours worked, days off), training (if any) in how to do ‘handoffs’, helpful or problematic attributes of work environment.

New study focuses on educational and clinical impact (NHLBI) To assess how educational outcomes (board scores) have changed with duty hour reform for residents in different specialties. To examine how clinical outcomes (mortality, FTR, PLOS, PSIs) have changed beyond the first two years post-duty hour reform. To examine how pre-reform hospital financial health and staffing levels predicted changes in staffing and educational and clinical outcomes.

Variables

Comparison across specialties

Questions and discussion