Trimming the Fat: Optimizing Overall Educational Value by Defining Factors Associated with Overall Educational Value and Service to Education Ratio Caroline E. Reinke, MD MSPH, Rachel R. Kelz, MD MSCE, Lori Pray, BA, Noel Williams, MD, Joshua Bleier, MD, Kenric Murayama, MD, Jon B. Morris, MD Division of Surgery Education Hospital of the University of Pennsylvania Philadelphia, PA

Nothing To Disclose Optimizing Overall Educational Value

Background Accreditation Council for Graduate Medical Education (ACGME) responsible for assessing and advancing the quality of resident physicians’ education –Founded in 1981 –Requires programs to achieve a balance between education and service With decreasing work hours, defining which characteristics are educational is critical to effective education Little is known about which modifiable rotation characteristics impact perceived service and education Optimizing Overall Educational Value ACGME Resident Survey: In your opinion, how often do your rotations and other major assignments provide an appropriate balance between your residency education and other clinical demands? How often has your clinical education been compromised by excessive service obligations?

Specific Aims AIM 1: To identify rotation characteristics associated with service to education ratio and overall educational value AIM 2: To understand the relationship between the perceived overall educational value and the service to education ratio Optimizing Overall Educational Value AIM 3: To understand how these relationships differ between junior (CY1-2) and senior residents (CY3-5)

Data Source Responses to an 11-item Web-based survey during FY2011 –Distributed to all residents at the end of each rotation Exclusion criteria: –Non-clinical rotations (simulation month) –Subspeciality-specific rotations (ie. plastic surgery) Survey responses were linked to additional rotation data: –Length of the rotation in number of weeks –Number of attending surgeons on each service –Presence or absence of fellows Optimizing Overall Educational Value

Rotation Characteristics12345 Volume of Operative Experience PoorFairAdequateVery GoodExceptional Complexity of Operative Experience Too SimpleBasicAppropriateChallenging Beyond my Capacity Autonomy of Operative Experience MinimalOccasional Commesurate with ability FrequentExcessive Patient Care Participation MinimalOccasionalAdequateFrequentExcessive Autonomy in Patient Care MinimalOccasional Commensurate with ability FrequentExcessive Time for Reading/Cognitive Development Minimal Would prefer more Sufficient Would prefer less Excessive Educational Value of Fellows and Speciality Residents DetrimentalInefficientIrrelevantMarginalExceptional Ratio of Educational to Service Activity 0% Educ, 100% Service 20% Educ, 80% Service 50% Educ, 50% Service 80% Educ, 20% Service 100% Educ, 0% Service Overall Educational Value NoneFairAdequateVery GoodOutstanding Duration of Rotation 2 months too long 1 month too short Just right 1 month too long 2 months too long Teaching Faculty are supportive of ACGME Duty Hour Rules NeverRarelyOccasionallyFrequentlyAlways Rotation Survey Optimizing Overall Educational Value

Study Population 57 general surgery training program rotations were evaluated by 85 residents Excluded rotations in which less than three residents in a given CY evaluated the rotation (n=9) Final study population = 48 general surgery rotations with 578 resident evaluations (range 3-36) Overall survey response rate = 87% Optimizing Overall Educational Value

Statistical Analyses Linear regression used to assess the relationship between rotation characteristics (RC), rotation data (RD), and –Overall Educational Value (OEV) –Service to Education Ratio (SER) Linear regression used to assess the relationship between OEV and SER Multivariate linear regression was used to examine the relationship between SER and OEV after controlling for significant RC and RD Subgroup analyses performed for junior (PGY1-2) and senior (PGY3-5) residents Optimizing Overall Educational Value

Relationship between OEV and SER Optimizing Overall Educational Value

Relationship between OEV and SER Optimizing Overall Educational Value

Rotation Characteristics (RC) of interestβ Coefficientp-value Overall educational value (OEV)-- Additional Rotation Data (RD) Rotation Length Number of attending surgeons Fellow Other Rotation Characteristics (RC) Volume of operative experience0.37<0.001 Complexity of operative experience0.69<0.001 Autonomy of operative experience0.58<0.001 Patient care participation Autonomy in patient care Time for reading and cognitive development0.85<0.001 Educational value of fellows on the rotation Duration of the rotation-1.63<0.001 Teaching faculty are supportive of ACGME Duty Hour Rules1.02<0.001 Optimizing Overall Educational Value RC, RD and the relationship with OEV

Rotation Characteristics (RC) of interestβ Coefficientp-value Overall educational value (OEV)-- Additional Rotation Data (RD) Rotation Length Number of attending surgeons Fellow Other Rotation Characteristics (RC) Volume of operative experience0.37<0.001 Complexity of operative experience0.69<0.001 Autonomy of operative experience0.58<0.001 Patient care participation Autonomy in patient care Time for reading and cognitive development0.85<0.001 Educational value of fellows on the rotation Duration of the rotation-1.63<0.001 Teaching faculty are supportive of ACGME Duty Hour Rules1.02<0.001 Optimizing Overall Educational Value RC, RD and the relationship with OEV

Rotation Characteristics (RC) of interestβ Coefficientp-value Overall educational value (OEV)-- Additional Rotation Data (RD) Rotation Length Number of attending surgeons Fellow Other Rotation Characteristics (RC) Volume of operative experience0.37<0.001 Complexity of operative experience0.69<0.001 Autonomy of operative experience0.58<0.001 Patient care participation Autonomy in patient care Time for reading and cognitive development0.85<0.001 Educational value of fellows on the rotation Duration of the rotation-1.63<0.001 Teaching faculty are supportive of ACGME Duty Hour Rules1.02<0.001 Optimizing Overall Educational Value RC, RD and the relationship with OEV

Rotation Characteristics (RC) of interestβ Coefficientp-value Overall educational value (OEV)-- Additional Rotation Data (RD) Rotation Length Number of attending surgeons Fellow Other Rotation Characteristics (RC) Volume of operative experience0.37<0.001 Complexity of operative experience0.69<0.001 Autonomy of operative experience0.58<0.001 Patient care participation Autonomy in patient care Time for reading and cognitive development0.85<0.001 Educational value of fellows on the rotation Duration of the rotation-1.63<0.001 Teaching faculty are supportive of ACGME Duty Hour Rules1.02<0.001 Optimizing Overall Educational Value RC, RD and the relationship with OEV

Rotation Characteristics (RC) of interestβ Coefficientp-value Service to Education Ratio (SER)-- Additional Rotation Data (RD) Rotation Length Number of attending surgeons* Fellow Other Rotation Characteristics (RC) Volume of operative experience <0.001 Complexity of operative experience <0.001 Autonomy of operative experience <0.001 Patient care participation* Autonomy in patient care* Time for reading and cognitive development <0.001 Educational value of fellows on the rotation* Duration of the rotation Teaching faculty are supportive of ACGME Duty Hour Rules <0.001 Optimizing Overall Educational Value RC, RD and the relationship with SER

Rotation Characteristics (RC) of interestβ Coefficientp-value Service to Education Ratio (SER)-- Additional Rotation Data (RD) Rotation Length Number of attending surgeons* Fellow Other Rotation Characteristics (RC) Volume of operative experience <0.001 Complexity of operative experience <0.001 Autonomy of operative experience <0.001 Patient care participation* Autonomy in patient care* Time for reading and cognitive development <0.001 Educational value of fellows on the rotation* Duration of the rotation Teaching faculty are supportive of ACGME Duty Hour Rules <0.001 Optimizing Overall Educational Value RC, RD and the relationship with SER

Rotation Characteristics (RC) of interestβ Coefficientp-value Service to Education Ratio (SER)-- Additional Rotation Data (RD) Rotation Length Number of attending surgeons* Fellow Other Rotation Characteristics (RC) Volume of operative experience <0.001 Complexity of operative experience <0.001 Autonomy of operative experience <0.001 Patient care participation* Autonomy in patient care* Time for reading and cognitive development <0.001 Educational value of fellows on the rotation* Duration of the rotation Teaching faculty are supportive of ACGME Duty Hour Rules <0.001 Optimizing Overall Educational Value RC, RD and the relationship with SER

Relationship between OEV and SER Optimizing Overall Educational Value

Relationship between OEV and SER Optimizing Overall Educational Value

The relationship between SER and OEV Rotation Characteristics (RC) of interest Overall Educational Value β Coefficientp-value Adj. R- squared Overall educational value (OEV)-- Service to education ratio (SER)-1.23<

Optimizing Overall Educational Value The relationship between SER and OEV Rotation Characteristics (RC) of interest Overall Educational Value β Coefficientp-value Adj. R- squared Overall educational value (OEV)-- Service to education ratio (SER)-1.23< After adjustment for other RC & RD Service to education ratio (SER)-1.08<

Optimizing Overall Educational Value Subgroup Analysis of Junior and Senior Rotations Junior Rotations (n=28) Overall Educational Value β Coefficientp-value Adjusted R-squared Overall educational value (OEV)-- Service to education ratio (SER), unadjusted-1.27< Service to education ratio (SER), adjusted-1.27< Senior Rotations (n=20) Overall educational value (OEV)-- Service to education ratio (SER), unadjusted-1.28< Service to education ratio (SER), adjusted

Summary We identified an inverse relationship between the service to education ratio and overall educational value, which was not fully explained by other rotation characteristics This relationship differed significantly for junior and senior rotations Individual rotation characteristics varied in their relationship with overall educational value –The value of fellows varied, but their simple presence or absence was not predictive of OEV –Number of attending surgeons was negatively correlated with overall educational value Optimizing Overall Educational Value

Thank You Optimizing Overall Educational Value

Rotation Characteristics (RC) of interestβ Coefficientp-value Overall educational value (OEV)-- Additional Rotation Data (RD) Rotation Length Number of attending surgeons Fellow Other Rotation Characteristics (RC) Volume of operative experience Complexity of operative experience Autonomy of operative experience Patient care participation Autonomy in patient care Time for reading and cognitive development0.86<0.001 Educational value of fellows on the rotation Duration of the rotation Teaching faculty are supportive of ACGME Duty Hour Rules Optimizing Overall Educational Value RC, RD and the relationship with OEV – Junior

Rotation Characteristics (RC) of interestβ Coefficientp-value Overall educational value (OEV)-- Additional Rotation Data (RD) Rotation Length Number of attending surgeons Fellow Other Rotation Characteristics (RC) Volume of operative experience0.60<0.001 Complexity of operative experience1.44<0.001 Autonomy of operative experience1.18<0.001 Patient care participation Autonomy in patient care Time for reading and cognitive development Educational value of fellows on the rotation Duration of the rotation Teaching faculty are supportive of ACGME Duty Hour Rules Optimizing Overall Educational Value RC, RD and the relationship with OEV – Senior

Background – ACGME Resident Survey In your opinion, how often do your rotations and other major assignments provide an appropriate balance between your residency education and other clinical demands? How often has your clinical education been compromised by excessive service obligations? Optimizing Overall Educational Value