1. Charles Holden, MD- Associate Program Director Thomas Hartranft, MD – Program Director Darrell Spurlock, Jr. PhD – Director of Research Mount Carmel Health System General Surgery Residency Program cholden@phy.mchs.com

2.  Ranking and selecting candidates for general surgery residency programs is an important but complex process  Various factors are considered in ranking and selecting candidates but the relative weights assigned to these factors are sometimes anecdotal  In an effort to examine the predictive nature of USMLE Step 1 and Step 2 scores on later ABSITE performance, a small study was conducted

3.  Several factors are important to consider when ranking and selecting residents for general surgery residency programs  Previous academic & professional performance  Organizational fit  Resident expectations of the program  Program expectations of residents  Congruence between career goals & training opportunities

4.  Academic achievement is measured in many ways, but perhaps the most objective is by the use of standardized exams, like the USMLE  Step 1 and Step 2 of the USMLE have been examined in past research for their relationship with in-training exam scores, but not in general surgery residency programs

5.  Results from research in other specialties (mainly orthopaedics, obstetrics & gynecology, and internal medicine) has shown mixed results  Step 1 is a good predictor in some cases, but Step 2 seems to be a better predictor – when studied (which it often is not)  We examined Step 1 and Step 2 scores in relation to ABSITE scores during the PGY1-5 years

6.  Design: Retrospective, descriptive, correlational study of graduated resident records  Participants: N = 34 residency completers from 1999-2009 from two Midwestern community teaching hospital general surgery residency programs  Correlation and regression analyses used to evaluate relationships between USMLE Step 1, Step 2, and ABSITE PGY1-5 year scores

8. Intercorrelations among Exam Scores (N = 34) Variable123456 1. USMLE Step 1- 2. USMLE Step 2.576**- 3. ABSITE Year 1.264.446*- 4. ABSITE Year 2.343.444*.531**- 5. ABSITE Year 3.457**.604**.355*.460**- 6. ABSITE Year 4.342.455*.319.412*.549**- 7. ABSITE Year 5.347.550**.311.352*.555**.662* * significant at P <.05 level ** significant at the P <.01 level

9.  Hierarchical regression analysis was run to examine the incremental predictive value of Step 1, beyond Step 2; no incremental improvement was noted  Simple linear regression showed that USMLE Step 2 scores were predictive of all years of ABSITE performance, but most significantly for PGY3 (R 2 =.365, F = 11.491, P =.003) and 5 (R 2 =.303, F = 8.674, P =.008)

10. USMLE Step 2ABSITE PGY3/5 Approximate 30-36% Shared Variance Between USMLE Step 2 and ABSITE PGY3/5 Scores

11.  USMLE Step 2 scores were better predictors of later ABSITE performance than were Step 1 scores  Because of the limited range of scores for Step 2 and the small sample size, establishing a “cut-score” was not possible when considering only residents who were accepted into the residency program

12.  Standardized test scores can be an important part of the selection criteria for choosing residents, but they must not be weighed too heavily as they account for only ~30-35% of the later ABSITE performance  Factors other than exam performance are important in the making of a good surgeon, and weren’t evaluated here, e.g., professionalism, ethics, etc.

13.  Future research is needed with larger samples to examine the relationship of other selection criteria on later in-training performance  With the addition of the USMLE Step 2 “Clinical Skills” component, further evaluation of the usefulness of Step 2 is warranted as well  Questions?