Effects of a Student-Initiated Test Preparation Curriculum Redesign on USMLE Step 1 Examination Scores Lindsay Schwartz, BSc; Abbas Hyderi, MD MPH; Matthew Lineberry, PhD; and Yoon Soo Park, PhD University of Illinois College of Medicine at Chicago April 28 th, 2015

Disclosures The presenters of this research have nothing to disclose

Learning Objectives 1.Determine the successes and limitations of this student-initiated redesign of USMLE Step 1 preparatory curriculum 2.Apply aspects of this program model to other institutions’ USMLE Step 1 preparatory curricula

USMLE Step 1 Exam Fast Facts Taken at end of second year of medical school 7 sections, 46 questions each  ~8 hours Tests basic sciences Each exam dynamically generated for each test taker Three-digit score calculated using statistical percentiles –Ensures scores from different years are on a common scale 2013: National Mean = 228 (SD = 21)

University of Illinois College of Medicine Fast Facts 1882: The College of Physicians and Surgeons of Chicago opens 1913: Officially becomes the College of Medicine of the University of Illinois 1970: Expanded to 3 additional sites (Peoria, Rockford and Urbana). Existing campus becomes “The University of Illinois College of Medicine at Chicago”. 2015: –2 nd largest medical school in the country (~1,350 students) –#25 nationally in NIH funding –Trains 1 in 6 Illinois physicians –#1 nationally in Hispanic graduates and #5 African American graduates –Formal clinical experiences from first year onward

Rationale for Curricular Redesign 2012 AAMC NRMP Program Director Survey –82% stated performance on the USMLE Step 1 Exam was an important determinant for interview selection and applicant ranking order –Therefore, the USMLE Step 1 Exam is an important determinant of future career prospects Historically, University of Illinois College of Medicine at Chicago (UIC) mean lower than national mean

UIC’s Performance on the USMLE Step 1 exam

Program Description The “10 for ‘15 Initiative” set out to redesign the UIC USMLE Step 1 Curriculum Provided all second-year students with: –First Aid for the USMLE Step –6 month subscription to the USMLE World Qbank Live interactive peer review sessions –Basic sciences –Organ systems

Purpose of This Research Goals of the “10 for ‘15 Initiative”: –Raise the mean score for examinees in 2013 by 10 points above the previous year’s –Increase the percentage of students passing on their 1 st attempt Did we achieve our goals? Why or why not?

Design & Methods Retrospective –165 “Class of 2015” students met inclusion criteria Quasi-experimental –Everyone received the interventions Investigated other variables to eliminate potential internal validity threats –“Class of 2014” (n = 155) vs. “Class of 2015” –MCAT & M1/M2 course scores in addition to USMLE Step 1 performance

Results Between 2012 and 2013, UIC’s mean score increased by 6.28 points –Significantly greater than the national rate of increase –Prior years’ increases not significantly different from national increases UIC’s 2013 mean score exceeded the national mean for the first time in 15 years (and possibly ever) Percentage of students passing on 1 st attempt increased from mean of 90.5% from to 97% in 2013

With 2013 Data

Results – Continued “10 for ’15 Initiative” implemented

Conclusions This program was partially successful –Nearly reached explicit “10 for ‘15” goal AND –Rate of increase in mean scores and percentage of students passing on first attempt improved significantly compared to the national average

Lessons Learned Quasi-experimental design not ideal –Many inherent threats to internal validity –Experiment could not have been carried out in complete fairness otherwise Multi-faceted intervention –Unclear which aspects were effective

Future Directions for this Research Determine which interventions were effective Student-initiated nature is unique –Warrants further study into effects on student motivation and study practices

Special Thanks To: Dr. Abbas Hyderi, MD MPH; Dr. Matthew Lineberry, PhD; Dr. Yoon-Soo Park, PhD The “10 for ‘15 Committee” –Gabriela Aguilar, Brian Bamberger, Geoanna Bautista, Tiffani Berkel, Mary Cronin, Lauren Fiske, Abbas Hyderi, David Hodge, Peter Lazzari, Julie Mann, Alexandra Melnyk, Juan Pablo Mosqueda, Matthew Orlando, Mark Urosev, Kelly-Ann Vassell, Stacey Walters, Raul Vasquez The UIC Department of Family Medicine The UIC COM Class of 2015

Questions? Please evaluate this session at: stfm.org/sessionevaluation