CLICK TO GO BACK TO KIOSK MENU Correlation Coefficient USMLE Scores Do Not Predict Ultimate Clinical Performance in an EM Residency Program CLICK TO GO BACK TO KIOSK MENU Karima R Sajadi*, Mark Saks#, Edward A Ramoska* *Drexel University College of Medicine, #Crozer Chester Medical Center Introduction Discussion Results “High-stakes” multiple choice exams are widely used to gauge mastery of basic and clinical science knowledge Scores on these exams are important screening and applicant ranking criteria We attempt to clarify the relationship between performance on two USMLE exams (Step 1, Step 2CK) with global clinical performance in an EM residency program We tested the hypothesis that USMLE scores do not predict clinical performance during residency training USMLE scores are important as resident selection criteria USMLE scores have been shown to correlate with future exams (ITE1,3, ABEM qualifying exam2,3) However, they should not be used to predict the applicant’s actual clinical performance as judged by residency leadership During the 8 years there were a total of 115 graduating residents 73 men (63%) and 42 women 109 (95%) had allopathic medical degrees; the remainder had osteopathic degrees Table 1 shows the distribution of the final consensus ranking. The inter-rater reliability was strong with an intraclass correlation coefficient = 0.845 (p<0.01) There was a statistically significant correlation between our ranking of clinical performance and Step 2 CK score (Spearman’s rho) There was no statistically significant correlation between clinical performance and Step 1 score Conclusion Neither USMLE Step 1 nor Step 2CK were good predictors of clinical performance during training USMLE scores may be overemphasized in the resident selection process Methods All graduating residents from our PGY 1-3 University-based EM residency program with complete USMLE scores From 2008 to 2015 Clinical performance was defined as a gestalt of the residency program’s leadership (PD/2 APDs) Initially blinded to each other’s grouping selections Classified the residents into 3 sets: top, average, and lowest clinical performer Dissimilarities of the rankings were adjudicated during a consensus conference Category Number Percentage Top 38 33% Middle 44 38% Bottom 33 29%   USMLE Step 1 USMLE Step 2CK Correlation Coefficient 0.067 0.205 P Value 0.49 0.04 N 109 106 References Table 1. FINAL RANKING OF RESIDENTS Thundiyil JG, Modica RF, Silvestri S, Papa L: Do United States Medical Licensing Examination (USMLE) scores predict in- training test performance for emergency medicine residents? J Emerg Med. 2010; 38:65-9. Harmouche E, Goyal N, Pinawin A, Nagarwala J, Bhat R: WesterUSMLE Scores Predict Success in ABEM Initial Certification: A Multicenter Study. West J Emerg Med 2017;18:544-549. Nelson M, CalandrellaC: Does USMLE Step 1 & 2 Scores Predict Success on ITE and ABEM Qualifying Exam: A Review of an Emergency Medicine Residency Program from its Inception Ann Emerg Med 2017: 70: S58-S59. Table 2. CORRELATION BETWEEN CLINICAL PERFORMANCE AND EXAM SCORES