Background: As students complete their clerkships throughout their M3 year they gain in clinical experience and confidence, which may translate into improved performance on various end-of-rotation assessments. Specialties such as surgery, obstetrics/gynecology, internal medicine, family medicine, and pediatrics have demonstrated a “timing bias” of improved scores on the NBME subject exams as students progress through their third year 1-2. This is relevant since students who wish to pursue a specific specialty may purposely choose that rotation later in the academic year, thereby “gaming the system.” Conversely, students who take a clerkship early due to interest may be penalized with a lower NBME grade. The literature for psychiatry clerkships has shown no change or slight increases in NBME scores as the year progresses 3. One study extended the concept of timing bias to 3 areas of student evaluation: the supervisors’ evaluations, SP exam (SPE), and NBME subject exam scores. While there was a weak linear trend for the NBME, the 2 other measures showed no positive trend 4. We wondered if these findings could be replicated at our institution.Methods: Data was collected from 2 academic years, Rotations for each year were numbered from 1-7, beginning with the first rotation in July of each year. The mean scores for three score components – supervisors’ evaluations, SPE scores, and NBME scores – were calculated for each rotation. Data from rotation #4 (December) was not included as that rotation was composed of 4th-year medical students taking their clerkship out of sequence. We conducted ANOVAs for each of the three score components across time looking for linear trends. Does Increased Clinical Experience Improve Performance in the Psychiatry Clerkship? Sean M. Blitzstein M.D. 1, Rachel Yudkowsky M.D. MHPE 2, and Mihaela Popescu 2 1 Department of Psychiatry and 2 Department of Medical Education University of Illinois at Chicago Discussion: Our study demonstrated a significant upward trend for scores on the NBME over time, consistent with findings in other medical specialties. However, the increased knowledge and the clinical experience gained in prior clerkships did not lead to improved supervisors’ evaluations or SPE grades. The lack of improvement in supervisor evaluations was surprising, given the commonly held belief that students improve in their performance over the year. While the evaluations used at our institution have some items particular to psychiatry (MSE, etc), there are many general skills which could be transferable across specialties (interviewing, data gathering, professionalism, response to feedback, etc). Our results may reflect the limited reliability and validity of end-of-rotation supervisor evaluations, well documented in the literature 5. The lack of a trend for the SPE grades over time was not surprising. The knowledge and skills for the SPE are likely case- specific as well as specialty (psychiatry)-specific, therefore not transferable from other specialties as the year progresses.Conclusions: Students who are considering a residency in psychiatry and want to ensure a good clerkship grade may feel they are at a disadvantage if selecting their psychiatry rotation earlier in the academic year. The 5-point NBME disadvantage could, in some cases, make the difference between an “outstanding” and “advanced” grade in the clerkship. Whalen 1 suggested adjusting NBME scores to compensate for the advantage of later experience, but this raises additional complications. In our opinion the NBME disadvantage does not outweigh the career- planning advantages of an early clerkship. We do not recommend that students interested in psychiatry postpone their rotation to later in the year.References: 1.Whalen JP, Moses VK. The effect on grades of the timing and site of third-year internal medicine clerkships. Acad Med. 1990; 65: Baciewicz FA, et al. Influence of clerkship structure and timing on individual student performance. Am J Surg. 1990; 159: Manley M, Heiss G. Timing bias in the psychiatry subject examination of the national board of medical examiners. Acad Psychiatry. 2006; 30: Park RS, et al. Relationship of rotation timing to pattern of clerkship performance in psychiatry. Acad Psychiatry. 2006; 30: Williams RG, et al. Cognitive, social and environmental sources of bias in clinical performance ratings. Teach Learn Med. 2003; 15: Purpose: At UIC the 6-week Psychiatry Clerkship begins at seven different times during medical students’ third (M3) year. We wanted to determine if scores on the psychiatry clerkship supervisor evaluations, standardized patient exam (SPE), and/or the National Board of Medical Examiners Psychiatry Subject Examination (NBME) increased in psychiatry rotations taken later in the M3 year. For more information, please contact Sean Blitzstein, MD The Three Score Components: The final grade of the six-week psychiatry clerkship is based on three score components: 1. Supervisor Evaluations Supervisors rate eight competencies (e.g., interviewing, professionalism, MSE) resulting in a summary global rating on a 4-point scale: 1 = needs remediation, 2 = proficient, 3 = advanced, and 4 = outstanding 2. Standardized Patient Exam (SPE) Each student encounters 3 SPs portraying common psychiatric disorders for 30 minutes each and documents the MSE and differential diagnosis. The SPE score is composed of 4 elements: SPs complete (1) an interview checklist and (2) a communications skills scale after each encounter. Faculty score (3) the MSE and (4) differential diagnoses. These four components are averaged to obtain a final SPE grade of needs remediation, proficient, advanced, or outstanding (same scale as supervisor evaluations). 3. NBME psychiatry subject exam Passing score of 60 Results: A total of 298 students participated in the psychiatry clerkship in Supervisors’ evaluations (N=297) were not equivalent across rotations. ANOVA F (5) =4.370, p=.001 Post-hoc analysis (Scheffe) showed rotation #3 differed significantly from #5 and #6, however no linear trend was observed. SPE scores did not show significant differences or linear trends over time (N=268), ANOVA F (5) =1.138, p=.341 NBME scores demonstrated a significant upward linear trend: (N=298), ANOVA F (5) =3.909, p=.002, Linear contrast F=16.760, p<.005 Supervisor Grade x2 SPE Grade NBME Grade Clinical Grade x2 Final Clerkship Grade Rotation number MonthSupervisor Grade Mean (SD) SPE grade Mean (SD) NBME grade Mean (SD) 1June3.25 (.52)2.64 (.65)73.70 (8.79) 2August3.19 (.58)2.74 (.74)73.28 (8.04) 3October3.48 (.64)2.80 (.67)75.15 (8.89) 5January3.00 (.65)2.62 (.70)78.12 (8.30) 6March3.09 (.64)2.74 (.75)76.96 (8.61) 7April3.36 (.62)2.93 (.66)79.79 (7.21)