Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD 2, and Norman L. Wool MD 1 Department of General Surgery, Rush University Medical Center, Chicago, IL 1 and Department of Psychology, Barnard College, New York, NY 2

Background Bias created by open file interviews  Robin, Bombeck, Pollak, Nyhus (1991)  Miles, Shaw, Risucci (2001) Closed file interviews designed to reduce bias

Purpose of the Study To determine the efficacy of enhanced interview and rating process

Research Questions Is there a difference between faculty ratings of applicant characteristics in open and closed file conditions? Do ratings of applicants, in these conditions, vary by gender and known status?

Study Participants Large, urban academic general surgery program 70 residency candidates (interviewees)  44 Male  26 Female 17 surgery faculty members (interviewers)  12 Male  5 Female

Closed File Rating Process Each applicant was interviewed by 2 faculty members (140 total interviews) Closed file interview  Name  Personal statement  Undergraduate institution  Medical school Faculty completed evaluation instrument

Open File Rating Process Faculty reviewed applicant files  ERAS application  Transcript  USMLE Scores  MSPE  Letters of Recommendation Faculty completed the same evaluation instrument

Evaluation Instrument Motivation for learning Motivation to do research Ability to work effectively with others Commitment to general surgery Likelihood to succeed in surgery Overall fit in our program Self-confidence Communication Intelligence Maturity Compassion Interpersonal Skills Motivation for Surgery Miles WS, Shaw V, and Risucci D. The Role of blinded interview in the assessment of surgical residency candidates. The American Journal of Surgery. 182 (2001)

Data Analysis Factor analysis to yield rating components MANOVA on:  Interview type (closed vs. open)  Known status (known vs. unknown)  Gender (male vs. female)  Ratings

Results No significant differences between  Ratings between open vs. closed file (F=.34, p=.56)  Open vs. closed file ratings by gender (F=.04, p=.84)

Results: Interaction Effects Significant interaction effects between:  Open vs. closed file ratings by known status (F=4.7, p=.04)  Open vs. closed file ratings by trait (F=1.8, p=.04)  Open vs. closed file ratings by trait, gender and known status (F=1.98, p=.02)  Trait by gender (F=3.1, p=.001)

Results: Closed vs. Open By trait (F=1.8, p=.04) Closed Open Intelligence Maturity Commitment to general surgery Qualifications to succeed

Results: Closed vs. Open By known status (F=4.7, p=.04) Known Unknown Maturity (open) Motivation for general surgery (closed) Working with others (closed) Qualifications to succeed (open) Likelihood to succeed (open) Fit to program (open)

Results: Closed vs. Open Closed Open Known females  Communication  Interpersonal skills  Motivation for research Known males  Qualifications to succeed

Results: Closed vs. Open Maturity Ratings Closed Open Known Male Unknown Male Known Female Unknown Female4.24.6

Results: Trait by Gender Self-confidence Communication4.3 Intelligence Maturity Compassion Interpersonal skills Motivation for surgery Motivation for learning4.6 Motivation for research Ability to work effectively with others Commitment to general surgery Qualifications to succeed in surgery Likelihood to succeed in surgery Overall fit for our program4.2 M F

Factor Analysis Results Social Traits  Communication = 0.87  Maturity = 0.85  Self-confidence = 0.77  Interpersonal skills =0.76 General Surgery  Commitment to general surgery =  Motivation to do general surgery = 0.85 Research  Motivation to do research = 0.89

Conclusions No significant change in closed and open file evaluations except:  Differences in ratings of known vs. unknown applicants  Differences in ratings of some traits by the gender of the applicant Constructs evaluated are social traits, motivation for general surgery, and research