1. Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD 2, and Norman L. Wool MD 1 Department of General Surgery, Rush University Medical Center, Chicago, IL 1 and Department of Psychology, Barnard College, New York, NY 2

2. Background Bias created by open file interviews  Robin, Bombeck, Pollak, Nyhus (1991)  Miles, Shaw, Risucci (2001) Closed file interviews designed to reduce bias

3. Purpose of the Study To determine the efficacy of enhanced interview and rating process

4. Research Questions Is there a difference between faculty ratings of applicant characteristics in open and closed file conditions? Do ratings of applicants, in these conditions, vary by gender and known status?

5. Study Participants Large, urban academic general surgery program 70 residency candidates (interviewees)  44 Male  26 Female 17 surgery faculty members (interviewers)  12 Male  5 Female

6. Closed File Rating Process Each applicant was interviewed by 2 faculty members (140 total interviews) Closed file interview  Name  Personal statement  Undergraduate institution  Medical school Faculty completed evaluation instrument

7. Open File Rating Process Faculty reviewed applicant files  ERAS application  Transcript  USMLE Scores  MSPE  Letters of Recommendation Faculty completed the same evaluation instrument

8. Evaluation Instrument Motivation for learning Motivation to do research Ability to work effectively with others Commitment to general surgery Likelihood to succeed in surgery Overall fit in our program Self-confidence Communication Intelligence Maturity Compassion Interpersonal Skills Motivation for Surgery Miles WS, Shaw V, and Risucci D. The Role of blinded interview in the assessment of surgical residency candidates. The American Journal of Surgery. 182 (2001) 143-162.

9. Data Analysis Factor analysis to yield rating components MANOVA on:  Interview type (closed vs. open)  Known status (known vs. unknown)  Gender (male vs. female)  Ratings

10. Results No significant differences between  Ratings between open vs. closed file (F=.34, p=.56)  Open vs. closed file ratings by gender (F=.04, p=.84)

11. Results: Interaction Effects Significant interaction effects between:  Open vs. closed file ratings by known status (F=4.7, p=.04)  Open vs. closed file ratings by trait (F=1.8, p=.04)  Open vs. closed file ratings by trait, gender and known status (F=1.98, p=.02)  Trait by gender (F=3.1, p=.001)

12. Results: Closed vs. Open By trait (F=1.8, p=.04) Closed Open Intelligence4.194.33 Maturity4.164.29 Commitment to general surgery4.324.41 Qualifications to succeed4.204.32

13. Results: Closed vs. Open By known status (F=4.7, p=.04) Known Unknown Maturity (open)4.024.43 Motivation for general surgery (closed)4.264.59 Working with others (closed)4.914.36 Qualifications to succeed (open)4.044.41 Likelihood to succeed (open)4.004.45 Fit to program (open)4.004.33

14. Results: Closed vs. Open Closed Open Known females  Communication 4.44.0  Interpersonal skills 4.43.7  Motivation for research 3.63.1 Known males  Qualifications to succeed 4.44.0

15. Results: Closed vs. Open Maturity Ratings Closed Open Known Male3.84.0 Unknown Male4.24.3 Known Female4.64.4 Unknown Female4.24.6

16. Results: Trait by Gender Self-confidence4.34.4 Communication4.3 Intelligence4.44.3 Maturity4.14.4 Compassion4.34.4 Interpersonal skills4.34.2 Motivation for surgery4.54.6 Motivation for learning4.6 Motivation for research4.23.6 Ability to work effectively with others4.54.4 Commitment to general surgery4.44.6 Qualifications to succeed in surgery4.24.3 Likelihood to succeed in surgery4.24.3 Overall fit for our program4.2 M F

17. Factor Analysis Results Social Traits  Communication = 0.87  Maturity = 0.85  Self-confidence = 0.77  Interpersonal skills =0.76 General Surgery  Commitment to general surgery = 0.871  Motivation to do general surgery = 0.85 Research  Motivation to do research = 0.89

18. Conclusions No significant change in closed and open file evaluations except:  Differences in ratings of known vs. unknown applicants  Differences in ratings of some traits by the gender of the applicant Constructs evaluated are social traits, motivation for general surgery, and research