1. CLICK TO GO BACK TO KIOSK MENU
Are Standardized Video Interview (SVI) scores predictive of in-person interview performance?
CLICK TO GO BACK TO KIOSK MENU
T Surles MD, J Willis MD, T Smith MD, M Silverberg MD, J Schechter MD, H LoCascio MD, S Kendall MD, A Regan MD, J Gernsheimer MD
Department of Emergency Medicine at SUNY Downstate Medical Center; and Kings County Hospital Center
INTRODUCTION
RESULTS (DATA)
RESULTS
BACKGROUND: The Standardized Video Interview (SVI) was developed by the Association of American Medical Colleges (AAMC) to be used in Electronic Residency Application Service (ERAS) applications for residency. The goal for the SVI is to aid program directors to identify applicants who do not perform well on other objective markers and increase the chance of an interview. The SVI tests interpersonal communication and professionalism which are factors evaluated in an interview. There is controversy whether the SVI score is valuable to the application and how it should be used. One proposed use of the SVI is to predict how an applicant will interview.
Objectives
-The objective is to determine if SVI scores predict how an applicant performs in an interview. We believe that the SVI score will correlate with interview scores.
253 applicants were interviewed in the season. The demographics at this time are similar with 147 male and 106 female. Students were selected from medical schools across the United States, and from medical schools abroad. One student conducted his interview via FaceTime, so he was not given an IE score, and his data was excluded. The results of the analysis produced a correlation coefficient of 0.22 showing a low degree of correlation at a p value of <0.01.
CONCLUSION
METHODS
The correlation coefficient shows a low degree of correlation. This suggests that the SVI does not correlate well with the interview evaluation at SUNY Downstate. The p value shows that the low correlation is significant and not likely due to chance. This data is limited to only in-person interview performance and does not correlate to the student’s overall rank in the match, nor does it relate to how successful the student will be in residency. As Emergency Medicine is the first speciality to use these scores more research needs to occur to determine their value. At this time the SVI score does not seem to add significant data to the ERAS application when used at SUNY Downstate.
Traditionally, at SUNY Downstate interviewees are scored based on their objective data (USMLE, Dean’s Letter, LORs) as well as an interview evaluation (IE) which reflects their performance during the interview. This application season the SVI score was not used in selection or ranking of students. Applicants were interviewed by 5 faculty and the median of the IE’s were used. Retrospectively, the SVI score is compared to the IE by a third party. This third party did not participate in interviewing and used de-identified data. The SVI and IE scores were converted to percentages and ranked to standardize the data. The null hypothesis that there is no statistically significant correlation was tested. A 2-tailed Spearman Rank Correlation Test with an alpha of 0.05 was used.

2. INTRODUCTION OBJECTIVE
Are Standardized Video Interview (SVI) scores predictive of in-person interview performance?
T Surles MD, J Willis MD, T Smith MD, M Silverberg MD, J Schechter MD, H LoCascio MD, S Kendall MD, A Regan MD, J Gernsheimer MD
Department of Emergency Medicine at SUNY Downstate Medical Center; and Kings County Hospital Center
According to the AAMC; “The purpose of the video interview is to provide objective, standardized information about applicants’ standing on two critical ACGME competencies: Interpersonal and Communications Skills and Knowledge of Professional Behaviors (Professionalism)”. This will help programs to identify relevant nonacademic information about applicants that may not be easily retrieved elsewhere in the ERAS application. Also, it will help to widen the pool of applicants invited to interview in person, including those who might not have otherwise been considered for interview
Format: The format of the interviews is highly regimented. It consists of 6 total questions. The applicants will have 30 seconds to read each prompt, then 3 minutes to answer each question. Each question is graded on a scale of 1 (rudimentary) - 5 (exemplary). Final scores are from 6-30.
The questions are a mixture of behavioral and situational questions that will test several things including, but not limited to, an applicants’ altruism, emotional intelligence, leadership abilities, oral communication, and empathy. Each applicant is graded by 6 different raters that are specially trained in bias reduction amd in the core competencies that are being tested in each question. This is done so that each rater does not have too much influence on each applicant’s grade as a whole.
INTRODUCTION
OBJECTIVE
SVI developed to identify applicants who do not perform well on other objective markers and increase the chance of an interview
Tests interpersonal communication and professionalism which are factors evaluated in an in person interview
One proposed use of the SVI is to predict how an applicant will interview
Determine if SVI scores predict how an applicant performs in an interview
We believe that the SVI score will correlate with interview scores

3. Are Standardized Video Interview (SVI) scores predictive of in-person interview performance?
T Surles MD, J Willis MD, T Smith MD, M Silverberg MD, J Schechter MD, H LoCascio MD, S Kendall MD, A Regan MD, J Gernsheimer MD
Department of Emergency Medicine at SUNY Downstate Medical Center; and Kings County Hospital Center
According to the AAMC; “The purpose of the video interview is to provide objective, standardized information about applicants’ standing on two critical ACGME competencies: Interpersonal and Communications Skills and Knowledge of Professional Behaviors (Professionalism)”. This will help programs to identify relevant nonacademic information about applicants that may not be easily retrieved elsewhere in the ERAS application. Also, it will help to widen the pool of applicants invited to interview in person, including those who might not have otherwise been considered for interview
Format: The format of the interviews is highly regimented. It consists of 6 total questions. The applicants will have 30 seconds to read each prompt, then 3 minutes to answer each question. Each question is graded on a scale of 1 (rudimentary) - 5 (exemplary). Final scores are from 6-30.
The questions are a mixture of behavioral and situational questions that will test several things including, but not limited to, an applicants’ altruism, emotional intelligence, leadership abilities, oral communication, and empathy. Each applicant is graded by 6 different raters that are specially trained in bias reduction amd in the core competencies that are being tested in each question. This is done so that each rater does not have too much influence on each applicant’s grade as a whole.
METHODS
METHODS
Traditional scoring used:
USMLE
Dean’s Letter
LORs
Interview Evaluation (IE)
SVI score was not used in selection or ranking of students
Applicants were interviewed by 5 Faculty and the median of the IE was used
Retrospectively, the SVI is compared to the IE by a third party
Third party did not interview and used de-identified data
SVI and IE scores were converted to percentages and ranked to standardize the data.
The null hypothesis that there is no statistically significant correlation
Tested using a 2-tailed Spearman Rank Correlation Test with an alpha of 0.05 was used

4. Are Standardized Video Interview (SVI) scores predictive of in-person interview performance?
T Surles MD, J Willis MD, T Smith MD, M Silverberg MD, J Schechter MD, H LoCascio MD, S Kendall MD, A Regan MD, J Gernsheimer MD
Department of Emergency Medicine at SUNY Downstate Medical Center; and Kings County Hospital Center
RESULTS (DATA)
Correlation coefficient of 0.22 showing a low degree of correlation at a p value of <0.01
Null Hypothesis is rejected that SVI scores are predictive of interview evaluation at SUNY Downstate
The demographics are similar with 147 male and 106 female
Students were selected from medical schools across the United States, and from medical schools abroad
Fig 1: The results of the 2-Tailed Spearman Rank Correlation Test.
Graph 1: Interview Evaluation vs Standardized Video Interview

5. Are Standardized Video Interview (SVI) scores predictive of in-person interview performance?
T Surles MD, J Willis MD, T Smith MD, M Silverberg MD, J Schechter MD, H LoCascio MD, S Kendall MD, A Regan MD, J Gernsheimer MD
Department of Emergency Medicine at SUNY Downstate Medical Center; and Kings County Hospital Center
Standardized Video Interviews have been used in the business word for several years now with good success. Our data suggests that it will not add very much to the methods used at SUNY Downstate that have been turning out quality residents for many years. It is possible that while this new data point does not work well with our emergency medicine residency, it may work well in other specialties, or other emergency medicine residencies. While the parameters being tested are universal, each residency has it’s own culture and sensibilities. Even within the same hospital, a surgery specialty is not the same as an emergency one. It is possible that the SVI will be invaluable for other specialties. In this way, the business world is drastically different from the medicine world. Most businesses have a unifying identity that makes the SVI applicable to every department. Currently, more research in different specialties is needed before we can fully assess the utility of the SVI.
CONCLUSION
Limitations
The correlation coefficient of .22 suggests there is low correlation between SVI and IE scores at SUNY Downstate
P value <.01 shows that the low correlation is significant and not likely due to chance
At this time the SVI score does not seem to add significant data to the ERAS application when used at SUNY Downstate
Limited to in-person interview performance and does not correlate to the student’s overall rank in the match or how successful the student will be in residency
More research needs to occur to determine the true value of the SVI
CITATIONS
1) ”The AAMC Standardized Video Interview: Essentials for the ERAS® 2019 Season”
2) “The Standardized Video Interview” Ashley Rider, MD,