Dealing with diversity in medical education Karen Stegers-Jager Institute of Medical Education Research Rotterdam (iMERR) My presentation this morning focuses on a major challenge that medical schools are currently facing: how to ensure that all students have similar chances of being successfull at medical school, including ethnic minority and first-generation university students. June 9th 2017

As this picture strikingly illustrates there are differences in the chances of becoming a medical doctor. Whereas the path for majority students appears smooth, the path for minority students has several obstacles.

Getting in – admissions1 A first obstacle is to get into medical school. Literature shows that non-traditional students are still underpresented in medical school. In a first study, we determined what the effect was of the use of our different selection criteria on student diversity. 1 Stegers-Jager et al Med Educ 2015;1:124-133.

Selection procedure Erasmus MC + The selection procedure of Erasmus MC consists of two parts. In the first, non-academic part, participants are assessed according to the quality and quantity of their extracurricular activities before application. The second, academic, part consisted of five cognitive tests, including logical reasoning, scientific thinking and, mathematics. At the time of this study pu-GPA was not yet taken into account.

Who gets in? - ethnicity Dutch < Non-Dutch < Sur/Ant & Tur/Mor/Afr Dutch ≈ Non-Dutch < Sur/Ant & Tur/Mor/Afr & Asian So who gets in? The bad news is that non-Dutch applicants were more likely to fail on any of the selection criteria compared with Dutch applicants. However, this difference was mainly attributable to two groups: Surinamese/Antillean applicants and Turkish/Moroccan/African applicants. Whereas 57% of the Dutch applicants failed on any of the selection criteria, this was 70% for the TMA and 78% for the Surinamese/Antillean applicants. The good news is that there were no significant differences between the ethnic subgroups in the chance of failure on the non-academic selection criteria However, non-Dutch applicants were more likely to fail on the academic selection criteria than Dutch applicants. 34% of the Dutch students failed on the academic criteria, this percentage was significantly higher for all other ethnic subgroups, except of the other Western applicants

Who gets in? – social background non 1st gen univ student ≈ 1st gen univ student non 1st gen univ student < 1st gen univ student Other good news was that first-generation university applicants were as likely to be selected as their traditional counterparts. Again there was no significant difference on the chance of failure on the non-academic criteria, but the first-generation university students were also more likely to fail on the academic part than non first gen univ applicants.

Discussion Ethnicity and social background independent predictors of admission, but surprising differences academic and non-academic criteria No differences non-academic criteria  self-selection before application? Poorer performance on cognitive tests non-traditional students Language skills?/ Cultural aspects? Let me first summarize our main findings: A first finding of our study is both ethnicity and social background were independent predictors of selection for medical school. However there were surprising differences in their relation with the non-academic and the academic selection criteria. One of the most surprising outcomes is the finding that there were no significant differences between the ethnic or social subgroups in the chance of failure based on extracurricular activities. Although these results are promising regarding increasing social and ethnic diversity; it cannot be ruled out that self-selection instigated by the selection procedure is stronger for applicants from non-traditional backgrounds. A third finding was that ethnicity and social background were related to poorer performance on the cognitive tests. This might be related to language skills and/or cultural aspects, but further research is required to explore these and other causes.

Practical implications Include non-academic criteria in selection procedure Use different selection criteria concurrently and explore impact of weighting on student diversity Our findings also have some implications for medical schools that aim to increase the diversity of their student population. A first suggestion would be to not only use academic criteria but also include non-academic criteria in the selection procedure, as these showed in our study to be most promising regarding increasing social and ethnic diversity. A second suggestion would be to use different selection criteria concurrently and to further explore the impact of different weightings of the criteria on student diversity.

Staying in – progression and retention1 A second challenge is that generally, non-traditional students, in particular those from an ethnic minority are reported to underperform compared with those from the ethnic majority. Therefore, we performed a second study to determine whether underperformance occurs across ethnic minority groups in pre-clinical and clinical training and whether this underperformance can be explained by confounders, and additional socio-demographic characteristics. 1 Stegers-Jager et al Med Educ 2012;46:575–85.

Results – Performance Year 1 course completion: Dutch (64%) > Non Dutch (52%) > Sur/Ant (37%), Asian (50%) Preclinical course completion: Dutch (41%) > Non Dutch (29%) Dutch (41%) > Sur/Ant (19%), Asian (24%) Good clinical performance: Dutch (88%) > Non Dutch (67%) Dutch (88%) > all 4 groups (54-77%) This slide shows the differences we found on three measures of medical school performance. Dutch students were more likely to complete the Year 1 course within one year compared to Non Dutch students. However, this difference was mainly attributable to two groups: Asian and Surinamese/Antillean students. Whereas 64% of the Dutch students completed the Year 1 course within 1 Year, this was only 50% for the Asian and 37% for the Surinamese/Antillean students. As one may expect, results on the preclinical course were comparable: Dutch students were more likely to complete the preclinical course within four years compared to Surinamese/Antillean and Asian students. To our surprise, the results on the clinical course showed a different picture. 88% of the Dutch students received three or more good or excellent grades for the first five clerkships. This percentage was significantly lower for all other ethnic subgroups, ranging from 54% to 77%. 10

Results – Performance (adjusted1) Year 1 course completion: Dutch (64%) > Non Dutch (52%) > Sur/Ant (37%), Asian (50%) Preclinical course completion: Dutch (41%) > Non Dutch (29%) Dutch (41%) > Sur/Ant (19%), Asian (24%) Good clinical performance: Dutch (88%) > Non Dutch (67%) Dutch (88%) > all 4 groups (54-77%) Age, gender, pu-GPA and socio-demographic factors could only explain the difference for Asian students in preclinical training. 1Adjusted for age, gender, pu-GPA & socio-demographic factors 11

Discussion Differences across ethnic minority groups in comparisons to majority students Greater ethnic disparities in clinical training  clinical skills?  communication styles?  stereotype threat?  examiner bias? A first finding of this study is that there are differences across ethnic minority groups in comparisons to majority students. Where the Surinamese/Antillean and Asian students underperformed throughout medical school, the Turkish/Moroccan and Western students only received lower clinical grades. Most of these differences could not be explained. A second finding was that the disparities were greater in clinical training. We considered several explanations: It might be that somehow non Dutch students are less well prepared for the clerkships, even those who do not clearly underperform in preclinical training. A second possible explanation are differences in communication styles. Another explanation may be stereotype threat. Stereotype threat suggests that underperformance in ethnic groups can be caused by increased anxiety arising due to the prospect of being negatively stereotyped. Stereotype threat might more prominent in clinical than preclinical training, due to the more subjective grading. A final explanation may be that this more subjective grading leads to examiner bias. 12

Practical implications Preclinical course: Targeted support Clinical course: Less subjective grading Awareness cultural bias Our findings also have some implications for practice: Since students from certain groups are more likely to fail than other students, targeted or proactive support for these groups might be appropriate. For example, additional support during Dutch preclinical training may be required for Surinamese/Antillean and Asian students. Within the clinical course, a first possible step is to make assessment less subjective, or at least to ensure that students from ethnic minorities are not disadvantaged. A second step might to create awareness of cultural bias and to develop a greater understanding of cultural differences through cultural competency training for both faculty and students 13

Staying in – different types of examinations1 As a follow-up we looked in a next study at whether underperformance occurs on different types of written and clinical examinations in preclinical training. 1 Stegers-Jager et al Adv Health Sci Educ 2016;21:1023-1046.

Different types of examinations Written Clinical Theoretical Clinical case A 75-year-old man, heavy smoker… Skills We included two types of theoretical knowledge tests: 3 end-of year clinical problem solving tests and two first-year end-of-block tests. CPSTs consists of a number of clinical cases, while block tests consists of series of mostly mc clinical theme-related questions. We also included the first-year language skills test and the three writing skills tests (one each year). Finally, the clinical skills examinations in year 2 and 3 of the bachelor degree course were included in this study.

Discussion – main findings Differences across ethnic subgroups and between different types of written and clinical exams CPSTs, language test & clinical exams: all 3 non-western groups ↓ Block tests & writing skills tests & Western minority: less consistent Age, gender, pu-GPA and socio-demographic variables could explain: YES: differences in theoretical exams (CPSTs & block tests) NO: differences in language, writing & clinical skills exams 1st generation university students only ↓ on language tests This study confirms earlier findings of ethnic disparities in preclinical training, but also shows that there are differences both across ethnic subgroups and between different types of written and clinical examinations. While all three non-Western ethnic minority groups underperformed on the CPSTs, the language skills test and the clinical exams, findings on the theoretical end-of-block tests and writing skills tests, and results for Western minority students were less consistent. Age, gender, pu-GPA and socio-demographic variables (including parental education and first language) could largely explain the ethnicity-related disparities in theoretical examinations, but not in language, writing and clinical skills examinations. First-generation university students only underperformed on the language skills test.

Discussion – explanations Difference between CPSTs & block tests Nature & required preparation No indication lower level clinical skills Western minority & 1st gen students More subjective grading in clinical training? Underperformance in language, writing & clinical skills exams not explained: Cultural differences in communication styles? One of the most surprising outcomes is the difference in findings on the CPSTs and the end-of-block tests, which are both written theoretical knowledge tests. A possible explanation lies in the nature of and the required preparation for these examinations. The CPSTs consist of a number of clinical cases that students have to prepare for in the week preceding the exam preferably with a group of fellow students. It is remarkable that there was no indication of a lower level of clinical skills for Western students & 1st gen university students. Apparently there is a difference in what is measured by the clinical exams in preclinical and in clinical training. The confounders and socio-demographic factors could not explain the ethnic related disparities on the language, writing and clinical skills examinations. This suggests that the underperformance in these examinations is due to other factors, such as cultural differences in communication styles.

Practical implications Avoid unintended effects of certain types of examinations for certain groups of students consider diversity in both test construction & implementation Offer additional exam-specific support for specific groups  e.g. formal meetings to prepare for CPSTs A first practical implication is to avoid possible unintended effects of certain types of examinations for certain groups of students. In order to ensure that non-traditional medical students are not disadvantaged, diversity should be considered both in test construction and implementation Additional support focused on specific examinations for specific groups of students might be appropriate. As an example, the additional support for the CPSTs might take the form of planning formal meetings for students to prepare for the examinations.

Take-home message In order to select and retain non-traditional students in the medical education pipeline, medical schools must design assessment strategies and, if necessary, additional targeted support programmes that create a level playing field for a diverse student population More information: k.stegers-jager@erasmusmc.nl

Statements In the Netherlands, we need minority doctors for minority patients Affirmative action for non-traditional applicants is necessary Differences in clinical grades are due to subjective grading Lower clinical grades reduce chances for specialty training Cultural sensitivity should be taught at (medical) school