Assessing Medical Student Skills in Using Interpreters STFM Annual Spring Conference, 30 April 2006 San Francisco, CA Désirée Lie, MD, MSEd, Paula Henderson, MD, Ellen Lewis, RN, MSN, FAAN, Sue Ahearn, RN, and John Boker, PhD University of California, Irvine and University of California, Los Angeles David Geffen Schools of Medicine

Assessing MS Use of Medical Interpreters Session Objectives: Identify skills and knowledge for effectively using medical interpreters. Describe methods for assessing skills. View and critique an assessment case. Discuss obstacles/solutions for assessing students.

Using Interpreters in Medical Encounters Audience Survey: How many currently teach use of medical interpreters in the formal curriculum? What teaching methods are used? What assessment methods are used? If not currently assessing students, how many plan to assess students in this skill?

Using Interpreters in Medical Settings Rationale for Teaching: 10 million Americans with LEP (Limited English Proficiency). LEP contributes to poor access to care, poor health outcomes and affects quality of clinician-patient relationship. Using interpreters is a core cultural competency. Improving clinician cultural competency shown to improve health outcomes. Ferguson W. Culture, Language, and the Doctor-Patient Relationship. Fam Med. 2002;34:353-61. Drouin J. Training Medical Students to Communicate with a Linguistic Minority Group. Acad Med. 2003;78(6):599-604. Karliner, LS, Eliseo J. Pérez-Stable, Ginny Gildengorin, The Language Divide. The Importance of Training in the Use of Interpreters for Outpatient Practice, JGIM, 19;175, Feb 2004

Using Interpreters in Medical Encounters What skills and knowledge are involved regardless of the language used (Audience)? Knowledge Skills U.S. Department of Health and Human Resources, Office of Minority Health. National Standards for Culturally and Linguistically Appropriate Services. Rockville, MD: DHHS. http://www.omhrc.gov/clas/

Assessing Medical Students Potential methods of assessment: Direct observation with feedback Patient surveys and feedback SP cases Self-assessment/reflection Written cases Computer/web-based simulations (eg. www://education.med.nyu.edu/interpmod/macy_teach/) Others?

Using Interpreters in Medical Encounters Challenges of designing SP Case Include/exclude culture-specific issues? Level of English proficiency of patient? Level of interpreting skill of interpreter?

Using Interpreters in Medical Encounters Challenges of Designing SP Case (contd.): 4. Who assesses the student? (faculty, patient, interpreter?) 5. What standards to use (e.g. when student has some vs. no language proficiency)? 6. Do skills generalize to encounters with other languages? 7. Other challenges?

Using Interpreters in Medical Encounters

Assessing Medical Students Pilot Project (UCI, 2005) Background: Chose MS3 9-station CPX to pilot interpreter station at end of year 3. Sought expert opinion (panel) and performed lit. review on smoking cessation and cultural issues. Adapted existing case. IRB approved.

Assessing Medical Students Pilot project participants (UCI, 2005) SP: Miguel Martinez Monolingual Latino male businessman age 40s, seeking help with smoking cessation. Interpreters: 11 students at end of training. MS3: Randomly selected 39/92 MS3 from required CPX.

Assessing Medical Students Measures (empiric) – UCI, 2005 SP assessment of MS3 (14-item yes/no). Interpreter assessment of MS3 (8-item yes/no). MS3 self-assessment (11 item). Skills assessed: History-taking, negotiation, communication (with SP and interpreter), explanatory facility, language proficiency (student)

Use of Interpreters in Medical Encounters Measures that were not used (UCI): Faculty assessment of MS3. MS3 assessment of interpreter skills.

Assessing Medical Students UCLA (2006) MS2 Case: Small group SP teaching with untrained interpreters deliberately making errors 3 months later, assessment with SP case

Assessing Medical Students Teaching Case MS2 (UCLA 2006) SP: Sylvia and Kenny Jimenez Monolingual Latina mother brings her 20 month old son with symptoms of gastroenteritis. Interpreter: Bilingual patient in the clinic who has no relationship with the patient and has never met her before. Simply another patient in the clinic who was asked to help out, and you are happy to do so.

Assessing Medical Students Assessment Case MS2 (UCLA 2006) SP: Soledad/Cesar Clemente Monolingual recent immigrant from Mexico who has had a dry, hacking cough for approximately 1 month. SI: Bilingual clerk at an Urgent Care Clinic in downtown LA who has been asked to translate for a patient who does not speak English. You do not have professional training as an interpreter.

Assessing Medical Students Measures (empiric) – UCLA 2006 SP assessment of MS2 (12-item yes/no). SI assessment of MS2 (8-item yes/no). MS2 self-assessment (11 item). Skills assessed: History-taking, negotiation, communication (with SP and SI), explanatory facility, language proficiency (student)

Assessing Medical Students UCLA Changes: Added global satisfaction measure for SP Assessed seating arrangement chosen by MS2 Instructed students to use interpreter regardless of own language skills Used same SP and self-assessment SP checklists as UCI

Using Interpreters in Medical Encounters Audience task: Select a checklist (SP or interpreter). Assess student #1 in video. Assess student #2 in video. Discussion: What are we measuring with checklists? How do we set performance standards? What remediation might follow?

Using Interpreters in Medical Encounters Results of Pilot Study (UCI) MS3 perception of encounter higher if self-identified as high Spanish fluency (α=0.80). MS rated interpreter participation good to excellent 80% of the time. SPs scored MS3 highly (80% items done, α=0.54).

Use if Interpreters in Medical Encounters Results of pilot study (UCI, contd.): SP’s gave higher scores to MS3 with lower Spanish proficiency. Interpreters gave lower score to students with lower Spanish proficiency but reliability was low (α=0.25).

Assessing Medical Students UCLA Results: SP: All PPI items are highly correlated with OS. Overall Satisfaction is slightly correlated with HX012 (.18) HX013 and HX014 (.22). discussed confidentiality. asked what the patient thought might be going on OR what s/he thought might be causing the problem OR what s/he is most concerned about or afraid of. asked how the patient has been coping with the problem and/or how it has affected his/her day-to-day activities.

Assessing Medical Students UCLA Results: (Regression Analysis) SP: The strongest predictor of OS is PPI016 (accounted for 50% of variance).  Combining with HX012, HX013 and PPI019, all 4 variables accounted for 55% of variance. PPI016. Made the patient feel supported and taken care of (verbally and non-verbally; showed interest in me as a person, not just my condition; did not make me feel rushed). PPI019. Explained things in a manner that could be understood (adapted language to accommodate both translator and patient; avoided jargon). HX12. discussed confidentiality. HX13. asked what the patient thought might be going on OR what s/he thought might be causing the problem OR what s/he is most concerned about or afraid of.

Assessing Medical Students UCLA Results: Sitting arrangement made no difference HX05 rearranged the chairs to facilitate the interview ( translator sitting next to interviewer or behind interviewer’s shoulder, so patient can look at both simultaneously) SPs gave higher scores to MS3 with lower Spanish proficiency.

Using Interpreters in Medical Encounters What next? Add faculty checklist Create cases for different languages (Spanish vs. Vietnamese), i.e. generalizability. Pre- and post test (MS2 vs MS3) Validation and reduction of checklists

Use of Interpreters in Medical Encounters What next (contd.)? SP assessments in ‘real’ encounters (unannounced SPs) Identifying students in difficulty for remediation.

Using Interpreters in Medical Encounters Getting started (audience suggestions): Which MS year? What setting? What performance measures? What outcomes? Consider: Unannounced SP’s Assess students ability to evaluate interpreter performance

Using Interpreters in Medical Encounters

Assessing Medical Students Acknowledgements: Grants (UCI and UCLA): NIH (NHLBI) grant K07 HL079256-01 and Molina Health Care. Contact: Désirée Lie, MD, MSED (dalie@uci.edu) Paula Henderson, MD (paulahenderson@mednet.ucla.edu) THANK YOU! QUESTIONS?