1. Assessing Medical Student Skills in Using Interpreters STFM Annual Spring Conference, 30 April 2006 San Francisco, CA
Désirée Lie, MD, MSEd, Paula Henderson, MD, Ellen Lewis, RN, MSN, FAAN, Sue Ahearn, RN, and John Boker, PhD
University of California, Irvine and University of California, Los Angeles David Geffen
Schools of Medicine

2. Assessing MS Use of Medical Interpreters
Session Objectives:
Identify skills and knowledge for effectively using medical interpreters.
Describe methods for assessing skills.
View and critique an assessment case.
Discuss obstacles/solutions for assessing students.

3. Using Interpreters in Medical Encounters
Audience Survey:
How many currently teach use of medical interpreters in the formal curriculum?
What teaching methods are used?
What assessment methods are used?
If not currently assessing students, how many plan to assess students in this skill?

4. Using Interpreters in Medical Settings
Rationale for Teaching:
10 million Americans with LEP (Limited English Proficiency).
LEP contributes to poor access to care, poor health outcomes and affects quality of clinician-patient relationship.
Using interpreters is a core cultural competency.
Improving clinician cultural competency shown to improve health outcomes.
Ferguson W. Culture, Language, and the Doctor-Patient Relationship. Fam Med. 2002;34:
Drouin J. Training Medical Students to Communicate with a Linguistic Minority Group. Acad Med. 2003;78(6):
Karliner, LS, Eliseo J. Pérez-Stable, Ginny Gildengorin, The Language Divide. The Importance of Training in the Use of Interpreters for Outpatient Practice, JGIM, 19;175, Feb 2004

5. Using Interpreters in Medical Encounters
What skills and knowledge are involved regardless of the language used (Audience)?
Knowledge
Skills
U.S. Department of Health and Human Resources, Office of Minority Health. National Standards for Culturally and Linguistically Appropriate Services. Rockville, MD: DHHS.

6. Assessing Medical Students
Potential methods of assessment:
Direct observation with feedback
Patient surveys and feedback
SP cases
Self-assessment/reflection
Written cases
Computer/web-based simulations (eg. www://education.med.nyu.edu/interpmod/macy_teach/)
Others?

7. Using Interpreters in Medical Encounters
Challenges of designing SP Case
Include/exclude culture-specific issues?
Level of English proficiency of patient?
Level of interpreting skill of interpreter?

8. Using Interpreters in Medical Encounters
Challenges of Designing SP Case (contd.):
4. Who assesses the student? (faculty, patient, interpreter?)
5. What standards to use (e.g. when student has some vs. no language proficiency)?
6. Do skills generalize to encounters with other languages?
7. Other challenges?

9. Using Interpreters in Medical Encounters

10. Assessing Medical Students
Pilot Project (UCI, 2005)
Background:
Chose MS3 9-station CPX to pilot interpreter station at end of year 3. Sought expert opinion (panel) and performed lit. review on smoking cessation and cultural issues. Adapted existing case.
IRB approved.

11. Assessing Medical Students
Pilot project participants (UCI, 2005)
SP: Miguel Martinez
Monolingual Latino male businessman age 40s, seeking help with smoking cessation.
Interpreters: 11 students at end of training.
MS3: Randomly selected 39/92 MS3 from required CPX.

12. Assessing Medical Students
Measures (empiric) – UCI, 2005
SP assessment of MS3 (14-item yes/no).
Interpreter assessment of MS3 (8-item yes/no).
MS3 self-assessment (11 item).
Skills assessed: History-taking, negotiation, communication (with SP and interpreter), explanatory facility, language proficiency (student)

13. Use of Interpreters in Medical Encounters
Measures that were not used (UCI):
Faculty assessment of MS3.
MS3 assessment of interpreter skills.

15. Assessing Medical Students
UCLA (2006) MS2 Case:
Small group SP teaching with untrained interpreters deliberately making errors
3 months later, assessment with SP case

16. Assessing Medical Students
Teaching Case MS2 (UCLA 2006)
SP:
Sylvia and Kenny Jimenez
Monolingual Latina mother brings her 20 month old son with symptoms of gastroenteritis.
Interpreter:
Bilingual patient in the clinic who has no relationship with the patient and has never met her before. Simply another patient in the clinic who was asked to help out, and you are happy to do so.

17. Assessing Medical Students
Assessment Case MS2 (UCLA 2006)
SP:
Soledad/Cesar Clemente
Monolingual recent immigrant from Mexico who has had a dry, hacking cough for approximately 1 month.
SI:
Bilingual clerk at an Urgent Care Clinic in downtown LA who has been asked to translate for a patient who does not speak English. You do not have professional training as an interpreter.

18. Assessing Medical Students
Measures (empiric) – UCLA 2006
SP assessment of MS2 (12-item yes/no).
SI assessment of MS2 (8-item yes/no).
MS2 self-assessment (11 item).
Skills assessed:
History-taking, negotiation, communication (with SP and SI), explanatory facility, language proficiency (student)

19. Assessing Medical Students
UCLA Changes:
Added global satisfaction measure for SP
Assessed seating arrangement chosen by MS2
Instructed students to use interpreter regardless of own language skills
Used same SP and self-assessment SP checklists as UCI

20. Using Interpreters in Medical Encounters
Audience task:
Select a checklist (SP or interpreter).
Assess student #1 in video.
Assess student #2 in video.
Discussion:
What are we measuring with checklists?
How do we set performance standards?
What remediation might follow?

21. Using Interpreters in Medical Encounters
Results of Pilot Study (UCI)
MS3 perception of encounter higher if self-identified as high Spanish fluency (α=0.80).
MS rated interpreter participation good to excellent 80% of the time.
SPs scored MS3 highly (80% items done, α=0.54).

22. Use if Interpreters in Medical Encounters
Results of pilot study (UCI, contd.):
SP’s gave higher scores to MS3 with lower Spanish proficiency.
Interpreters gave lower score to students with lower Spanish proficiency but reliability was low (α=0.25).

23. Assessing Medical Students
UCLA Results:
SP:
All PPI items are highly correlated with OS.
Overall Satisfaction is slightly correlated with HX012 (.18) HX013 and HX014 (.22).
discussed confidentiality.
asked what the patient thought might be going on OR what s/he thought might be causing the problem OR what s/he is most concerned about or afraid of.
asked how the patient has been coping with the problem and/or how it has affected his/her day-to-day activities.

24. Assessing Medical Students
UCLA Results: (Regression Analysis)
SP:
The strongest predictor of OS is PPI016 (accounted for 50% of variance). 
Combining with HX012, HX013 and PPI019, all 4 variables accounted for 55% of variance.
PPI016. Made the patient feel supported and taken care of (verbally and non-verbally; showed interest in me as a person, not just my condition; did not make me feel rushed).
PPI019. Explained things in a manner that could be understood (adapted language to accommodate both translator and patient; avoided jargon).
HX12. discussed confidentiality.
HX13. asked what the patient thought might be going on OR what s/he thought might be causing the problem OR what s/he is most concerned about or afraid of.

25. Assessing Medical Students
UCLA Results:
Sitting arrangement made no difference
HX05 rearranged the chairs to facilitate the interview ( translator sitting next to interviewer or behind interviewer’s shoulder, so patient can look at both simultaneously)
SPs gave higher scores to MS3 with lower Spanish proficiency.

27. Using Interpreters in Medical Encounters
What next?
Add faculty checklist
Create cases for different languages (Spanish vs. Vietnamese), i.e. generalizability.
Pre- and post test (MS2 vs MS3)
Validation and reduction of checklists

28. Use of Interpreters in Medical Encounters
What next (contd.)?
SP assessments in ‘real’ encounters (unannounced SPs)
Identifying students in difficulty for remediation.

29. Using Interpreters in Medical Encounters
Getting started (audience suggestions):
Which MS year?
What setting?
What performance measures?
What outcomes?
Consider:
Unannounced SP’s
Assess students ability to evaluate interpreter performance

30. Using Interpreters in Medical Encounters

31. Assessing Medical Students
Acknowledgements:
Grants (UCI and UCLA): NIH (NHLBI) grant K07 HL and Molina Health Care.
Contact:
Désirée Lie, MD, MSED
Paula Henderson, MD
THANK YOU!
QUESTIONS?