Immersive Virtual Characters for Educating Medical Communication Skills T. Bernard, C. Oxendine, D. S. Lind, P. Wagner Dept of Surgical Oncology (Medical College of Georgia) K. Johnsen, A. Raij, R. Dickerson, R. Wells, B. Lok Dept of Comp and Info Science and Eng (College of Eng) M. Cohen, A. Stevens Dept of Surgery (College of Med) J. Cendan, M. Duerson, R. Pauly Dept of Community Health and Family Med (College of Med) R. Ferdig College of Education

Previous Work: Technology for Medical Communication Skills Traditional approaches [Bearman 2003] –Narrative –Problem Solving Human Patient Simulator [Meurs 1997] –Mechanical simulation –Motors and actuators for I/O JUST VR [Ponder 2002] –Immersive approach (Stereo Projection) –Trains students to react to emergency situations –Uses human instructor to control system Virtual Standardized Patient [Hubal 2000] –Commercial Desktop application by RTI –Speech Recognition –Natural Language Engine

Project Description Simulate a standardized patient encounter Allow repeated interaction with an Immersive, interactive virtual patient in a constrained scenario Virtual patient, DIANA Virtual instructor, VIC To address issues with SPs –Experience diversity –Quality Control –Feedback Communication skills No physical diagnosis Interpersonal Simulator

Play Video Things to look for: interaction modalities

System Low Cost –< $8,000(USD) COTS Components Potential: –Every Hospital

Natural Interaction Input No Keyboard, No Mouse Speech Recognition –Dragon Naturally Speaking 8 Pro –Accuracy 90+% with 10 minutes training –70% match to database Track Communication Cues –Non-Verbal Track head gaze Track left hand Track body lean –Verbal InflectionJargon Gesture Recognition –Pointing, handshake

Natural Interaction Output DIANA and VIC look at user Life-size characters Animation –Hand gestures –Head movement Perspective-Correct Rendering Why this works –Does not rely on complete sentences –Constrained scenario –Students trained on specific questions

Eight Studies 2004 –April: Project initiated –August: Prototype (n=7) UF –October: Experts (n=3) UF –December: Pilot Test (n=10) UF 2005 –June: Two Institutions (n=16) UF/MCG –July: VP vs SP (n=16, n=8) UF/MCG –October: Cultural Bias (n=16) MCG –October: Class Integration (n=33) UF n = 101 Testing Centers –Harrell Center at UF –Medical College of Georgia –Diana was in Exam Room #3 (video)

Eight Studies

VP ≈? SP How is experiencing an interpersonal scenario with a virtual person similar to – and different from – experiencing an interpersonal scenario with a real person? Clearly different –But… in what important ways? Patient-doctor interview provides a constrained scenario The study asks: –Are post-encounter impressions similar? –Are empathy and other emotions and attitudes similarly expressed? –Which social constructs are followed? These questions must be explored to: –Determine the extent to which interpersonal scenarios can be simulated with virtual humans –Identify how component technologies need to improve to enable effective interpersonal virtual human systems

Overall Performance Similar Experiences –Same % of participants asked key questions –Out of 12: 6.3 ± 1.7, 5.5 ± 2.1 (α = 0.37) –Same % of participants passed the scenario –SP = 50%, VP = 36% But… –Some critical items were not asked at the same frequency –Sexual activity (SP = 0.88, VP = 0.44) –Nausea (SP=0.88, VP=0.25) –Related to virtual patient expressiveness

Behavioral Measures Empathy –Empathetic moment – “I’m scared, can you help me?” –Expressed the same % and same # of times (SP = 2.2, VP = 1.3) But… –Appears less genuine (very robotic) –Conversation flow is “rapid-fire” Confirmatory phrases statistically different (SP = 20, VP = 3.5)

Lessons Learned Overall experiences similar –Questions asked –Global measures Education goals met –Students rated educational merits similarly –Students rated difficulty similarly Global measures of realism do not work –Battery of specific measures more accurate Practice tool in addition to SPs Diana rated a 6.36(.85) on a 1 to 10 scale (Average is a 7.47 for real SP’s) VR works out of the lab

Papers Computer Science – –Raij,et al., “Interpersonal Scenarios: Virtual≈Real?”, IEEE Virtual Reality 2006 – –Johnsen, et al., “Evolving an Immersive Medical Communication Skills Trainer”, Journal on Presence: Teleoperators and Virtual Environments – –Dickerson, et al., “Virtual Patients: Assessment of Synthesized Versus Recorded Speech”, Medicine Meets Virtual Reality 14 – –Johnsen, et al., “Experiences in Using Immersive Virtual Characters to Educate Medical Communication Skills”, IEEE Virtual Reality 2005 – –Dickerson, et. al., “Evaluating a Script-Based Approach to Simulating Patient-Doctor Interaction”, SCS 2005 Int’l Conf on HCI Adv for Modeling and Sim Medicine – –Stevens, et al., “The Use of Virtual Patients to Teach Medical Students History Taking and Communication Skills“, American Journal of Surgery – –Lind and Lok, “The Role of Virtual Patients in Medical Education: Teaching Tool Versus Technological Trend”, FOCUS on Surgical Education – –Stevens, et al., “Implementing a Virtual Patient into the Medical School Curriculum at the University of Florida”, Southern Group on Education Affairs 2006 (pres.) – –Cohen, et al., “Do Health Professions Students Respond Empathetically to a Virtual Patient?”, Southern Group on Education Affairs 2006 (pres.) – –Bernard, et al. “A Multiinstitutional Pilot Study to Evaluate the Use of Virtual Patients to Teach Health Professions Students History-Taking and Communication Skills”, Society of Medical Simulation 2006 – –Stevens, et al., “The Use of Virtual Patients to Teach Medical Communication Skills”, Assoc of Surg Educ 2005, Southern Group on Educational Affairs 2005 (pres.)

Current Work Incorporation into the classroom UF Essentials of Patient Care –n=34 students this year –n=135 next year MCG Patient Communication –n=200 each year Study –Real Speech vs. Synethsized Speech –Immersive vs. Non-immersive

Do people carry real world biases into the virtual world?

The Use of a Virtual Scenario to Teach Communication Skills for Geriatric Patients

Patient Diversity Do students treat different virtual patients similarly? Measures Measures Eye gaze Body Lean Vocal Inflection TimeInterruptions

Communication Skills Fundamental to clinical practice. Patient & physician satisfaction. Patient understanding & adherence. Healthcare outcomes. Malpractice litigation. Learned skill & experiential learning alone is insufficient.

Virtual Objective Structured Exam (VOSCE) NBME

Curriculum integration UF 1 st year Medical Students n = 135 MCG 1 st year Medical Students n = 200 Virtual Patient Experience (Early 1 st semester) EPC courses (Semesters 2-4) ECM courses (Semesters 2-4) EPC courses (Semester 1) ECM courses (Semester 1)

Virtual Patient Teams T. Bernard, C. Oxendine, D. S. Lind, P. Wagner Dept of Surgical Oncology (Medical College of Georgia) K. Johnsen, A. Raij, R. Dickerson, R. Wells, B. Lok Dept of Comp and Info Science and Eng (College of Eng) M. Cohen, A. Stevens Dept of Surgery (College of Med) J. Cendan, M. Duerson, R. Pauly Dept of Community Health and Family Med (College of Med) R. Ferdig College of Education