1. Virtual Experiences Research Group PhD Students –Kyle Johnsen, Aaron Kotranza, John Quarels, Andrew Raij, Xiyong Wang, Brent Rossen Undergraduates –Joshua Horton, Harold Rodriguez Funding –National Science Foundation (CAREER, REU), University of Florida Colleges of Engineering and Medicine, Medical College of Georgia, Keele University, School of Pharmacy

2. Graduating Students Kyle Johnsen –http://www.cise.ufl.edu/~kjo hnsen Andrew Raij –http://www.cise.ufl.edu/~raijhttp://www.cise.ufl.edu/~raij Harold Rodriguez –Undergraduate

3. Immersive Virtual Humans for Educating Medical and Pharmacy Communication Skills K. Johnsen, A. Raij, B. Rossen, A. Kotranza, X. Wang, B. Lok Computer and Information Science and Engineering M. Cohen, A. Stevens Surgery J. Cendan Community Health and Family Medicine R. Ferdig Education A. Deladisma, D. S. Lind Surgical Oncology S. Chapman, L. Bracegirdle Pharmacy

4. Can Virtual Humans Enable…

5. VOSCE Project Overview Started Spring ’04 n > 250 students –Medical –Nursing –Physician Assistant –Pharmacy One of the most popular VH/VR experiences Three institutes –University of Florida –Medical College of Georgia –Keele University (U.K.) Team –VR/HCI/CS – 1 PhD, 6 grad students, 2 undergrads –Medicine – 6 MDs, 2 medical students –Education – 2 PhDs Research focus on interfaces

6. Play Video

7. Why Virtual Humans? Students –Repetition –Feedback –Longitudinal learning Educators –Standardization –Dynamic Abnormal findings Cultural competency –Aggregate performance Researchers –Study the extent of impact of VHs –Easy to run studies (Twiddle one thing)

8. What can a Virtual Human do? Respond to a sneeze Show empathy “I’m scared, can you help me?” “Could this be cancer?” Sneezing Evoke emotion (e.g. anxiety) Physiological effects Social conventions Similar to a human So what happens with VH’s of different backgrounds?

9. VHs and Bias VHs elicit racial/ethnic bias Working to mitigate effects of bias

10. Mixed Reality Humans Virtual humans have limitations –Open research problems (e.g. AI, speech) –No tactile feedback Merge the real and virtual spaces –Real tools –Real simulators Complete patient interaction Physical Exams (e.g. eye exam) –Point at eye chart –Ophthalmoscope –Follow my finger –1 or 2 fingers?

11. Breast Simulator Integration Breast simulator integrated –Dr. Carla Pugh, Northwestern –Student does a patient history –Asks to remove gown (physiological measures) –Performs a breast exam –VP winces at too much pressure Future work –Pelvic simulator –Central line simulator [Kotranza (submitted) VR2008]

12. Potential Applications for VHs Training Interpersonal Scenarios Employee Training –Social situations Sexual Harassment Cultural Sensitivity/Competency –Business Dealings Patron diversity (physical, emotional, mental) Park Visitor Experience Enhancement –Kiosks for patrons to interact with their own culturally knowledgeable guide –Personalized “ride” that allows patrons to have immersive experiences

13. IPS System Inputs –Natural speech –Tracking data (head, hands, chair, tools) –Video –Physiological measures Outputs –Speech and animation –Life-size projection (or HMD) –Perspective correct rendering –Reactive virtual human COTS components –2 PCs –2-4 video cameras –Data projector or HMD –Wireless microphone –Bodymedia Sensewear –< $10,000(USD) Potential: –Every Hospital

14. Bias Study (n=9), people reported that the African-American VH had less education and money Identical: animations, words spoken Differences: skin-tone, voice Other biases to study –Race/Ethnicity –Gender –Age –Weight –Social ‘hot topics’ Student interaction with VH 1 Student interaction with VH n Behavior Cue Analysis After-Action Visualization Δ? Studying diversity issues with VHs

15. Abnormal Findings Conditions difficult to represent with existing education methods Conditions –Psychomotor –Neurological –Age/race/ethnicity dependent conditions Blurry vision scenario –Cranial Nerve III (due to brain tumor) –Corneal ulcer –Retinal detachment Benefits –Curricular planning of medical student exposure –Supplement SP experiences –Leverage the dynamic nature of VPs [Wang ISMAR 2007, submitted VR 2008] Play Video

16. Current Work Classroom incorporation –Communication course 2 nd year MS Surgical rotation 3 rd year MS Pharmacology 1 st year –n>120 per year –Potentially 30,000 interactions a year @ existing institutions Virtual Instructor Real-time response to tracked cues –Physiological measures –Posture cues –Verbal cues Level of Interactivity Normal Abnormal Standardized Patients Virtual Patients Physical Simulators (Harvey, HPS, etc.) Patient Condition Textbooks and Journal Articles

17. Scenarios and VPs Acute abdominal pain Breast mass Dyspepsia Sexual history Eye exam Patients –DIgital ANimated Avatar (DIANA) –Elderly Diana (Edna) –Manniquin Diana (Mandi) Building the following VPs –Male –Personality (e.g. irate) –Intelligence (e.g. mentally retarded) –Appearance (e.g. disfigurement, limbs, burns)

18. Join Us! Application domain outside medicine –Additional locations to install and test system Support –Research directions –Students Thank you! http://www.cise.ufl.edu/research/vegroup Questions?

19. Virtual Experiences Research Group PhD Students –Kyle Johnsen, Aaron Kotranza, John Quarels, Andrew Raij, Xiyong Wang, Brent Rossen Undergraduates –Joshua Horton, Harold Rodriguez Funding –National Science Foundation (CAREER, REU), University of Florida Colleges of Engineering and Medicine, Medical College of Georgia, Keele University, School of Pharmacy

20. After Action Review of VH Interactions Taking a cue from flight simulators End-Users –Self-reflection –Understanding –Feedback Educators –Quantitative analysis –Identify trends and outliers Researchers –Capturing H-VH interaction –Analysis Process, Filter, Visualize [Raij (submission) VR 2008, SIGCHI 2008]