Moving Beyond Single User, Local Virtual Environments for Rehabilitation Patrice L. (Tamar) Weiss Department of Occupational Therapy Faculty of Social Welfare and Health Sciences University of Haifa, Israel Evelyne Klinger Arts & Métiers Paris Tech Angers Team Handicaps et Innovations Technologiques P&I Lab, Laval, France

Applications of Virtual Reality

Virtual Reality Assets for Rehabilitation and Special Education School of Occupational Therapy, Hebrew University 2. Chaim Sheba Medical Center 3. Dept. of Occupational Therapy, University of Haifa Ecological validity Grade cognitive & motor demands Convenient times & locations Performance can be documented Easy to test & change Safe for user

Virtual Reality Continuum for Rehabilitation Desktop Video Capture Head Mounted Displays Fully Immersive – CAVE - CAREN Presence Cost Complexity High Tech Low Tech

Example VAP-S: Virtual Action Planning Supermarket to assess and treat executive function deficits

Traditional Executive Function Assessment Standard neurocognitive tests: –Lack of sensitivity –Far away from everyday life situations (Godefroy et al., 2004) Ecological neurocognitive tests: –Real time, Real situation (MET) –Limited use for patients without autonomy (Shallice & Burgess, 1991) Test of scripts : –Plan generation –No action (Abbott et al., 1985) WCST

VAP-S: Virtual Action Planning Supermarket Designed with two main software: 3DStudioMax and Virtools Klinger E., Marié R.M. et al., , GREYC-ENSICAEN, CHU de Caen

Shopping Task Verbal information: –In the supermarket, you should buy: one baguette green apples 2 kg bag of detergent one kilo of flour one t-shirt for child two artichokes beige socks. –You may pay by clicking on the purse icon that is on the screen. List display on the screen Marié et al., 2003

The user –Selects items by using the mouse The items Appear in the cart Disappear from the icon list Record of trajectory, time and actions Assessment Session

Performance comparison Control : Duration : 8 min Distance : 215 m Stops : 25 Patient with PD : Duration : 25 min Distance : 469 m Stops : 67 Klinger et al., 2006

Example IREX for Physical Fitness of young adults with intellectual disabilities

Objective To test the effectiveness of a VR-based exercise program in improving the physical fitness of adults with IDD.

Methods Research group (N=30) mean age = 52.3 ± 5.8 years IDD level - moderate) Matched control group (no VR-based exercise) Two sub-groups: ambulatory and wheelchair users Fitness program 5-6 weeks two 30 min sessions per week game-like exercises provided by the IREX or Sony PlayStation II EyeToy video capture VR system Pre & post-intervention changes in physical fitness Energy Expenditure Index (EEI) modified 12 min walk\run Cooper test Total Heart Bit Index (THBI)

Study Participants VariablesResearch (n=28) Control (n=31) Demographics Mean age (yr) Age range (yr) Males1615 Females1216 Mean weight (kg) Mean height (m) Mean rest pulse (bpm) Mean activity pulse Assistive technology Wheelchair1215 Rolator1314 Crutches22 Walking cane10 Facility Quitmann1312 Kfar Nahman156 Ruhama----6 Bare Dror----9

Pre-Post Intervention Tests GroupNAmbulation style Cooper test (m) Mean ± (SD) Energy Expenditure Index (EEI) Mean Total heart Beat Index (HB/m) Mean PrePostPrePostPrePost Research 28All *** * 16Walking Aid ** Wheelchair ** Control 31All Walking Aid Wheelchair P value < *P value < **P value <0.001*** Link to video clip for slide 15

Tele-rehabilitation Second Life StoryTable, NNR-Table

DiamondTouch Interactive Table Mitsubishi Electronic Research Lab (MERL) Large horizontal interactive surface Table-top interaction modality Operated through PC using Flash programming Images are top projected 18

19 DiamondTouch Interactive Table Multi User Actions (MUAs): Interaction modality that requires the simultaneous participation of two or more users

Story Table –Interface to support pairs of children in the activity of collaborative storytelling –Implements “Enforced Collaboration” paradigm 20 Alternate story backgrounds Story characters Voice recording Lady bugs to hold audio snippets Story sequence

Story Table: Initial Study 21

OUTCOME MEASURES 1. Behavioral Checklist (based on Bauminger, Aviezer & Rogers, 2004) positive social interactions (e.g., look at peer with positive affect) negative social interactions (e.g., ignoring action of peer) autistic behaviors (e.g. perseveration) 2. Analysis of language usage in the interactions and in the narrations 22 Story Table: Initial Study

MarbleWorks Free structure play construction game Pre- and Post-tests Low-Tech ST scenarios with cardboard figures Story Table: Initial Study Link to video clip for slide 23

Comparison of Pre and Post outcomes increase in key positive social interactions decrease in negative interactions increase in narrative and play ability Story Table: Initial Study

Tele-Rehabilitation Patient Home Unit Central System (phase II) Central DB (phase II) Therapist / Clinician Unit Link to video clip for slide 27b Link to video clip for slide 27a

Second Life is an Internet-based virtual world launched in 2003 by Linden’s Lab (  “Residents” inhabit virtual worlds and interact via avatars  Users participate in individual and group activities, and to create and trade items and services using Linden dollars  By the end of March 2008, 13 million registered accounts. About 38,000 residents are logged on to Second Life at any particular moment.  number of medical & health educational projects:  Nutrition Game  Occupational Therapy at the Virtual Neurological Education Centre  Brigadoon designed for people with Asperger’s syndrome. Brigadoon - a controlled environment where users are encouraged to feel comfortable and learn socialization skills at their own pace (Lester).

Issues For Future Consideration User perspective - effectiveness of providing users with first, third or bird's eye perspectives. In past such decisions based primarily on the technology selected to render VE. In the future, should be also driven by educational or therapeutic needs of users.

Issues For Future Consideration User perspective Role of virtual presence – generally assumed that increasing the level of virtual presence helps to facilitate the achievement of therapeutic goals due to its impact on motivation and performance. Important to establish the role of virtual presence in multi-user, remote location VEs due to the added difficulty in achieving it in such settings.

Issues For Future Consideration User perspective Role of virtual presence Technology considerations – Access to remote locations, especially in real-time, adds additional cost and technical complexity to the design and implementation of VEs. Considerations of increased band width and the use of sensors capable of transmitting high fidelity data must be taken into account.

Issues For Future Consideration User perspective Role of virtual presence Technology considerations Compliance – user’s motivation to be a willing partner in any educational process of great importance. One of VRs major assets has been the use of game-like environments to increase motivation, participation and performance. Will this be lost in remote settings?

Issues For Future Consideration User perspective Role of virtual presence Technology considerations Compliance Ethical considerations – Use of VEs in the traditional single user, local setting retained all elements of privacy that were guarded during conventional rehabilitation. The addition of other users and the transmission of data, images, and communication over the Internet clearly introduce ethical issues not previously considered.

Issues For Future Consideration User perspective Role of virtual presence Technology considerations Compliance Ethical considerations Availability of software supporting functional VEs – most functional VEs have been customized by specific research groups, and often unavailable to other educators and clinicians. Recently publicized NeuroVR initiative ( provides a cost- free VE editor, which allows non-expert users to easily setup and tune VEswww.neurovr.org