Playful Stimulation against Parkinson’s Disease - a tentative exploration Final presentation, May 29, 2015 Adrian Hänni and Padideh Pezeshki University of Fribourg

Outline Introduction Dandelion Game Rules Types of Modalities used MYO Sphinx 4 Interfaces Commands CASE / CARE Architecture Video Conclusion Questions

Introduction Parkinson’s disease (PD) Exergames Around 6.3 million people worldwide Caused by the lack or reduced levels of dopamine in brain First affect one side, then develop to both sides of body, dominant on primary side Primary sign of PD is tremor, mostly on hands and fingers Current treatments are medication, physiotherapy, and surgery Exergames Developing exergames for health care is currently a hot topic in the field of virtual rehabilitation (VRH) We aimed to develop one for VRH of PD patients using multimodal interfaces

Dandelion Exergame prototype Game rules The character is a dandelion hovering in the space Collect blue flowers / avoid thorn flowers Game rules Play time 60s Collecting blue flowers Adds 10 points Increases play time by 2s An additional bonus score is added and increases by 1 Thorn flower Subtracts 10 points Decreases play time by 5s Resets additional bonus score to 0

Types of Modalities used Gesture Use of gestures intends to animate patients suffering PD to perform physiotherapy exercises Hardware: MYO gesture recognition device Software: MYO Connect Speech Voice recognition as supportive alternative when certain movements cannot be performed Microphone Software: Sphinx 4 running on JAVA 1.7 VM

MYO Features Electromyographic (EMG) sensors Accelerometer (Vector of Units of g) Gyroscope (Vector of Degrees / Sec) Orientation (Roll / Pitch / Yaw as Quaternion) Requires calibration of world orientation Drawbacks Warm-up time (2min) Requires every time new calibration profile for best results

Sphinx 4 Pure Java speech recognition library for speech recognition, to identify speakers, adapt models, and transcription Dictionary Grapheme to phoneme (G2P) conversion exit EH K Z IH T exit(2) EH G Z IH T exit(3) EH K S AH T Acoustic model Provided by Sphinx data package Set to “en-us” Grammar Written manually in JSpeech Grammar Format (JSGF) format public <WORDS> = ( up | down | left | right | score | help | exit | etc. ) ;

Interfaces Interface A Interface B MYO: Direction Voice: Speed Level MYO: Speed Level Voice: Direction Both interfaces use Voice and/or electromyographic gestures for the game’s menu

Commands Speech Gesture Menu Screens START / HELP / SCORE / EXIT / NEXT Game Screen FASTER / SLOWER / UP / DOWN / LEFT / RIGHT SPEED <LEVEL> <LEVEL> = FAST / MEDIUM / SLOW Gesture WAVE IN / WAVE OUT / FIST / FINGER SPREAD / DOUBLE TAP <Arm direction> = UP / DOWN / LEFT / RIGHT / REST POSITION <Arm direction> = Q<transition> = Q<rotation> * Q<rest position>^-1

CASE / CARE CASE CARE Exclusive Commands are exclusive and happen one after another in the order performed CARE Assignment Only FINGER SPREAD pose can set the MYO rest position Character control Depending on interface type, speed and direction can be performed only by one modality Equivalence Choice between Speech and Gesture in the Menu screens

Architecture

Video

Conclusion Absence of modality fusion is intended Game is aimed to be used by patients suffering Parkinson disease Different health conditions increase the importance of alternatives, respectively equivalence Focus on providing equivalent inputs of different modalities, as provided by the interfaces can be considered to be beneficial Evaluation is pending

Questions