Annemarie Kokosy and Matthew Pepper ISEN / EKHUFT April 24 th, 2012 University of Essex 1 Part-financed by the European Regional Development Fund Definition of the second scenario for the clinical evaluations

Part-financed by the European Regional Development Fund Plan of presentation I. Why we must define and agree the scenario? II. Project plan III. Clinical goals IV. Future challenges and the work to be done V. Main goals 2

Part-financed by the European Regional Development Fund Why we must define and agree the scenario? For a collaborative cross border work, it is necessary to have a common objective For a more effective work accelerating the implementation of the algorithms on the powered wheelchair from ISEN and University of Kent, we must define the inputs and outputs of each partner We must take into account the user’s needs expressed through the European survey (end date: April 30 th for the French side) We must work in collaboration with the medical staff from EKHUFT and GHICL 3

Part-financed by the European Regional Development Fund Why we must define and agree the scenario? At the end of the project we must have implemented on the powered wheelchair from ISEN or University of Kent the results from each activity 4

Part-financed by the European Regional Development Fund Project plan – Time line 55 Feb/2012Aug/2012Feb/2013Aug/ Dec/2013 End 1st scenario (collision avoidance) Technical work 4/126/1210/1212/124/136/1310/1312/13 Clinical trials 2nd scenario (assisted navigation in semi-autonomous way) 3rd scenario (autonomous navigation) Technical work Clinical trials Technical work Clinical trials Ethics Approval April 24 th, 2012

Part-financed by the European Regional Development Fund Clinical goals Second scenario – Assisted/Semi-autonomous Navigation – First evaluation: environment without obstacles - pass along a corridor and through a doorway and stop in front of a table/desk (user in the loop); – Second evaluation: environment with obstacles - pass along a corridor and through a doorway and stop in front of a table/desk (user in the loop); Evaluations at University of Kent and EKHUFT with healthy people Clinical evaluations at GHICL The project will be a success if this is achieved 66

Part-financed by the European Regional Development Fund Future challenges and the work to be done Step 1: Develop the hardware and software for the detection of the doors (minimal dimension of the door is 90cm) and tables ISEN: system with IR and US (works only if the wheelchair is in front of door) UoK - Michel: will try to detect by using a camera (to be developed) Step 2: Two switches: one to turn on/off the interface device (called SYSIASS) and the second to turn on/off the wheelchair 77

Part-financed by the European Regional Development Fund Future challenges and the work to be done Step 3: Develop software for assisted navigation – along a corridor and through a doorway ISEN: potential field (in progress) UoK - Michel: “localised potential field” (to be developed) Step 4: Develop software to store, analyse and present that data. Data to store: what the user doing, what the device doing, sensors, time – What user doing: direction and amplitude – What the device doing: modified amplitude, direction – All sensors: to be done – Time: to be done 88

Part-financed by the European Regional Development Fund Future challenges and the work to be done Step 5: Visual feedback (how the device is intervening) or voice or vibrations Jean Marc: visual feedback and vibration Michel: the voice approach (especially the sentences) if he wants Step 6: Secure Data Transmition (John, Gareth, Jenya, Klaus) Step 7: Drive the wheelchair using the multi-modal HMI (Huosheng, Lei Wei, Dongbing Gu, Theo and Ericka Rechy- Ramirez) 99

Part-financed by the European Regional Development Fund Future challenges and the work to be done Work within the requirements of the Medical Devices Directive - MDD 93/42/EEC [rev 2007] Get permission from University of Kent for evaluations with healthy people Clinical evaluations at GHICL (the Hospital of Garches has not funds) Sign a licence agreement with DynamicControl Wheelchair Users Questionnaire for the English side 10

Part-financed by the European Regional Development Fund Future challenges and the work to be done Project strategy ( decision on March 29 th, 2012 – University of Kent ) 11 Joystick DynamicControl electronics CAN-DX BUS Joystick board Wheelchair “intelligence” board Sensors Analog signals Digital signals Will be replaced by Serial GPSB Module from DynamicControl 11

Part-financed by the European Regional Development Fund Future challenges and the work to be done Project strategy ( decision on March 29 th, 2012 – University of Kent ) 12 Joystick DynamicControl electronics CAN-DX BUS Serial GPSB Module Sensors board Wheelchair “intelligence board” (PC or mother board) Sensors

Part-financed by the European Regional Development Fund Future challenges and the work to be done Project strategy ( decision on March 29 th, 2012 – University of Kent ) 13 Multi-modal HMI DynamicControl electronics CAN-DX BUS Serial GPSB Module Sensors board Wheelchair “intelligence board” (PC or mother board) Sensors

Part-financed by the European Regional Development Fund Main goals 14 Activity 1 Localization and perception  using monocular camera/Kinect/odometers/US/IR System modeling and identification  using the dynamic model of powered wheelchair Navigation  using A*/potential field/nonlinear dynamic optimization Robust control  using sliding mode

Part-financed by the European Regional Development Fund Main goals 15 Activity 2 Development of ICMetrics boards  using PIC or Arduino & ARM Development of an experimental platform for feature collection and analysis Design of a flexible and secure channel  using Shamir secret sharing technology in conjunction with the ICmetric system Activity 3 Development of a multi-modality HMI  using facial muscle signals (EMG) and face image information

Part-financed by the European Regional Development Fund Main goals 16 Activity 3: powered wheelchair prototype  clinical trials of the first scenario at the Hospital of Garches  second powered wheelchair equipped with US/IR/Arduino at ISEN  third powered wheelchair equipped with PC/IR/US at University of Kent  own powered wheelchair equipped with PC/camera/sensors at University of Essex