2.  Before a BCI can be used for control purposes, several training sessions are necessary ◦ Operant conditioning  Feed back, real-time changes to the user ◦ Machine learning  Adaptive algorithms to detect brain patterns

3.  A technology which allows a user to interact with a computer-simulated environment.

4.  VE (Virtual Environment) ◦ Allow users  to be shielded from the outside world  to be able to focus on the required mental task

5. Feedback  Visualization vs. VE feedback  Ron Angevin et al. (2004) ◦ Control group (standard BCI feedback) reacted faster ◦ VR group achieved less error

6.  Basic principle ◦ detection & classification of motor-imagery related EEG patterns  Sensorimotor rhythms are analyzed ◦ C3, Cz and Cz

7.  VE ◦ To let a user become immersed in a 3D scene  Cave ◦ Multiprojection stereo-based head-tracked VE system

8.  3D virtual environment 1.Creation of a 3D model of the scene  3D modeling Software Packages Performer  Maya 2.Generation of a VR-application that controls and animates the modeled scene  Virtual Research V8 HMD  640 X 480 pixels, refresh rate 60Hz  Vrjuggler + single back-projected wall + shutter glasses  Cave-like system

9.  BCI experiments ◦ Require a subject in a sitting position  No positional information had to be considered ◦ Rotational information from the tracking system was ignored  Rotation should be controlled by the BCI

10. Roation in a VE by Left- and Right-Hand Motor Imagery

11.  Imagination of left and right hand movement  Subjects ◦ 2 male(23, 26 years old), 1 female(28 years old) ◦ 7 months  Feedback conditions 1.A standard horizontal bar graph on a desktop monitor 2.Virtual conference room presented with an HMD 3.Virtual pub populated with animated avatars (including music and chatter of the avatars)

12.  The order of feedback conditions ◦ Bar graph → HMD → Cave → HMD → bar graph  Instruction ◦ To imagine left or right hand movements depending on an acoustic cue (single or double beep)  Control ◦ Either the length and the orientation of the horizontal bar graph (in case of the standard BCI feedback) ◦ Rotation angle and direction within VR

13.  During experiments ◦ Cue at second 3 ◦ Feedback for 4s ◦ Screen update (including rotation) 24 times/s ◦ One run 40 trials

15.  No difference between HMD and Cave  Performed well with VR than bar graph

16. Moving Forward in a Virtual Street by Foot Mortor Imagery

17.  Imagination in the virtual street ◦ Right hand movement: to stop ◦ Foot movement to move (constant speed) ◦ Walking distance is scored  CAM (Cumulative achieved mileage)  Male 23, 28 and 30 years old.

19. Scouting through a Virtual Apartment Asynchronous freeSpace Experiments

20.  Virtual apartment on a single back-projected stereoscopic wall  Subject could decide freely where to go along predefined pathways (through the corridors or rooms) ◦ Turn right, left, or straight)  System automatically guided the subject to the next junction ◦ Small map in the bottom right corner of the display

22.  Using VR ◦ High classification accuracy (low error rate) can be achieved.  Subjects felt more natural in VE compared with BCI experiments with standard feedback  Each subject preferred the Cave experiments to the HMD and both were favored over BCI session on a desktop PC  Motivation seems to improve the BCI performance, but too much excitement might also distract the subject  Despite distraction from auditory and moving visual stimuli in VE, motor imagery and its classification in the ongoing EEG is still possible

23. Thank you !