Brain-Computer Interfaces for Communication in Paralysis: A Clinical Experimental Approach By Adil Mehmood Khan

Contents ● Introduction ● BCI for communication in paralysis ● BCI software ● Self-regulation of SCPs and training ● Spelling through brain-cmputer communication

Brain-Computer-Interface (BCI) ? “A system for controlling a device e.g. computer, wheelchair or a neuroprothesis by human intention which does not depend on the brain’s normal output pathways of peripheral nerves and muscles” [Wolpaw et al., 2002]. HCI – Human Computer Interface DBI – Direct Brain Interface (University of Michigan) TTD – Thought Translation Device (University of Tübingen)

BCI Some examples of BCI applications Leeb et al., Computational Intelligence and Neuroscience, 2007 (doi:10.1155/2007/79642)

Brain Computer Interfaces ● Allow patients to control a computer by concious changes of brain activity ● Provide a means of communication to completeley paralysed patients: amyotrophic lateral sclerosis (ALS), cerebral palsy, locked in syndrome ● Can be used to control a cursor, select symbols, control external devices like orthesis / prothesis (depending on type of BCI) ● Have a very low data rate, typical a few bit per second or less ● First results in the 1970ies (Vidal, visual evoked potentials, VEP-BCI)

Brain Computer Interfaces Principles of operation:

Brain Computer Interfaces – Electrophysilogical Activities used ● SCP Slow Cortical Potentials ● Mu Movement Imagination ● P300, SSVEP ERP-Analysis ● cortical neurons, direct brain interfaces The control information is extracted from the real time EEG-recording http://www.wired.com/news/images/full/thoughtlock1_f.jpg

Brain Computer Interfaces – SSVEP ● Steady State Visual Evoked Potentials derived from the visual (occipital) cortex ● focussing attention to visual stimuli of different frequency shows up in the EEG freqeuncy bands ● relibable and high transfer rate, but some prerequisites (eyes) http://www.iua.upf.es/activitats/ semirec/semi-Reilly/

Brain Computer Interfaces – SCP BCIs ● detection of slow cortical potentials (SCPs) ● needs DC EEG Amplifiers ● first successful device end 1990‘s: Niels Birbaumers Thought translation device intensive training was necessary to gain control over the SCP waves SCPs: DC-shifts, slow negativation of cortical areas Preparation of movement and cognitive tasks, Several hundert milliseconds before the task Patinet using TTD to write a letter http://www.heise.de/ct/06/18/088/bild1.jpg

Brain Computer Interfaces - μ-rhythm BCIs ● μ–rhythm is the idle-rhythm of the motor cortex ● frequencies around 10 and 18 Hz. ● ERD / ERS – event related desynchronisation / synchronisation movements or imagination of movements inhibit the μ–rhythm Berlin-BCI, http://www.fraunhofer.de/

Brain Computer Interfaces - P300 BCIs ● P300 wave – posivite component in the event related potential, 300ms after a stimulus ● natural response to events considered as important ● selection of a symbol: count the flashes, algorithm averages trails and finds a P300 P300 runtime user interface

Brain Computer Interfaces - μ / P300 comparison μ - BCIs P300 BCIs Require training do not require training 2d-control possible 1D control only movement imagination concentration / decision affected by movement affected by distraction

BCI for Communication in Paralysis An overview of different approaches to BCIs developed at Institute of Medical Psychology and Behavioral Neurobiology. Thought-Translation Device (TTD). Brain-Controlled Web Browser. Visual and Auditory feedback modes. Oscillatory Features based Classification.

SCPs: A Brief History An initial application: Epileptic Seizures Down regulation of brain potentials towards a positive amplitude. BCI for “locked-in syndrome” Communication through self-regulation of SCPs Also known as TTD

BCI Software Thought Translation Device: Components EEG Amplifier EEG8 system G.tec amplifiers BrainAmp system Two Monitors One for operator (supervise the training) One for patient (feedback) Sampling frequency: 256Hz Digitized with 16 bits/sample Amplitude range: +(-) 1mV Low frequency cutoff: 0.01 Hz High frequency cutoff:: 40-70 Hz

TTD Feedback and Communication System The current version of TTD software is derived from BCI2000 standard

TTD Feedback and Communication System

TTD Software Data acquisition and storage Online signal processing Classification Feedback and application interface

Brain Computer Interfaces - BCI2000 ● Research Platform for BCI Systems ● Written by Gerwin Schalk, Wadsworth Center, Albany (NY) ● Modular structure: Signal Aquisition, Signal Processing and User Application communicatie via TCP/IP ● Operator module used for configuration of the other modules ● various user tasks availbale: 1D/2D cursor, Speller, P300, SCP ● free for academic use ● driver for OpenEEG available http://www.bci2000.org/

BCI2000: Components Filters Spatial, temporal, and spectral Online artifact detection and correction Classification Linear Discriminant Analysis (LDA) Simple Threshold Classification Support Vector Machine (SVM) MATLAB interfaces.

Self-Regulation of SCPs ● Slow event-related direct-current shifts of the EEG. ● They last from 0.3 seconds up to several seconds. ● Occur as a result of external or internal events. ● Negative shift is related to excitability of neurons. ● Positive shifts are measured during the execution of cognitive tasks ● Healthy subjects as well as patients can learn to produce positive or negative SCPs ● Training requires feedback Visual Auditory

Self-Regulation of SCPs ● Recording site for feedback signal is usually Cz. ● EEG is usually recorded from 3-7 Ag/AgCI-electrodes placed at Cz, C3, C4, Fz, and Pz. ● vEOG is recorded using a bipolar channel for online and offline artifact correction. ● A fixed percentage of vEOG signal is subtracted from the SCP signal at Cz for EOG correction. ● SCPs are calculated by applying a 500ms moving average to EEG signal.

Self-Regulation of SCPs ● With the visual feedback modality: Subjects viewed the course of their SCPs as the vertical movement of feedback cursor. Vertical cursor movement corresponded to the SCP amplitude. Task was to move the cursor towards the modality indicated by a red rectangle.

Self-Regulation of SCPs: Training Process Target Presentation interval Selection interval Response interval

Self-Regulation of SCPs: Training Process Target Presentation interval Selection interval Response interval ● First 2-4 sec of the trial. ● Target is illuminated in red. ● Allows the subject to prepare for the corresponding SCP regulation.

Self-Regulation of SCPs: Training Process Target Presentation interval Selection interval Response interval ● Feedback is provided by the vertical position of the steady horizontally moving cursor. ● Cortical negativity moved the cursor up. ● Positivity moved the cursor down. ● Center of the screen corresponded to the baseline. ● Task is to move the cursor to the red area.

Self-Regulation of SCPs: Training Process Target Presentation interval Selection interval Response interval ● A response is classified as correct if: Average potential carried the correct polarity. Or is inside the target boundaries of the required goal. ● Classification methods, such as LDA or SVM can be used for improvement of the correct response rate.

Self-Regulation of SCPs: Performance 1st Trial: Target presentation , Selection, Response 2nd Trial: Target presentation , Selection, Response NthTrial: Target presentation , Selection, Response ● Performance Percentage of correct responses in valid trials. After a rate of 75% correct responses, subjects were trained to select letters and write messages ● Subjects typically reach this level after 1 to 5 months of training, with 1 to 2 training days per week. ● A training day comprises 7 to 12 runs, and a run comprises 70 to 100 trials.

Applications Spelling by Brain-Computer Communication: A program driven by “yes” or “no” responses, which serve as “select” or “reject” commands. Requires three intervals in one trial. Allows user to select letters from a language alphabet and to combine letters into words and sentences. Presentation Selection Response

Applications Spelling by Brain-Computer Communication: A program driven by “yes” or “no” responses, which serve as “select” or “reject” commands. Requires three intervals in one trial. Allows user to select letters from a language alphabet and to combine letters into words and sentences. Presentation Selection Response Presentation of the letter set. Displayed in target rectangles on the screen.

Applications Spelling by Brain-Computer Communication: A program driven by “yes” or “no” responses, which serve as “select” or “reject” commands. Requires three intervals in one trial. Allows user to select letters from a language alphabet and to combine letters into words and sentences. Presentation Selection Response Feedback is provided. Self regulation of SCP amplitudes is used to select or reject the letter set.

Applications Spelling by Brain-Computer Communication: A program driven by “yes” or “no” responses, which serve as “select” or “reject” commands. Requires three intervals in one trial. Allows user to select letters from a language alphabet and to combine letters into words and sentences. Presentation Selection Response Response interval indicating to user the result of the selection. Error correction is done using a “go-back” option.

Spelling by Brain-Computer Communication

Spelling by Brain-Computer Communication Performance: Writing the most conveniently situated letter , “E,” takes 5 trials. Writing the most remote sign requires 9 trials, i.e. 36 – 45 sec. Improvement A simple personal dictionary to make free spelling less time consuming. Contains words that are frequently used by patients. A complete word is suggested after at least two letters have been written. This word can then be chosen with a single selection response.