BCI Systems Brendan Allison, Ph.D. Institute for Automation University of Bremen 6 November, 2008

2 A BCI enables communication without movement. Some patients cannot use any interface requiring movement. What is a BCI?

3

4 Most BCIs translate your brain’s electrical activity (EEGs) into messages or commands. Performing mental tasks produces electrical activity detectable with electrode caps. What is a BCI?

5 BCIs may be: Non-invasive (usually EEG) Invasive ECoG (surface of cortex) depth recording (in brain) What is a BCI?

6 When a neuron is active, its voltage may change by 100 mV or more. Electrical activity in a single neuron. How do EEGs work? Neural communication produces electrical activity.

7 This activity may be detectable to electrodes on the scalp. How do EEGs work? Conventional electrode caps from EGI, Neuroscan, and Electro-Cap.

8 Newer EEG recording systems: Require less or no prep time and skill Require less or no gel Require fewer electrodes Are more portable Handle artifacts better Are wireless Are cheaper How do EEGs work? Field recording systems from Quasar, Advanced Brain Monitoring, and Pineda et al (2003).

9 QUASAR Hybrid or eIBE sensors Audio Headset EOG/EEG Glasses QUASAR IBE Electrodes How do EEGs work?

10 Emotiv and NeuroSky systems How do EEGs work?

11 Invasive BCIs record activity from electrodes under the scalp. Pyramidal neurons (Kandel et al., 1988) Electrocorticogram (ECoG) recording montage in a human patient. (Leuthardt et al., 2004) How do EEGs work?

12 Other functional imaging approaches: Functional Magnetic Resonance Imaging (fMRI) Magnetoencephalography (MEG) Positron Emission Tomography (PET) Functional Near Infrared (fNIR) How do EEGs work? A functional MRI An MEG machine Functional Near Infrared

13 BCIs cannot read minds or literally interpret mental activity. hello yes pain What isn’t a BCI?

14 BCIs are a popular topic in sci fi! The Matrices FirefoxStrange DaysNeuromancer Johnny Mnemonic Star Trek: The Cage Dreamscape The X – Mens What isn’t a BCI?

15 More bci fi …. Total RecallMinority Report Brainstorm The Lawnmower Men The Govenator!! What isn’t a BCI?

16 BCIs cannot: Read your thoughts. What isn’t a BCI?

17 BCIs cannot: Operate without your knowledge or free will. What isn’t a BCI?

18 BCIs cannot: Write to the brain. What isn’t a BCI?

19 BCIs cannot: Repair injured areas. (??) What isn’t a BCI?

20 These similar approaches are not BCIs: ?? Biofeedback Prosthetics Retinal or cochlear implants Medical EEGs EEG or fMRI Lie Detection Neuromarketing Employee screening Attention or fatigue monitors What isn’t a BCI?

21 BCIs rely on voluntary mental activities such as: Imagined movement What is a BCI? Torrey Pines golf course

22 BCIs rely on voluntary mental activities such as: Emotional imagery What is a BCI?

23 BCIs rely on voluntary mental activities such as: Selective attention (SSVEP) What is a BCI?

24 BCIs rely on voluntary mental activities such as: Selective attention (P300) What is a BCI?

25 BCIs rely on voluntary mental activities such as: Certain mental tasks 6 X 9 42 Object rotation Math Singing What is a BCI?

26 What is a BCI? Emerging new tasks for BCIs: Perceived error Imagined music New modalities and variants (Passive BCIs): Anticipation Alertness/fatigue Familiarity/recognition

27 How do BCIs work? oGeneral Schematic oP300 BCI oMu BCI oOther BCIs Components

28 A BCI requires the following: -At least 2 electrodes -An amplifier designed for EEGs -A mediocre personal computer -An A/D card in the computer -Software Components

29 All BCIs have at least four components: 1)Signal Acquisition 2)Feature Extraction 3)Translation Algorithm 4)Operating Environment Components The Four BCI Components (Wolpaw et al., 2002; Allison et al., 2007)

30 Acquisition: The user performs a task that produces a distinct EEG signature for that BCI Extraction: Salient features are extracted from the EEG Translation: A pattern classification system uses these EEG features to determine which task the user performed Environment: The BCI presents feedback to the user, and forms a message or command Components

31 Motor imagery: ERD

32 Many sources create noise. Electrode cap Raw EEGs from a field setting Acquisition

33 Acquisition A headband used for a 1D mu BCI (Pineda et al., 2003) Several brain areas responsible for movement.

34 Extraction, Translation Mu activity used for one dimensional control (Wolpaw et al., 1991)

35 Recording from a neuron (Kandel et al., 1988) Top: Utah intracranial electrode array Bottom: Cone electrode Cortical Neurons

36 A severely disabled subject controls a switch (Kennedy et al., 2004) A patient with an implanted BCI (Cyberkinetics, Inc.) Cortical Neurons

37 Cortical Neurons ECog activity in several human patients during hand or tongue movement (Miller, in press)

38 The P300 only occurs after target flashes (Allison and Pineda, 2003). Selective attention: P300

39 Environment User task: Imagine movement to move the cursor down. Relax to move it up. A mu BCI using BCI2000 (Schalk et al., 2004)

40 The P300 only occurs after target flashes (Allison, 2003). It is often largest over site Cz or Pz (central or parietal areas). Acquisition, Extraction