1. Papavasileiou-1 CSE 5810 Brain Computer Interface in BMI Ioannis Papavasileiou Computer Science & Engineering Department The University of Connecticut 371 Fairfield Road, Unit 4155 Storrs, CT 06269-2155 papabasile@engr.uconn.edu

2. Papavasileiou-2 CSE 5810 What is BCI?  BCI is:  System that allows direct communication pathway between human brain and computer  It consists of data acquisition devices, and appropriate algorithms  How is it used in BMI:  Clinical research  Disease-condition detection and treatment  Human computer interfaces for  Control  Emotions detection  Text input - communication

3. Papavasileiou-3 CSE 5810 Research areas involved  Computer science  Data mining  Machine learning  Human computer interaction  Neuroscience  Cognitive science  Engineering

4. Papavasileiou-4 CSE 5810 Key challenges  Technology-related:  Sensor quality – low SNR  Supervised learning – “curse of dimensionality”  System usability  Real-time constraints  Non-invasive EEG information transfer rate is approx. 1 order of magn. lower  People-related  People are not always familiar with technology  Preparation – training phases are not fun!  Concentration, attention consciousness levels  Task difficulty

5. Papavasileiou-5 CSE 5810 BCI components  Data acquisition  Electroencephalography (EEG)  Electrical activity recording  Invasive or not  Functional Near Infrared Spectroscopy (fNIRS)  Recording of infrared light reflections of the brain  Functional magnetic resonance imaging (FMRI)  Detection of changes in blood flow  Data Analysis  Data mining & machine learning  Decision making  Output & Control  HCI

6. Papavasileiou-6 CSE 5810 Typical BCI architecture

7. Papavasileiou-7 CSE 5810 Electroencephalography (EEG)  What is it:  Recoding of the electrical activity of the brain  Types:  Invasive  Non-invasive  Properties:  High temporal resolution  Low spatial resolution  Scalp acts as filter!

8. Papavasileiou-8 CSE 5810 International 10-20 standard  Electrodes located at the scalp at predefined positions  Number of electrodes can vary

9. Papavasileiou-9 CSE 5810 The EEG waves  Alpha – occipitally  Beta – frontally and parietally  Theta – children, sleeping adults  Delta – infants, sleeping adults

10. Papavasileiou-10 CSE 5810fMRI  Functional magnetic resonance imaging  Fact:  Cerebral blood flow and neuronal activation coupled  Detection of blood flow changes  Use of magnetic fields  High spatial resolution  Low temporal resolution  Clinical use:  Assess risky brain surgery  Study brain functions  Normal  Diseased  Injured  Map functional areas of the brain

11. Papavasileiou-11 CSE 5810fNIRS  Functional Near Infrared Spectroscopy  Project near infrared light into the brain from the scalp  Measure changes in the reflection of the light due to  Oxygen levels associated with brain activity  Result absorption and scattering of the light photons  Used to build maps of brain activity  High spatial resolution  <1 cm  Lower temporal resolution  >2-5 seconds

12. Papavasileiou-12 CSE 5810 BMI & clinical applications  Diagnose:  Epilepsy – seizures  Brain-death  Alzheimer’s disease  Physical or mental problems  Study of:  Problems with loss of consciousness  Schizophrenia (reduced Delta waves during sleep)  Find location of:  Tumor  Infection  bleeding Source: http://www.webmd.com/, http://www.nlm.nih.govhttp://www.webmd.com/ http://www.nlm.nih.gov

13. Papavasileiou-13 CSE 5810 Sleep disorders & mental tasks  Sleep disorders study  Insomnia  Hypersomnia  Circadian rhythm disorders  Parasomnia (disruptions in slow sleep waves)  Mental tasks monitoring  Mathematical operations  Counting  Etc.

14. Papavasileiou-14 CSE 5810Neurofeedback   Applications in  Autistic Spectrum Disorder (ASD)  Anxiety  Depression  Personality  Mood  Nervous system  Self control

15. Papavasileiou-15 CSE 5810 Feedback EEG-BCI architecture

16. Papavasileiou-16 CSE 5810 Typical data analysis process  Data acquisition and segmentation  Preprocessing  Removal of artifacts  Facial muscle activity  External sources, like power lines  Feature extraction  Typically sliding window  Time-frequency features  Latency introduced

17. Papavasileiou-17 CSE 5810 Feature extraction  Model-based methods  Require selection of the model order  FFT (Fast Fourier Transform) – based methods  Apply a smoothing window  Features used:  Specific frequency band power  Band-pass filtering and squaring  Autoregressive spectral analysis  Many times a feature selection or projection is done to reduce the huge feature vectors

18. Papavasileiou-18 CSE 5810 Data Classification  Typical classifiers used  Artificial Neural Networks (ANN)  Linear Discriminant analysis (LDA)  Support Vector Machines (SVM)  Bayesian classifier  Hidden Markov Models (HMM)  K-nearest neighbor (KNN)  Parameters for each classifier can affect the performance  # of hidden units in ANN  # of supporting vectors for SVMs  Etc.

19. Papavasileiou-19 CSE 5810 Human computer interaction  BCIs are considered to be means of communication and control for their users  HCI community defines three types:  Active BCIs  Consciously controlled by the user  E.g. sensorimotor imagery (multi-valued control signal)  Reactive BCIs  Output derived from reaction to external stimulation  Like P300 spellers  Passive BCIs  Output is related to arbitrary brain activity  E.g. memory load, emotional state, surprise, etc.  Used in assistive technologies and rehabilitation therapies

20. Papavasileiou-20 CSE 5810 BCI & Assistive Technologies  Communication systems  Basic yes/no  Character spellers  Virtual keyboards  Control  Movement imagination  Cursor  Wheelchairs  Artificial limbs & prosthesis  Automation in smart environments  Current BCI systems have at most 10-25 bits/minute maximum information transfer rates  It can be valuable for those with severe disabilities

21. Papavasileiou-21 CSE 5810 P300 spellers  Most typical reactive BCI  3-4 characters / min with 95% success

22. Papavasileiou-22 CSE 5810 P300 wave  Event related potential (ERP)  Elicited in the process of decision making  Occurs when person reacts to stimulus  Characteristics:  Positive deflection in voltage  Latency 250 to 500 ms  Typically 300 ms  Close to the parietal lobe in the brain  Averaging over multiple records required

23. Papavasileiou-23 CSE 5810 Other ERP uses  Lie detection  Increased legal permissibility  Compared to other methods  ERP abnormalities related to conditions such as:  Parkinson’s  Stroke  Head injuries  And others  Typical ERP paradigms  Event related synchronization (ERS)  Event related de-synchronization (ERD)

24. Papavasileiou-24 CSE 5810 Other Control BCI paradigms  Lateralized readiness potential  Game control  1~2 seconds latency  Negative shift in EEG develops before actual movement onset  Steady-state visually evoked potentials (SSVEPs)  Slow cortical potential (SCP)  Imaged movements affect mu-rhythms  They shift polarity (+ or -) of SCP  Sensorimotor cortex rhythms (SMR)  EMG

25. Papavasileiou-25 CSE 5810 SCP & SMR vs P300  Typically SCP and SMR BCIs require significant training to gain sufficient control  In contrast P300 BCIs require less as they record response to stimuli  However, they require some sort of stimuli like visual (monitor always in place) or audio  Also SCP BCIs have longer response times

26. Papavasileiou-26 CSE 5810 Binary speller control  User imagines movement of cursor  Typically hand movement  The goal is to select a character

27. Papavasileiou-27 CSE 5810 Wheel chair control  All the mentioned BCI paradigms have been applied to wheelchair control  Either using a monitor for feedback  Or active paradigms as sensorimotor imagery (SMR)  Similar approaches have been applied to robotics  Artificial limbs  etc

28. Papavasileiou-28 CSE 5810 Environment control  BCIs used by disabled to improve quality of life  Operation of devices like  Lights  TV  Stereo sets  Motorized beds  Doors  Etc  Typically use of P300, SMR and EMG related BCIs

29. Papavasileiou-29 CSE 5810 EMG-based human-robot interface example  Motion prediction based on hand position  EMG pattern classification as control command  Combination of both yields motion command to prosthetic hand

30. Papavasileiou-30 CSE 5810 Emotions detection  Use of facial expressions to imply user emotions  ERD/ERS based BCIs  Emotional state can change the asymmetry of the frontal alpha  P300 - SSVEP  Emotional state can change the amplitude of the signal from 200ms after stimulus presentation

31. Papavasileiou-31 CSE 5810 BCIs for recreation  Games  EPOC headset  Mindset  Virtual reality  Outputs of a BCI are Shown virtual environment  Creative Expression  Music  Generated form EEG signals  Visual art  Painting for artists who are locked in as a result of ALS – amyotrophic lateral sclerosis

32. Papavasileiou-32 CSE 5810 Security and EEG  EEG has been used in user authentication  Every brain is different  Different characteristics of EEG waves are used in user authentication  Pros  User has nothing to remember  Harmless  Automatically applied  Cons  User has to wear an EEG headset  Accuracy is still not 100%  Still not used in practice

33. Papavasileiou-33 CSE 5810 Thank you!