1. HCII2007-ACI2007-BCI Workshop Beijing, China, July 23, 2007 Development of a Hybrid EEG Sensor Array for Brain-computer Interfaces Leonard J. Trejo, Ph.D. Quantum Applied Science and Research www.quasarusa.com International Workshop on Brain-Computer Interface Technology and Second BCI2000 Workshop Beijing, China, July 23-24, 2007

2. HCII2007-ACI2007-BCI Workshop Beijing, China, July 23, 2007 Research Team Robert Matthews Neil McDonald Harini Anumula

3. QUASAR Breakthroughs in Sensor Technology for Applications from Minerals to Medicine

4. Our Vision for Brain-Computer Interaction Truly deployable unobtrusive biosensors –Nearly invisible to the user –Seamlessly integrated with clothing and appliances End-to-end systems for operational biosensing –Broad application spectrum (e.g., EEG, EOG, ECG and EMG) –Low-power, long lasting, wireless electronics –Robust algorithms tolerant of noise and sensor dropout –Intelligent systems adapt for situational or individual differences Focus of this presentation: Hybrid EEG Sensors NO ADHESIVES NO WIRES NO PREP NO GELS

5. Barriers to Truly Deployable EEG Sensors Large common mode signals Frequent user motion will generate signal artifacts Sensor harness physicality issues –Lack of user compliance –Possible incorrect application of sensors –Fitting sensors into wearable garments –Variability over subjects –Hygiene Battery power issues Size and weight issues Long-term usability and reliability

6. QUASAR EEG Sensor Technologies QUASAR Capacitive sensors (on skin EEG, EOG, ECG) QUASAR Next Gen sensors (standoff detection) QUASAR Hybrid sensors (through hair) Validation Studies Completed 20042006 2008-2009 Fingers

7. Pros and Cons of Current EEG Sensors Conductive sensors –Generally good signal quality, low noise –Most require some skin preparation –Produce skin irritation and risk of infection –Micro- or nano- needle sensors break off and remain in skin –Cannot be worn for extended periods of time (electrode drying) Capacitive sensors –Sensitive to static charges, stray E-fields, and motion artifacts –No skin preparation, electrolyte drying or skin irritation

8. Hybrid EEG Biosensors (the best of both worlds) Capacitive QUASAR (IBE) EEG sensors –Presently record high-quality EEG signals on bare skin –Through-hair EEG sensing is not currently feasible –Through-hair will be feasible with next-gen technology Advanced QUASAR capacitive sensors systems make new hybrid EEG sensor technology possible –Hybrid sensors use ultra high resistive contact in conjunction with capacitive contact –Hybrid sensors do not modify or invade the skin and consequently do not cause skin irritation –Hybrid sensors are a new option for wearable EEG arrays

9. Deployable EEG/EOG Sensor Concepts QUASAR Hybrid EEG Electrodes Audio Headset EOG/EEG Glasses QUASAR Capacitive Electrodes

11. Initial Hybrid Sensor Demo (3/2006) QUASAR and wet electrode in the C z position – filtered in 5-15 Hz bandwidth Test set up – standard electrode positions Correlation between two sensors Trial for an outside entity Sensor used (old) New sensor

12. Four HCI contexts tested with side-by-side “wet” electrodes and QUASAR Hybrid EEG sensors 1.Baseline: Eyes Open/Closed (desynchronization of EEG alpha rhythm) –30 s open / 30 s closed; repeated 3x over session 2.Real/Imaginary Button-press (desynchronization of EEG  -Rhythm) –Reach and press key on keyboard; repeat with imaginary motion 3.Visual Oddball (modulation of P300 ERP component) –Count the number of visual targets (row or column of matrix) 4.Memory/Cognition (modulation of EEG alpha and theta rhythms) –Count backwards from 100 by ones or sevens for 60 s All conditions: recorded Fz, C3, Cz, C4, referred to right ear Ag-AgCl disc electrodes, QUASAR IBE, and hybrid sensors

13. QUASAR Headband System

14. EEG Eyes Open/closed: Correlation of Signals over 30-s Long Segments Eyes-closed EEG task, signals normalized to zero mean and unit standard deviation, displaced vertically to allow comparisons. From the top down: QUASAR Cz, wet Cz, QUASAR Fz, wet Fz, QUASAR C3, wet C3, EOG, and ECG. Red and blue lines are for the QUASAR sensor and wet electrode recordings, respectively.

15. EEG Eyes Open/closed: Distribution of Correlation Coefficients for 1-s Segments Histogram of correlations over consecutive 30 1-s segments for QUASAR Fz - wet Fz. In 26/30 segments the QUASAR-wet correlation was 0.80 or higher. EEG signal band-limited to 0-40 Hz.

16. EEG Eyes Open/closed: Correlations of Power Spectra. Power spectral density functions of 30-s EEG recordings during eyes-closed session and following eyes-open session (Welch’s method, window = 2 s, overlap = 1 s, FFT length = 1024, sampling rate = 100 Hz). QUASAR-Wet Correlations Eyes closed: 0.9979 Eyes open: 0.9922

17. Button-press Tasks: Sample Power Spectra (Hybrid Sensors only). PSDs EEG recordings at Cz during button-press (60 s), imaginary button-press (60s), and eyes-open tasks (30 s). Pearson r values for QUASAR-wet sensor pairs at Cz, Fz, and C3 ranged from 0.9941 to 0.9994.

18. Counting tasks: Sample Power Spectra PSDs of 30-s EEG recordings during easy and hard levels of counting task. Easy-hard spectral differences appear in the theta (4-7), alpha (8-12), and SMR (14-16 Hz) bands of the EEG spectrum. Inter-sensor correlations between QUASAR and wet sensors at Cz in the easy and hard conditions were 0.9993 and 0.9987, respectively. r =.9993 r =.9987

19. Visual Oddball Task: Sample ERP Averages Oddball (P300) Task: count deviant stimulus (row or column; P = 1/7) Post-stimulus Time (ms)

20. QUASAR hybrid sensor EEG signals correlate extremely well with simultaneously recorded wet electrodes. –QUASAR – wet sensor correlations in the time domain were typically at 0.8 or better (interchannel correlations of wet sensors ranged from 0.81 - 0.87) –Correlations of QUASAR - wet frequency spectra within the EEG band of 0-40 Hz were typically above 0.99. QUASAR hybrid sensors can gauge EEG important for AugCog & HCI –modulation of EEG rhythms, such as alpha, theta, and SMR –detection of P300 ERPs for discrete stimuli Conclusions from Sensor Tests

22. Development of an Integrated Neurocognitive Sensor Array Through clothing ECG On skin EOG On skin EEG Through hair EEG sensor Algorithms to classify even in noisy environments Fully deployable Bio monitoring system Data Acquisition Sensor Suite AUGCOG & others Aberdeen SBIR Fully Deployable Bio monitoring system Unfunded Sensor Harness Phase II program

23. QUASAR-Wadsworth NIH SBIR Project (2008)

24. Specific Aims Design a dry EEG sensor array and amplifier system tailored for use as a new front-end for the existing brain-computer spelling system. Show that the existing brain-computer interface can operate effectively with the new sensor array as the front end. Compare the dry EEG sensor array to the standard wet electrode array in eight able-bodied human subjects who will operate the brain-computer interface speller in a controlled experiment. Ensure that the dry EEG sensor array is easy to apply and comfortable when worn for an extended period of time.

25. Specific Aims (cont’d) Using standardized rating scales, we will compare the comfort of the dry EEG sensor array with the existing wet EEG electrode array. Perform controlled experiments and administer a battery of questionnaires regarding the two systems pre- and post-test. Our hypotheses are that a) subjects’ perceived comfort of the dry EEG sensor array matches or exceeds the comfort of the existing wet EEG sensor array, b) that subjects’ self-reported mood during sessions performed with the dry EEG sensor array, matches or is improved over subjects’ self-reported mood during sessions performed with the existing wet EEG sensor array.

26. 10-Channel EEG Prototype DAQ Module 10 channel EEG Ultra low power (2 days on 2 AA batteries) Over 80dB common mode rejection Up to 1kHz sample rate Advanced Wireless A/D System

27. Wireless Device uC + Radio (nRF24E1) 50 ohm Matching Network Regulator (LTC3204-3.3) Chip Antenna (AN9520) Dual MOSFET (NTHD4508N ) Tri-Axial Accelerometer + Temp Sensor (H34C) Interface Flex. PCB High bandwidth device low power optimized for duty cycling

28. Algorithm Development* On-line classifier using adaptive KPLS feature extraction and adaptive classification networks −Can adapt for individual or situational differences in features and feature mappings −Applications tested: − On-line cognitive fatigue assessment (3.5 s update rate) − Multimodal real-time advisory system for human-automation reliability (2 s update rate) − Brain-computer interface (2 s update rate) Basic advances in signal processing (denoising, kernel methods, software/hardware artifact removal) *QUASAR & Partners: NASA, Pacific Development & Technology

29. Real-time EEG/Mental State Classifier

30. PLS Algorithm in BCI Moving Map Application (2005)

31. EEG-based Classification with Low-density Arrays or Low-SNR Signals Robust algorithm applied to mental fatigue classification using multichannel EEG spectra Trained KPLS-DLR classifier in Air Force pilots using 19 electrodes Corrupted signal from SNR of 0 dB to -18 dB Reduced density from 19 to 4 electrodes Preserved >80% accuracy with -9 dB SNR and four electrodes

32. Multimodal Assessment of Cognitive Overload EEG Probe ERP ECG Cognitive overload algorithm Real-time alert or advisory signal KPLS-LDA classification of 20-ch EEG spectra from 2-s epochs