Convolutional Neural Network for Multi-Category Rapid Serial Visual Presentation BCI [Ran Manor and Amir B.Geva] Yehu Sapir Outlines Review EEG/ERP (Event-Related Potential) P300 Spatio-temporal data CNN Network + regularization Results and discussion

Review Categories of electrical cortical activities in Brain Computer Interface (BCI) μ β μ (8–12 Hz) and β (12–30 Hz) rhythms. Response of motor output of the brain. P300 Evoked Potential (EP). dementia / Alzheimer dementia / Alzheimer Auditory, visual, or somatosensory Event-Related Potentials (ERP) (1–30 μV). 300–500 ms post-stimulus Visual N100 and P200, ERPs with short latency Response of the brain to a rapid visual stimulus. Steady-State Visual Evoked Potentials (SSVEP). Visual stimulations (3.5 - 75 Hz) generates brain responses at the same freq. Slow Cortical Potentials (SCP). slow potential variations 0.5–10 sec after presenting stimulus users control these potentials and use them to control the movement of a cursor on screen movement reduced cortical activation.

Review P300 signal P300 ERP at electrode Pz, P300 = target detection for RSVP (Rapid Serial Visual Presentation) tasks. The amplitude and latency have a large variance Average of multiple single trials Amplitude Pz Single trial response Time [ms]

Subjects & Stimuli Aim Subjects Stimuli classify ERPs into target and non-target images. Subjects 15 subjects participated in a RSVP experiment, 8 females, 7 males with age 25±5 years. Stimuli 360 × 360 pixels, gray-scale images. The images have the same mean luminance and contrast. 5 different categories including 145 exemplars each of faces, cars, painted eggs, watches, and planes. one category is the target

Experimental Procedure Brain computer interface (BCI) Images presented in 4 blocks, 6525 images in each block, presented every 90–110ms (~10Hz). 20% of the images were targets, randomly distributed within each block. different target category in each block watches were not used as targets. Use same blocks order across subjects. Subjects should count targets. Pause every 80–120 image Reporte the Number of target. Restart the count.

EEG Acquisition and Preprocessing 64 electrodes + 7 additional electrodes (to reject trials when blinking). 256Hz sampling rate 51Hz online LPF to prevent aliasing of high frequencies and remove powerline noise. 0.1Hz offline high-pass filter to remove slow drifts (DC). Data segmentation: 1sec event-related segments (-100ms to +900ms of image presentation) Single trial =1 Image 256 Samples 64 channels (electrodes) Down-sampling, Normalizing & Removing DC 64 target 64 µ=0,σ=1 +900ms -100ms µ=0,σ=1

Neural Network Architecture Implementation on Caffe and GPU Time Samples ReLU ReLU Dropout ReLU Dropout ReLU ReLU softmax channels # output filters

Spatio-Temporal Regularizer penalty term : Where: - conv. output l at time t - regularization coefficient , Where: - a matrix of zeros with 1 in position i, j Regularized No-regularization

Parameters updates Train the Net. By Minimize the loss function where Nsamples = training samples, x(i) = i training sample, y(i) = true label of sample i hk = NN output unit k Minimized with SGD, Momentum update: V= µ*V + η*dW Weights updates: W=W-V Learning rate (η) = 0.001 Momentum coefficient (µ) = 0.9 network parameters were chosen empirically by cross-validation.

Performance Correct is distorted, due to imbalanced classes positives + negatives random cross-validation procedure 80% training 20% testing (4 blocks per subject) average of 10 runs (true positive + true negative) /2 true positive false positive Area Under the Curve Correct is distorted, due to imbalanced classes Mean across subjects

Compare to Spatially Weighted FLD-PCA (SWFP)

Features Analysis Representative spatial filters and their temporal features from a sample subject (504). Spatial maps are similar to the spatial distribution of the P300 with a high amplitude at central-parietal electrodes. spatial filters distributed on the scalp mean temporal activations of the spatial filter

Sample of spatial filters and temporal activations for subjects 501 502 503 507

Sample of spatial filters and temporal activations for subjects 508 511 512 513

BCI Competition 2004 Benchmark Compare our CNN model to BCI Competition, 2004 P300 Speller task. Two subjects (A, B) F E D C B A L K J I H G R Q P O N M V U T S false alarm rate % hit rate correct classification % Subject 27.36 68.13 71.9 A [BCI Competition, 2004] 29 67 70.5 A [Cecotti and Gräser, 2011] 19.11 70 79.1 B [BCI Competition, 2004] 20 69 78.19 B [Cecotti and Gräser, 2011]

Cross-session performance correctly classified (true positive + true negative) /2 true positive false positive

Thank You!