NEUROPHONE: BRAIN- MOBILE PHONE INTERFACE USING A WIRELESS EEG HEADSET Andrew T. Campbell, Tanzeem Choudhury, Shaohan Hu, Hong Lu, Matthew K. Mukerjee!, Mashfiqui Rabbi, and Rajeev D. S. Raizada Dartmouth College, Hanover, NH, USA

Motivation Mobile phones and neural signals are present are accessible to many people. Recent advances in technology has led to the development in low-cost EEG headsets. Smart phones are now powerful enough to run sophisticated machine learning algorithms. It is thus easy to interface neural signals with mobile computing paradigms.

Introduction This group proposed to used neural signals to control a mobile phone. They developed the NeuroPhone system that translates and decodes neural signals to drive a mobile app using off-the-shelf wireless EEG headsets. This paper demonstrates their brain-controlled address app: An application that uses the brain signals to select address contacts to call.

Introduction They implement their mobile app using two different paradigms: P300 dialing and “Wink”-triggered dialing. P300 signals are positive transient deflections in EEG that are elicited in response to a rare or novel stimulus The eye “Wink” is a type of EMG signal that is generated in response to the contraction of skeletal muscle contraction.

Challenges Research Grade EEG headsets Expensive (Often costing tens of thousands of dollars) Offer very robust and reliable EEG signals Off-the-shelf EEG headsets More affordable ($100-$500) Electrode design and amplification are not as robust Results in noisy, low-quality signals. Require more sophisticated processing techniques to classify neural events. Most Off-the-shelf headsets are wireless and thus encrypt the EEG signals. They are designed for synchronization with a computer (using wireless dongle). They complicate the process of developing a clean brain-mobile interface.

Challenges There is an energy cost for brain-mobile interfacing: Continuously streaming raw brain-signals wirelessly Running classifiers on the phone introduces heavy processor loads. Brain-mobile phones could likely be used in applications such as: walking, riding in a car or bicycle, shopping, etc. Many of these cases present significant noise artifacts in the EEG signals. These signals will need to be filtered out to improve the brain-mobile interface

NeuroPhone The NeuroPhone system uses the iPhone to display pictures of contacts in the phone’s address book. The pictures are displayed and flashed in random order. For the EEG mode, the user concentrates on a picture of the person they wish to call. For the wink mode, the person winks with the left or right eye to make the intended phone call

P300 Whenever the user concentrates on a target stimulus among a pool of non-target stimulus, the target stimulus (flash) will elicit a positive peak in the EEG at around 300ms after stimulus onset (P-300). The P300 signal can be found on most EEG channels Common on central and parietal channels

NeuroPhone - P300 Paradigm In This case, there are 6 total stimuli on the screen (5 non-target and 1 target). The user visually attends to one of the photos while each photo is flashed in a random order. Whenever the target photo flashes, a P300 should be generated.

Wireless EEG Headset Emotiv EPOC headset 14 data electrodes (2 reference electrodes) Transmits encrypted data wirelessly to a windows-based machine. (802.11) 2.4GHz Low SNR Contains build in gyroscope ~$300

Pre-Processing Signals were band-passed filtered to keep only the relevant information within the P300 range. Signal averaging was performed to increase the SNR This improves the quality of the signal while simultaneously adding lag to the system

Classification To reduce complexity, only a subset of relevant channels are used for classification. Wink Mode Multivariate, naive Bayesian classifier. P300 Mode Decision stump classifer

Implementation Laptop relay is used for decoding of the encrypted Emotiv signals Encrypted EEG signals are sent from the phone to a laptop for decryption (via WiFi). Decrypted EEG signals are sent back to the phone. Signals are sampled at 128 samples per second and transferred to the phone at 4kbps per channel.

Wink Mode Classification Emotiv head-set was put on backwards to place two electrodes directly above the eyes. Data was collected by having the subject wink multiple times. –Data were labeled as “wink” or “non-wink” A Bayesian classifier was trained by calculating the mean and variance of each wink and non- wink and building respective Gaussian models. –As can be seen, the two models do not overlap leading to good classification

P300 Classification The Gaussian distributions overlap too much and therefore cannot be classified with a Bayesian classifier. Signals from each of the six stimuli were band-passed filtered between 0-9Hz. The highest signal segment at around 300ms after stimulus onset is extracted. For classification, a decision stump is used where the threshold is set to the maximum value of the extracted segment.

Multiple sessions were collected on three subjects. Subjects performed the test while sitting and while walking The classifier was trained on five sessions from a single subject and then tested on the remaining subjects. (I think). Results are shown in table 1 –Precision: % of classified winks that are actual winks –Recall: % of actual winks that are classified as winks. –Accuracy: % of total events that are classified correctly Results (Wink-Mode)

Results (P300 mode) Data was collected with same set of subjects while sitting, with loud background music and while standing up.

Discussion Although data was classified using the P300 mode, large amounts of averaging is needed to get decent classification accuracies. This “unresponsiveness” of the system proves to be very frustrating for the end user. i.e. it can take 100 seconds to initiate a phone call with only 89% chance of dialing the right person (with six to choose from). This System is currently not in any form to be used by subjects on a regular basis. Looking into single trial classification techniques to speed up the system.

Phone Loading Statistics The CPU usage when running the application: 3.3% for the iPhone (iphone 3g?). Total memory usage: 9.40MB memory used (9.14MB are for GUI elements). Continuous streaming raw EEG channels to the phone, and processing signals lead to battery drain (no quantitative measure given) Looking into duty cycling to solve this phone.