Correcting Robot Mistakes in Real Time Using EEG Signals Andres F. Salazar-Gomez, Joseph DelPreto, Stephanie Gil, Frank H. Guenther, and Daniela Rus 2017 IEEE International Conference on Robotics and Automation (ICRA) Humanoid Robots | COMS 6731

Introduction Human-robot interaction is not an intuitive paradigm and improving that interaction would have many benefits in robotics. Direct brain interaction is an exciting possibility, but brain data, specifically from electroencephalograms (EEG), is difficult to work with and can be hard to decipher. A natural, obvious EEG signal that could be readily identified and used could be a critical component in BCI interfaces that provide robots with commands or feedback. Humanoid Robots | COMS 6731

Background An EEG collects signal data, but it can be difficult to classify and scale as everyone’s brain is different and the signals are rarely clear. This data is very difficult to use in the most desired scenario: online, closed-loop feedback. Many current approaches often requires offline analysis and cannot be used in a closed-loop system. Speaker notes: EEG places a series of electrodes around the brain and collects voltage differences Scenarios Closed-loop implies robot and human directly affect one another throughout the task Open-loop implies that the robot does not receive feedback from human even thought feedback is collected Online performance implies that EEG classification and analysis occurs in realtime (system must acquire data for less than one second and make a classification within milliseconds), any longer and the efficacy of closed-loop is hurt. Offline performance is classification and analysis on data that has been pre-recorded and has no time constraint performance and as a result can generally perform better. Humanoid Robots | COMS 6731

Error-related Potential Signal Developing a new EEG signal for a subject is costly. Recent research has discovered a naturally occuring signal called an Error-Related Potential (ErrP) that occurs in most people (consciously or unconsciously) when that person recognizes an error is being either by that person or something else. Current theory states that ErrP’s are a natural component of the natural trial-and-error learning process. These signals have a characteristic shape even in people with no training or other adjustments and are seen within 500ms of the observed error. Speaker notes: Some BCI’s require extensive user training, extra cognitive load (focused thought), or constant visual stimuli Spikes likely come from the same centers of the brain, but from different electrodes due to different subjects head/brain shapes and sizes Humanoid Robots | COMS 6731

Application of ErrPs to robotics Labeled data or feedback are critical to machine learning systems and are not easy to generate. Direct feedback from ErrPs shows potential for use in reinforcement learning. These signals have been applied in certain settings such as in self-driving cars, but they have not yet been used in a closed loop feedback scenario. Speaker Notes: We already saw the difficulties in creating accurate value functions. Humanoid Robots | COMS 6731

Paper contributions The paper makes distinct contributions in confirming the existence and applicability of ErrP signals by demonstrating their ability to influence robots in real-time. It applies these signals in an online, closed-loop scenario during which the ErrP signals are the sole means of communication with the robot. The closed-loop scenario allows for the discovery and incorporation of secondary ErrP signals that follow the robot’s use of feedback. Demonstrates a human-centric communication system where the robot interprets the person’s natural (unforced) state in a methodology that does not require user screening or training. Humanoid Robots | COMS 6731

Experimental Design Baxter is making a binary choice between reaching towards two different items, with a chance of being correct or incorrect. An untrained subject is outfitted with an EEG cap to record the subjects brain signals as that subject observed the behavior of Baxter. Open-loop sessions (no real-time feedback) allowed for data collection and classifier training. Closed-loop sessions (with real-time feedback) were separated into four blocks: First block with no classification to train the classifier. Remaining blocks classified ErrP signals, trained new classifiers, and randomly triggered a follow-up error to induce a secondary ErrP signal. Speaking Notes: The secondary classification of ErrP signals was only done offline The open loop sessions confirmed the presence of the ErrP signals and optimized the parameter selection methods for use in the closed-loop sessions Humanoid Robots | COMS 6731

Experimental Methodology Two LEDs flash to get the subjects attention. One LED then signifies the correct choice Baxter randomly chooses a target and makes an initial movement. EEG signals are captured, classified, and relayed to Baxter (or not in open-loop case). Baxter is either confirmed or corrected. Baxter will randomly make a mistake even after being corrected to induce an ErrP signal. Speaking Notes: Done in both open-loop and closed-loop scenarios with 50/50 and 70/30 chance of correct choice respectively The secondary classification of ErrP signals was only done offline Humanoid Robots | COMS 6731

Experimental Details Baxter is using following a binary choice paradigm. Experiments took 1.5 hours per subject including cap preparation with 4 blocks of closed-loop data collection or 5 blocks for open-loop collection. Each block was 50 trials and lasted 9 minutes. Baxter was either programmed to have a 50/50 chance of making the correction selection or a 70/30 chance in the cases of open-loop and closed-loop respectively. Research was conducted at the MIT Distributed Robotics Laboratory Speaking Notes: The secondary classification of ErrP signals was only done offline 4 or 5 block determination made between two different subject populations Curtains added to minimize distraction Humanoid Robots | COMS 6731

Subject Selection Subjects were not pre-screened based on their ability to produce the correct EEG signals, nor were they trained to produce a new signal, which is hugely important for generalizability. 12 individuals were chosen. 7 participated in open-loop trials. 5 participated in closed loop (1 of which was omitted because that subject performed the task in a meditative state). Speaking notes: Which group went first? Humanoid Robots | COMS 6731

Experimental Video VideoWiki VideoLink Humanoid Robots | COMS 6731 https://www.youtube.com/watch?v=Zd9WhJPa2Ok VideoWiki VideoLink Humanoid Robots | COMS 6731

Experimental Video VideoWiki VideoLink Humanoid Robots | COMS 6731 https://www.youtube.com/watch?v=-iZbntmYoSE VideoWiki VideoLink Humanoid Robots | COMS 6731

Robot Control and EEG acquisition The experiment has 4 major subsystems Experiment controller written in python that controls the experiment, decides the correct target, tells Baxter where it should reach, and monitors timing. Baxter robot uses ROS to communicate with the controller. Its display shows a neutral face or an embarrassed face if it makes a mistake. EEG system uses 48 electrodes in a mask that feeds into a Matlab and Simulink process. Arduino Uno interfaces with the EEG, Baxter, and experiment controller. An important aspect is timing because of the ranges needed to accurately determine valid EEG signal timing. A hardware switch is attached to Baxter’s arm to determine the precise timing of the arm’s movement. Speaker Notes: Matlab/Simulink captures, processes, and classifies the input data and produces a single bit answer in each trial to indicate the detection of an Errp. Arduino process uses 8-9 bits of communication between the EEG system, controller, and robot to avoid any synchronization issues (low-level Arduino port manipulation). The process works well on Baxter initial motion, but has trouble with the second ErrP signal Secondary error detection occurs when human sees robot misinterpreted feedback (either kept going or made bad switch) The arm is in free motion at this time The exact moment when the human mentally interprets the target destination is unclear and a solution is challenging Humanoid Robots | COMS 6731

Signal Classification Pipeline Signal pre-processing starts by running data from all 48 EEG channels through a Butterworth filter. The mean of all channels is then removed from all channels to reduce noise. Dimensionality is then reduced to 9 central electrode channels. The means of all the trials are passed through an xDAWN filter and the filtered trials’ covariances are vectorized to yield 190 features. The correlation between the electrode channels also yields 9 features. An Elastic Net classifier was trained to yield linear regression values. A threshold was trained with a minimized biased cost function. The threshold is applied to the regression values for a binary choice output. Speaking Notes: Butterworth filter is a maximally flat magnitude filter that creates a frequency response that is as flat as possible in the band range “An ideal electrical filter should not only completely reject the unwanted frequencies but should also have uniform sensitivity for the wanted frequencies” Dimensionality reduction makes your classifier learning more efficient and removes electrodes with extraneous artifacts like blinking Spikes likely come from the same centers of the brain, but from different electrodes due to different subjects head/brain shapes and sizes XDAWN filter is a spatial filter meant to help signal to noise ratio and help with dimensionality reduction. Elastic net is linear or logistic regression technique that combines L1 and L2 penalties. Humanoid Robots | COMS 6731

Results: Confirming Error-related Potential Signals Primary result confirms the existence of and ability to classify error-related potential signals. In agreement with existing theory, there is a negative peak around 250ms after feedback onset followed by a positive peak. Speaker Notes: What is each graph saying? Green line is where no error was made by Baxter, red is when there was an error, black is the difference X axis is time, Y axis is the electrode ID on top left and is amplitude/voltage Humanoid Robots | COMS 6731

Results: Demonstrating Secondary Error Signals More unique result is the demonstration of the existence of the secondary ErrP signals when Baxter misinterprets the feedback. Speaking notes: CI means correct initial selection II means incorrect initial selection Even more interesting result is the increased amplitude (double that of the primary) and consistency of the shape of the signal (greater consistency across subjects). Humanoid Robots | COMS 6731

Performance Speaker notes: ROC stands for Receiver Operating Characteristic (plot of true positive vs false positive), AUC is Area Under Curve, AUC of a ROC curve is probability Offline closed-loop used 4 blocks of training from online closed-loop, trained and optimized, then used in a single trial Area classification errors because the signal was mis-classified or not seen? Offline performs better because more data. Online can only use data from previous blocks, less training data. Performance metrics gathered from closed-loop online, closed-loop offline, open-loop offline sessions. Performance of classification of secondary ErrP signals is examined as well as primary. Humanoid Robots | COMS 6731

Improvement with Secondary ErrP Signals Speaker notes: CI is most common because Baxter makes correct initial choice 70% of the time. Chance is the performance of random. Its determined by randomly shuffling trial labels (10 iterations of 10-fold cross-validation). Above chance is performance over random The inclusion of the secondary ErrP greatly improves the classification accuracy and boosts the performance by over 20% in the CI case. Offline analysis shows that including the secondary ErrPs greatly increases the true positive and true negative rates. Humanoid Robots | COMS 6731

Conclusion and Future Work Brain computer interfacing represents an exciting frontier in human-computer interaction, especially for use in reinforcement learning. Naturally occurring error-related potential signals serve as a viable and generalizable signal in this area, making the approach potentially viable for most users in the general population. The existing field of ErrP signal usage can be greatly improved upon through the use of secondary ErrP signals in a closed-loop setting to provide the robotic system with continuous feedback. Humanoid Robots | COMS 6731

Questions and discussion Is an EEG interface a viable tool for larger scale usage as a means of brain computer interfacing? Is brain interfacing significantly better than just telling the robot if it is correct or incorrect? Could this be applied in real-world settings in a way that adds value? Humanoid Robots | COMS 6731