1. Automated Interpretation of EEGs: Integrating Temporal and Spectral Modeling Christian Ward, Dr. Iyad Obeid and Dr. Joseph Picone Neural Engineering Data Consortium College of Engineering Temple University Philadelphia, Pennsylvania, USA

2. NEDC TutorialNovember 8, 2013 1 Abstract The goal of this presentation, the second part of a two-part presentation on our research on EEGs, is to describe how we apply contemporary machine learning technology to the problem of interpreting EEGs. The input to the system is an EEG. The output is a transcribed signal and a probability vector containing probabilities associated with various diagnoses. Machine learning is used to optimize the parameters of the underlying mathematical models. Central to this process is the existence of a large corpus of EEGs. In this case, we leverage the TUH EEG Corpus, which contains over 20,000 EEGs collected over a 11-year period from 2002 to 2013. The corpus contains patient histories and diagnoses, but does not contain time-aligned detailed transcriptions of the signals. A major goal of this presentation is to explain how we can build powerful systems using a semi-supervised, or even unsupervised, process. [… this paragraph will eventually say something about performance …]

3. NEDC TutorialNovember 8, 2013 2 Transcriptions prm_07 prm_03 prm_05 prm_09 prm_01 N-channel EEG Signal with transcriptions: What are primitives? A set of events (e.g. prm_07 above), or artifacts, appearing in the EEG waveforms that physicians use to confirm a diagnosis. Each diagnosis can be represented as a sequence of these primitives. Typically a set of 6 to 12 types of artifacts (e.g., spike, GPED, eye blink). What is a transcription? A sequence of primitives, or labels, that describes each segment, or epoch of the waveform. A time-aligned transcription includes start/stop times of each symbol.

4. NEDC TutorialNovember 8, 2013 3 Transcriptions and Graphs prm_07 prm_03 prm_05 prm_09 prm_01 N-channel EEG Signal with transcriptions: Transcriptions, often called “truth markings”, are used by machine learning systems to discover a mapping between the waveform (input) and the transcription (output). Transcriptions can be simple high-level classifications (e.g., stroke) or lower-level descriptions (e.g, “noise prm_07 noise prm_05 noise”). Transcriptions can be linear (a left to right graph with one level) or hierarchical (containing high level symbols that can be represented as sequences of primitives). Transcriptions can be represented as graphs and are often coded in XML.

5. NEDC TutorialNovember 8, 2013 4 Supervised Learning prm_07 prm_03 prm_05 prm_09 prm_01 N-channel EEG Signal with transcriptions: Modern pattern recognition systems often require transcriptions, but not time alignments, for training. A generative model is trained by increasing the likelihood that the model produced the data given the transcription. Different ways transcriptions are used during the training process:  Fully supervised: every sample of the waveform is assigned to one or more labels; time-alignments for labels are provided (start and stop times).  Semi-supervised: only the labels are provided; unlabeled data is often assumed to be “background noise”; no alignments.  Partially-supervised (or flexible): some labels are missing.  Unsupervised: no labels; self-organization (e.g. clustering).

6. NEDC TutorialNovember 8, 2013 5 Forced Alignment prm_07 prm_03 prm_05 prm_09 prm_01 N-channel EEG Signal with transcriptions: Assume a single channel is labeled: “background” prm_07 “background” prm_05 “background” Further, assume each primitive is represented by some sort of finite state machine (e.g., a hidden Markov model). During semi-supervised training, the system attempts to find the best alignment between the primitives and the data. The result of this optimization process is a “state-frame map” – each state of the model is mapped to one or more frames of data. Every frame of data is accounted for and mapped to some state in some model.

7. NEDC TutorialNovember 8, 2013 6 Case Study: Speech Recognition and Phonemes

8. NEDC TutorialNovember 8, 2013 7 Case Study: Hidden Markov Models prm_07: Each primitive is represented by an N-state hidden Markov model (HMM). The topology of the model is typically a 3-state “left-to-right” HMM as shown. The supervised learning process aligns the models to the data, and then reestimates the parameters of the model based on all the data assigned to that state (based on EM and Baum-Welch training). The process is iterative and the models are guaranteed to converge to the maximum likelihood solution. Three passes of training is usually adequate for each state of parameter training.

9. NEDC TutorialNovember 8, 2013 8 Future Work (1) Baseline System No. 1 (Jan. 1): Simple MFCC-like features Fixed-size epochs (approx. 1 sec) Process each channel independently Classify each epoch on each channel as normal/abnormal Classify session as normal/abnormal based epoch classifications (2) Baseline System No. 2 (Feb. 1): MFCC-like features / channel-independent processing HMM models for 6 to 12 primitives Standard HMM training using semi-supervised transcriptions NN or RF classifier for mapping HMM scores to diagnoses (3) Optimal System (Jun. 1): Deep learning for the primitive models NN or RF classifier for mapping to diagnoses Explore different features, topologies, etc. using systems (2) and (3)

10. NEDC TutorialNovember 8, 2013 9 Brief Bibliography of Relevant Documentation [1] Picone, J. (1993). Signal modeling techniques in speech recognition. Proceedings of the IEEE, 81(9), 1215–1247. [2]Picone, J. (1990). Continuous speech recognition using hidden Markov models. IEEE ASSP Magazine, 7(3), 26–41.