1. klinické neurofyziologie
Data mining EEG data:
When models can be trusted?
Pavel Kordík
Miroslav Čepek
Computational Intelligence Group
Department of Computer Science and Engineering
Faculty of Electrical Engineering
Czech Technical University in Prague
55. Společný sjezd
klinické neurofyziologie

2. FFT, Automated preprocessing
Data mining model
FFT, Automated preprocessing
Raw EEG signals
Feature extraction
Manual annotation
GAME neural network
Training data set
Data mining
This presentation
Model (classifier)
Classification
Annotated signal

3. Raw EEG signals Training data set Newborn sleep stage EEG dataset
Institute for Care of Mother and Child, Prague
FFT, statistics
annotation:
fp1_min fp1_max fp1_skewness fp1_kurtosis fp1_mean fp1_std fp1_mean_abs_first_derivation fp1_fft_abs_delta … fp2 … Wake Active Quiet
-60.94
35.94
-0.31
2.56
-8.01
19.21
2.77
18.74
3.97
29.67 … 1
-53.12
51.56
0.31
2.80
-9.52
19.41
2.93
14.06
4.12
15.61
-59.38
50.00
0.15
2.85
-8.28
20.56
2.75
10.94
3.51
17.20
Training data set

4. GAME Neural Network
GAME models grow automatically using the training data set.

5. Model (classifier) Classifier Wake: output equals 1 for Wake class
0 for Active,Quiet
Classifier for Active
Classifier for Quiet
Wake (0,1)

6. Models can be written as equation
Wake=0.257*exp(1.236/(1+exp(7.718*exp(-4.307*c4_fft_rel_gamma+0.43)-1.985*ecg1_mean_abs_second_derivation+22.58)) *fp1_fft_rel_beta1+0.14*png1_skewness)-0.269
Quiet=…
Active=…

7. Recall of model on testing data
annotation:
fp1_min fp1_max fp1_skewness fp1_kurtosis fp1_mean fp1_std fp1_mean_abs_first_derivation fp1_fft_abs_delta … fp2 … Wake Active Quiet
-60.94
35.94
-0.31
2.56
-8.01
19.21
2.77
18.74
3.97
29.67 … 1
-53.12
51.56
0.31
2.80
-9.52
19.41
2.93
14.06
4.12
15.61
-59.38
50.00
0.15
2.85
-8.28
20.56
2.75
10.94
3.51
17.20
GAME classifier
WAKE
GAME classifier
ACTIVE
GAME classifier
QUIET
0.02
0.99
0.2
Correct classification
But:
0.22
0.21
0.1
Usual approach: Choose a classifier with maximum response
0.99
0. 99
0.99
???????????

8. Where is the problem? Can we trust classifiers?
Sometimes, their output is random (not based on training data)
How to estimate credibility of the classifier?
Our approach:
Create ensembles of classifiers

9. Ensemble classifiers – what is it?
The collection of classifiers is trained for the same task.
Input variables
Classifier WAKE 1
Classifier WAKE 2
Classifier WAKE 3 …
Classifier WAKE n
Output variable
Output variable
Majority or Multiply
Ensemble output

10. Recall of ensembles on testing data
annotation:
fp1_min fp1_max fp1_skewness fp1_kurtosis fp1_mean fp1_std fp1_mean_abs_first_derivation fp1_fft_abs_delta … fp2 … Wake Active Quiet
-60.94
35.94
-0.31
2.56
-8.01
19.21
2.77
18.74
3.97
29.67 … 1
-53.12
51.56
0.31
2.80
-9.52
19.41
2.93
14.06
4.12
15.61
-59.38
50.00
0.15
2.85
-8.28
20.56
2.75
10.94
3.51
17.20
GAME classifier
WAKE
GAME classifier
WAKE
GAME classifier
ACTIVE
GAME classifier
QUIET
GAME classifier
WAKE
GAME classifier
WAKE
GAME classifier
WAKE
GAME classifier
WAKE
GAME classifier
WAKE
GAME classifier
WAKE
GAME classifier
WAKE
GAME classifier
WAKE 4
GAME classifier
ACTIVE 4
GAME classifier
QUIET 4
{0.2, 0.6, 0.45, 0.8}
{0.99, 0.98, 1, 0.99}
{0.99, 0.01, 1, 1}
MUL:
AVG:
STD: high low high

11. Fuzzy prediction Unclear, guess WAKE Definitely not ACTIVE Definitely
Testing example 1
Testing example 2
Testing example 3
Testing example 4
Unclear,
guess WAKE
Definitely
not ACTIVE
Definitely
not WAKE, QUIET
Most probably ACTIVE
Totally unclear,
guess ACTIVE,
probably not QUIET
Clear WAKE,
not ACTIVE,QUIET

12. WAKE
AVG:
MUL:

13. QUIET
AVG:
MUL:

14. ALL
AVG:
MUL:

15. Conclusion Credibility of classification
Some examples cannot be correctly classified, based on information contained in training data
We propose the approach, how to identify these examples
FAKE GAME project demostration