1. Investigating the role of individual neurons as outlier detectors
Carlos López-Vázquez
Laboratorio LatinGEO
SGM+Universidad ORT del Uruguay
September 15th, 2015

2. Agenda Motivation for Outlier detection stage ANN as a regression tool
Formulation of the rule
Case 1 of application: small dataset
Case 2 of application: large dataset

3. Why to worry with outliers?
Outliers are unusual (in some sense) events
Might adversely affect further calculations OR
Might be the most valuable result!
Usually ANN produces an output given an input
Always!
What about the consequences?
We might want to detect spurious inputs

4. Example #1: Medical
From Lucila Ohno-Machado, 2004
Given some inputs, detect/classify a possible coronary disease
Lucila Ohno-Machado
Decision Systems Group
Brigham and Women’s Hospital
Department of Radiology

5. Myocardial Infarction Network
Duration
Intensity
Elevation
Pain
Pain
ECG: ST
Smoker
Age
Male
Answer: just a number
y=“Probability” of MI
0.8
No room for I DON'T KNOW!

6. Example #2: Autonomous Land Vehicle
NN learns to steer an autonomous vehicle.
960 input units, 4 hidden units, 30 output units
Driving at speeds up to 70 miles per hour
ALVINN System
Image of a
forward -
mounted
camera
From
bi.snu.ac.kr/Courses/g-video12s/files/NN_suppl.ppt
Biointelligence Laboratory
Department of Computer Engineering
Seoul National University
Weight values
for one of the
hidden units

7. Coming soon?

8. Goal Identify unlikely coming events
And thus (maybe) refuse to estimate outputs!
Supplement ANN answer (numerical, categorical) with some credibility flag
How?
Showing unlikely events during training (supervised)
Relying on already trained ANN (unsupervised)

9. Multi Layer Perceptron (MLP)
y=18.4*v1-22.1*v2+10.2*v3
y=10.4*v1+5.12*v2+8.9*v3
y=20.2*v1+0.18*v2-9.1*v3 X1
adjustable
weights
18.4
-22.1
10.2
10.4
5.12
8.9
20.2
0.18
-9.1 X2 v1 X3 v2 y v3 X4 X5

10. Why weights are so different?
Conjecture:
It might denote a specific role for the neuron
Such role can be connected to outliers
Wow! Which one are candidates?
Large weights? Small weights?
Preliminary analysis suggested that
Large WeightsOutlier detectors
But... convince me!

11. Two different problems
1) Does the rule indeed works?
If so:
2) How it performs when compared with other outlier detection procedures?

12. Example #3: Iris Flower Classification
3 species of Iris – SETOSA, VERSICOLOR, VIRGINICA
Each flower has parts called sepal & petal
Length and Width of sepal & petal can be used to determine iris type
Data collected on large number of iris flowers
For example, in one flower petal length=6.7mm and width=4.3mm also sepal length=22.4mm & sepal width =62.4mm. Iris type was SETOSA
An ANN can be trained to determine specie of iris for given set of petal and sepal width and length

13. Iris training and testing data
Sepal Length
Sepal Width
Petal Length
Petal Width
Iris Class
0.224
0.624
0.067
0.043
Setosa
0.749
0.502
0.627
0.541
Veracolor
0.557
0.847
1.000
Virginica
0.110
0.051
0.722
0.459
0.663
0.584
0.776
0.416
0.831
0.196
0.667
0.612
0.333
0.812
0.875
0.055
0.082
0.165
0.208
0.592
0.027
0.376
0.639
0.498
0.710
0.306
0.086
0.000
0.424
0.694
0.792
0.137

14. Classification using regression
Somewhat unusual paper: used regression with a single output instead of the common three binary outputs!
Unusual paper: internal weights of the ANN were published!
petal width
petal length
sepal width
sepal length v1 v2 y v3
From Benítez et al., 1997

15. Classification using regression
From Benítez et al., 1997
petal width
petal length
sepal width
sepal length v1 v2 y v3
The ANN can be simplified...

16. Pruned ANN and the classification is still good despite not exact
sepal length
sepal width
petal length
petal width
Which role had the other two?

17. All misclassifications now announced by z=1!
Modified version
z=“credibility flag”
y=2.143v3
All misclassifications now announced by z=1!

18. Example #4: daily rain dataset
Weather records typically have missing values
Many applications require complete databases
Well established linear methods for interpolate spatial observations exist
Their performance is poor for daily rain records
Why not ANN?

19. Data and test area description
30 years of daily records for 10 stations were available
30 % of the events have missing values
More than 80% of the readings are of zero rain, evenly distributed in the year
Annual averages ranges from 1600 to 500 mm/day; time correlation is low

23. Non-linear interpolants: ANN
We used ANN as interpolators, with 9 inputs and 1 output
The training was performed with one third of the dataset using backpropagation and minimizing the RMSE
Some different architectures were considered (one and two hidden layers; different number of neurons, etc.) as well as some transformations of the data

24. Skipping other details…
We applied our rule to each of the 10 ANN
Run a Monte Carlo experiment, seeding known outliers at random and locating them afterwards
Thorough comparison against state-of-the-art alternatives (details in the paper)
The ANN-based outlier detection tool performed very well
Best, when outlier size (Mozilla effect) was ignored
Satisfactory otherwise

25. Pros… Training stage is as usual; no special routine is required
We inspect the internal weights; no need to retraining
Unsupervised classifications: outliers are not declared as such in advance
Might offer an objective criteria to suspect underfitting

26. Cons…
Weights might be sensible to outliers (masking effect) which in turn might prevent to detect them
Which outliers are located? Only some suitable ones?

27. SGM+Universidad ORT del Uruguay
Questions?
Carlos López-Vázquez
Laboratorio LatinGEO
SGM+Universidad ORT del Uruguay
September 15th, 2015