1. Using Clustering to Make Prediction Intervals For Neural Networks
Claus Benjaminsen
ECE539 - final project fall 2005

2. What is a prediction interval?
An interval within which the true target value is predicted to be
Prediction intervals are often defined by
Interval end values and an associated probability
A Gaussian distribution with a certain mean and variance

3. Motivation Give user a more informative output
Precision of prediction (width of interval)
Max and min limits (end values of interval)
Certainty of prediction (associated probability)
Make user able to make better decisions

4. Approach 1 Use clustering to group training feature vectors
Associate the mean training error of all the features within each cluster to the corresponding cluster center
Estimate prediction interval for all training features in a cluster by scaling the associated error

5. Approach 2
Given a new input its membership to one of the cluster centers is determined
The prediction interval for the new input is then the same as for the training data belonging to that cluster

6. Data set used Synthetic data set 1 input feature 1 output target
256 samples
Divided into:
156 training samples
100 testing samples

7. Results
Plot of the test data along with prediction intervals estimated by the clustering method

8. Results 2 cost_test_clus 0.001277
The cost function is calculated as the mean squared distance from the edge of the prediction interval to the test target for all targets not inside prediction intervals
The clustering method has the best performance of the three different methods
cost_test_clus
cost_test_var
cost_test_basis

9. Discussion
Model can become big, when input feature space is big and the number of training samples is high
Problems when new inputs doesn’t “look” like any of the training inputs

10. Conclusion
The clustering method can estimate prediction intervals, which yields a very good performance
Data used is very well suited for clustering method – might not show same good result for other types of datasets
This will have to be tested in the future!