1. Prediction of a nonlinear time series with feedforward neural networks Mats Nikus Process Control Laboratory

2. The time series

3. A closer look

4. Another look

5. Studying the time series Some features seem to reapeat themselves over and over, but not totally ”deterministically” Lets study the autocovariance function

6. The autocovariance function

7. Studying the time series The autocovariance function tells the same: There are certainly some dynamics in the data Lets now make a phaseplot of the data In a phaseplot the signal is plotted against itself with some lag With one lag we get

8. Phase plot

9. 3D phase plot

10. The phase plots tell Use two lagged values The first lagged value describes a parabola Lets make a neural network for prediction of the timeseries based on the findings.

11. The neural network y(k+1) ^ y(k) y(k-1) Lets try with 3 hidden nodes 2 for the ”parabola” and one for the ”rest”

12. Prediction results

13. Residuals (on test data)

14. A more difficult case If the time series is time variant (i.e. the dynamic behaviour changes over time) and the measurement data is noisy, the prediction task becomes more challenging.

15. Phase plot for a noisy timevariant case

16. Residuals with the model

17. Use a Kalman-filter to update the weights We can improve the predictions by using a Kalman-filter Assume that the process we want to predict is described by

18. Kalman-filter Use the following recursive equations The gradient needed in C k is fairly simple to calculate for a sigmoidal network

19. Residuals

20. Neural network parameters

21. Henon series The timeseries is actually described by