1. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 1 FORECASTING USING NON-LINEAR TECHNIQUES IN TIME SERIES ANALYSIS AN OVERVIEW OF RELATED TECHNIQUES AND MAIN ISSUES Michel Camilleri

2. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 2 EARLY FORECASTING Maltese Stone Age Hunter–Gatherer used Mnajdra to forecast seasons (among other purposes)

3. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 3 FORECASTING TODAY Non linear time series techniques are being used to to forecasting sun spot activity (among other uses) Data/Images acquired at EOS/OCS, CIF-US, Universidad de Sonora, Mexico. Observer(s): M.C. Marianna Lyubarets

4. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 4 APPLICATION AREAS  MEDICAL  MILITARY  MANAGEMENT  FINANCE  ASTRONOMY  DEMOGRAPY …………

5. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 5 TIME SERIES TECHNIQUES LINEARVS NON LINEAR

6. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 6 LINEAR TECHNIQUES  Linear methods try to model closely underlying subsystems  Require identification & measurement of several system features - seasons, trends, cycles, outliers

7. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 7 NON LINEAR TECHNIQUES  Non-linear techniques exploit measurement data and computer power:  Mimic dynamic system without having to understand exactly the underlying processes  Better results than Linear in certain areas

8. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 8 BASIC STEPS TO FORECASTING  COLLECT DATA  EXAMINE DATA  PREPROCESS DATA  OPTIMIZE PARAMETERS  APPLY PREDICTION TECHNIQUES  MEASURE PREDICTION ERROR  REVIEW AND UPDATE

9. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 9 A PRACTICAL EXAMPLE  CREATE OWN DATA SET (3000 pts) WITH RANDOM NOISE  SEPARATE TRAINING SET, ATTRACTOR, FUTURE (HIDDEN SET)  EXAMINE DATA  PREPARE DATA  PREDICT  MEASURE SUCCESS OF PREDICTION  OPTIMISE PARAMETERS

10. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 10 DATA CREATION Created function X(t) vs t Where t = DISCRETE VALUES OF TIME (1..3000) And X(t) = A1 * SINE (t * F1) + A2 * COS (t * F2) * Random () * N Amplitude A1 = 0.1, Frequency F1 = 5 Amplitude A2 = 0.2, Frequency F2 = 0.33 Noise factor N = 3

11. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 11 UNDERLYING SUBSYSTEMS SINE FUNCTION + COSINE FUNCTION

12. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 12 MEASUREABLE SIGNAL SUBSYSTEMS + NOISE

13. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 13 SEPARATE THE DATA

14. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 14 FUTURE SET (HIDDEN FROM SYSTEM)

15. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 15 EXAMINE DATA  VISUAL INSPECTION  STATIONARITY  PHASE SPACE MAPPING  AUTOCORRELATION  LYAPUNOV EXPONENT  DELAY SPACE EMBEDDING  MINIMAL EMBEDDING DIMENSION

16. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 16 PHASE STATE

17. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 17 PHASE SPACE MAP

18. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 18 AUTO CORRELATION SUM

19. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 19 MAX LYAPUNOV EXPONENT

20. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 20 TIME DELAY EMBEDDING THE ATTRACTOR DIMENSIONS = 100 TIME DELAY = 1PREDICTOR POINT

21. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 21 PREPROCESSING DATA  FILTERING  NOISE REDUCTION  TEMPORAL ABSTRACTIONS  CATEGORIZE ETHERNET PACKETS BY SIZE  CATEGORIZE ECG SIGNALS BY TYPE

22. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 22 NON LINEAR NOISE REDUCTION Noise reduced by 8 %

23. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 23 APPLY PREDICTION TECHNIQUE  Set initial parameters –Time delay, dimensions, distance, box size, number of future steps ahead  Choose measure of success and apply it to output (Various)  Find optimal set of parameters

24. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 24 COMPARE ATTRACTOR ALONG TRAINING SET

25. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 25 FINDING A NEIGHBOUR

26. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 26 FIND ALL NEIGHBOURS OF SELECTED POINT Neigbour Time point Neighbor 1252391769 22711101770 3447111956 41013121958 51768132145 61203142334 71392152335 81581 ID 9, M=9 Err= 2 NEIGHBOURS FOUND 15

27. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 27 FIND PREDICTED SET FOR NEIGHBOUR

28. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 28 AVERAGE of PREDICTION SETS PREDICTION SETS OF ALL NEIGHBOURS FINAL PREDICTION

29. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 29 FIRST PREDICTION ATTEMPT

30. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 30 NEED TO VARY PARAMETERS I

31. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 31 EXAMINE MORE CLOSELY I

32. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 32 A BETTER ATTRACTOR Time Delay = 9, Dimensions = 9

33. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 33 A BETTER PREDICTION Delay=9,Dim=9,Err=2 neighb=15,rms = 1.09

34. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 34 CHANGE DELAY, DIMENSIONS Delay=1,Dim=20,Err=2 neighb=1,rms = 1.37

35. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 35 CHANGE DISTANCE Delay=1,Dim=20,Err=3 neighb=34,rms = 1.09

36. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 36 PROCESSING CONSIDERATIONS  Multiple attempts at prediction,calculation of invariants, noise reduction, require increasing orders of operations  Each operation may require comparison of every point on attractor with respective points for each training point.  Number of operations to find neighbours can be reduced by comparing attractor only to points in same phase state e.g. Box or Tree assisted neighbour search in Phase space.

37. Forecasting using Non Linear Techniques in Time Series Analysis – Michel Camilleri – September 2004 37 THE END (AS FORECAST)