Local Polynomial Method for Ensemble Forecast of Time Series Satish Kumar Regonda, Balaji Rajagopalan, Upmanu Lall, Martyn Clark, and Young-II Moon Upmanu Lall, Martyn Clark, and Young-II Moon Hydrology Days 2005 Colorado State University, Fort Collins, CO

Stochastic models (AR, ARMA,………) Stochastic models (AR, ARMA,………) Presume time series of a response variable as a realization of a random process Presume time series of a response variable as a realization of a random process x t =f(x t-1,x t-2,….x t-k ) + e t Noise, finite data length, high temporal and spatial variation of the data influences estimation of “k” Noise, finite data length, high temporal and spatial variation of the data influences estimation of “k” Randomness in the system limits the predictability which could be Randomness in the system limits the predictability which could be A result of many independent and irreducible degrees of freedom A result of many independent and irreducible degrees of freedom Due to deterministic chaos Due to deterministic chaos Time series modeling

What is deterministic Chaos? Three coupled non-linear differential equations Three coupled non-linear differential equations System apparently seems erratic, complex, and almost random (and that are very sensitive to initial conditions), infact, the system is deterministic. System apparently seems erratic, complex, and almost random (and that are very sensitive to initial conditions), infact, the system is deterministic. Lorenz Attractor

Chaotic systems Logistic Equation: Xn+1 =A* Xn*(1-Xn) ‘A’ is constant ‘X n ’ is Current Value ‘X n+1 ’ is Future Value How these will be predicted? - “nonlinear dynamical based time series analysis”

Nonlinear Dynamics Based Forecasting procedure 1. x t =  x 1,x 2,x 3,…,x n  2. State space reconstruction (or dynamics recovery) using ‘m’ and ‘  ’ 3. Forecast for T time steps into future i.e., x t+T = f (X t ) +  t X t is a feature vector X t is a feature vector f is a linear or nonlinear function f is a linear or nonlinear function m = 3 and  = 2

Local Map f Forecast for ‘T’ time step into the future Forecast for ‘T’ time step into the future x t+T = f (X t ) +  t Typically, f(.) estimated locally within neighborhood of the feature vector Typically, f(.) estimated locally within neighborhood of the feature vector f (.) approximated using locally weighted polynomials defined as LOCFIT f (.) approximated using locally weighted polynomials defined as LOCFIT Polynomial order ‘p’ Polynomial order ‘p’ Number of neighbors K ( =  *n,  is fraction between (0,1] ) Number of neighbors K ( =  *n,  is fraction between (0,1] )

M is estimated using correlation dimension (Grassberger and Procaccio..xx) M is estimated using correlation dimension (Grassberger and Procaccio..xx) or False Neighbors (Kennel…xx) Tau is estimated via Mutual Information (Sweeney, xx; Moon et al., 19xx) Tau is estimated via Mutual Information (Sweeney, xx; Moon et al., 19xx) Estimation of m and tau..

In real data, due to Noise (sampling and dynamical) the phase space parameters (i.e., Embedding dimension and Delay time) are not uniquely estimated. In real data, due to Noise (sampling and dynamical) the phase space parameters (i.e., Embedding dimension and Delay time) are not uniquely estimated. Hence, a suite of plausible parameters of the state space i.e. D, , , p. Hence, a suite of plausible parameters of the state space i.e. D, , , p. Need for Ensembles..

Cont’d Forecast for ‘T’ time step into the future Forecast for ‘T’ time step into the future x t+T = f (X t ) +  t Typically, f(.) estimated locally within neighborhood of the feature vector Typically, f(.) estimated locally within neighborhood of the feature vector f (.) approximated using locally weighted polynomials defined as LOCFIT f (.) approximated using locally weighted polynomials defined as LOCFIT Polynomial order ‘p’ Polynomial order ‘p’ Number of neighbors K ( =  *n,  is fraction between (0,1] ) Number of neighbors K ( =  *n,  is fraction between (0,1] )

Cont’d General Cross Validation (GCV) is used to select the optimal parameters (  and p) General Cross Validation (GCV) is used to select the optimal parameters (  and p) Optimal parameter is the one that produces minimum GCV Optimal parameter is the one that produces minimum GCV e i is the error n-number of data points m-number of parameters

Forecast algorithm 1. Compute D and  using the standard methods and choose a broad range of D and  values. 2. Reconstruct phase space for selected parameters 3. Calculate GCV for the reconstructed phase space by varying smoothening parameters of LOCFIT 4. Repeat steps 2 and 3 for all combinations of D, , , p 5. Select a suite of “best” parameter combinations that are within 5 percent of the lowest GCV 6. Each selected best combination is then used to generate a forecast.

Applications Synthetic data Synthetic data The Henon system The Henon system The Lorenz system The Lorenz system Geophysical data Geophysical data The Great Salt Lake (GSL) The Great Salt Lake (GSL) NINO3 NINO3

Time series Henon X- ordinate GSL Lorenz X- ordinate NINO3

Synthetic Data The Lorenz system The Lorenz system A time series of 6000 observations generated A time series of 6000 observations generated Embedding dimension: 2.06 and 3.0 Embedding dimension: 2.06 and 3.0 Training period: 5500 observations Training period: 5500 observations Selected parameters: D = 2 & 3,  = 1 & 2, and, p = 2 with various neighbor sizes (  ). Selected parameters: D = 2 & 3,  = 1 & 2, and, p = 2 with various neighbor sizes (  ). Forecasted 100 time steps into the future. Forecasted 100 time steps into the future. Predictability less than the Henon system ( large lyapunov exponent) Predictability less than the Henon system ( large lyapunov exponent)

Blind Prediction Index 5368Index 5371 Unstable RegionStable Region ~ 3 to 5 time steps~ 35 points

The Great Salt Lake of Utah (GSL) It is the fourth largest, perennial, closed basin, saline lake in the world. It is the fourth largest, perennial, closed basin, saline lake in the world. Biweekly observations Biweekly observations Superposition of strong and recurrent climate patterns at different timescales created a tough job of prediction for classical time series models. Superposition of strong and recurrent climate patterns at different timescales created a tough job of prediction for classical time series models. Closed basin-----integrates hydrologic response and filters out high-frequency phenomenon and results into low dimensional phenomena. Closed basin-----integrates hydrologic response and filters out high-frequency phenomenon and results into low dimensional phenomena. GSL – is a low dimensional chaotic system (Sangoyomi et al. 1996) GSL – is a low dimensional chaotic system (Sangoyomi et al. 1996)

GSL Attractor Annual cycle is approximately motion around the smaller radius of the ‘spool’ Annual cycle is approximately motion around the smaller radius of the ‘spool’ Longer term motion which has larger amplitude moves the orbits along the longer axis of the ‘spool’ Longer term motion which has larger amplitude moves the orbits along the longer axis of the ‘spool’

Results Embedding dimension: 4 ; delay time: 14 Embedding dimension: 4 ; delay time: 14 GCV values computed over D = 2 to 6 and  = 10 to 20, and p = 1 to 2, with various neighbor sizes GCV values computed over D = 2 to 6 and  = 10 to 20, and p = 1 to 2, with various neighbor sizes Fall of the lake volume Fall of the lake volume D = 4 & 5,  = 10,14, &15, p = 1 &2,  = D = 4 & 5,  = 10,14, &15, p = 1 &2,  = Rise of the lake volume Rise of the lake volume D = 4 & 5,  = 10 &15, p = 2,  = D = 4 & 5,  = 10 &15, p = 2,  =

Fall of the lake volume Blind Prediction

Rise of the lake volume Blind Prediction

NINO3 Time series of averaged monthly SST anomalies in the tropical Pacific covering the domain of 4 o N-4 o S and 90 o -150 o W Time series of averaged monthly SST anomalies in the tropical Pacific covering the domain of 4 o N-4 o S and 90 o -150 o W Monthly observations from 1856 onwards Monthly observations from 1856 onwards ENSO characteristics (e.g. onset, termination, cyclic nature, partial locking to seasonal cycle, and irregularity) explained presuming system as low order chaotic system (embedding dimension 3.5; Tziperman et al and 1995) ENSO characteristics (e.g. onset, termination, cyclic nature, partial locking to seasonal cycle, and irregularity) explained presuming system as low order chaotic system (embedding dimension 3.5; Tziperman et al and 1995)

Results El Nino Events (1982 and 1997): El Nino Events (1982 and 1997): : D = 4 and  = : D = 4 and  = : D = 5 and  = : D = 5 and  = 13 Selected parameters range: D = 2 to 5,  = 11 to 21 ( 8 to 16), p = 1& 2,  = 0.1 – 1.0 Selected parameters range: D = 2 to 5,  = 11 to 21 ( 8 to 16), p = 1& 2,  = 0.1 – 1.0 Forecasted issued in different months of the event Forecasted issued in different months of the event Ensemble prediction did a slightly better job compared to best AR-model Ensemble prediction did a slightly better job compared to best AR-model

El Nino Blind Prediction

Results La Nina events (1984 and 1989 ) La Nina events (1984 and 1989 ) Both events yielded a dimension and delay time of 5 and 17 respectively. Both events yielded a dimension and delay time of 5 and 17 respectively. Selected parameters range: D = 2 to 5,  = 12 to 22, p = 1& 2,  = 0.1 – 1.0 Selected parameters range: D = 2 to 5,  = 12 to 22, p = 1& 2,  = 0.1 – 1.0 Forecasted issued in different months Forecasted issued in different months Both, ensemble and AR, methods performed similarly with increasing skill of the predictions when issued closer to the negative peak of the events Both, ensemble and AR, methods performed similarly with increasing skill of the predictions when issued closer to the negative peak of the events

La Nina Blind Prediction

Recent prediction May 1, 2002July, 2004 (GSL) (NINO3)

Summary A new algorithm proposed which selects a suite of ‘best’ parameters that captures effectively dynamics of the system A new algorithm proposed which selects a suite of ‘best’ parameters that captures effectively dynamics of the system Ensemble forecasts provide Ensemble forecasts provide A natural estimate of the forecast uncertainty A natural estimate of the forecast uncertainty The pdf of the response variable and consequently threshold exceedance probabilities The pdf of the response variable and consequently threshold exceedance probabilities Decision makers will be benefited as forecast issued with a good lead-time Decision makers will be benefited as forecast issued with a good lead-time Performs better than the best AR-model Performs better than the best AR-model It could be improved in several ways It could be improved in several ways

Acknowledgements Thanks to CADSWES at the Univ. of Colorado at Boulder for letting use of its’ computational facilities. Thanks to CADSWES at the Univ. of Colorado at Boulder for letting use of its’ computational facilities. Support from NOAA grant NA17RJ1229 and NSF grant EAR are thankfully acknowledged. Support from NOAA grant NA17RJ1229 and NSF grant EAR are thankfully acknowledged.

Publication: Publication: -Regonda, S., B. Rajagopalan, U. Lall, M. Clark and Y. Moon, Local polynomial method for ensemble forecast of time series, (in press) Nonlinear Processes in Geophysics, Special issue on "Nonlinear Deterministic Dynamics in Hydrologic Systems: Present Activities and Future Challenges", Thank You

dx / dt = a (y - x) dy / dt = x (b - z) - y dz / dt = xy - c z Lorenz Equations: Lorenz attractor derived from a simplified model of convection to see the effect of initial conditions. The system is most commonly expressed as 3 coupled non-linear differential equations, which are known as Lorenz Equations. a, b, and c are the constants

Chua Attractor (electronic circuit)Duffing (nonlinear Oscillator)

State space reconstruction Time series of observations Time series of observations x t =  x 1,x 2,x 3,………,x n  Embedding time series into ‘m’ dimensional phase space i.e., recovering dynamics of the system Embedding time series into ‘m’ dimensional phase space i.e., recovering dynamics of the system X t = {x t, x t+ , x t+ 2 ,…., x t+(m-1)  } (Takens, 1981) m – embedding dimension  - Delay time

(Lall et al. 1996)

Henon Lorenz D=3 Lorenz D=2, Tau=1 Lorenz D=3, Tau=1

Does GSL show Chaotic nature? Long term persistence of the climate variations i.e., Long term persistence of the climate variations i.e., because of superposition of strong, recurrent patterns at different scales makes difficult to justify classical, time series models because of superposition of strong, recurrent patterns at different scales makes difficult to justify classical, time series models Closed basin-----integrates hydrologic response and filters out high-frequency phenomenon and results into low dimensional phenomena. Closed basin-----integrates hydrologic response and filters out high-frequency phenomenon and results into low dimensional phenomena. GSL – is a low dimensional chaotic system (Sangoyomi et al. 1996) GSL – is a low dimensional chaotic system (Sangoyomi et al. 1996)

El Nino Blind Prediction

La Nina Blind Prediction

Nonlinear dynamics Based Time Series 1. State space reconstruction (Takens’ embedding theorem) Time series of a variable x t =  x 1,x 2,x 3,………,x n  X t = {x t, x t+ , x t+ 2 ,…., x t+(m-1)  } m – embedding dimension  - Delay time 2. Fit a function that maps different states in the phase space

U. Lalll, 1994 Cont’d Importance of  ‘m’ and ‘  ’

Chaotic behavior in geophysical processes Lower order chaotic behavior observed in various geophysical variables (e.g., rainfall, runoff, lake volume) on different scales Lower order chaotic behavior observed in various geophysical variables (e.g., rainfall, runoff, lake volume) on different scales Diagnostic tools Diagnostic tools Grassberger-Procaccia algorithm Grassberger-Procaccia algorithm False Nearest Neighbors False Nearest Neighbors Lyapunov Exponent Lyapunov Exponent

Results The Henon system The Henon system 4000 observations of x-ordinate 4000 observations of x-ordinate Embedding dimension 2 and delay time 1 Embedding dimension 2 and delay time 1 Training period: 3700 observations and searched over D = 1 to 5 and  = 1 to 10. Training period: 3700 observations and searched over D = 1 to 5 and  = 1 to 10. Selected combinations resulted 15 combinations and parameter values D = 2,  = 1, p=2 and with various neighborhood sizes (i.e.  ) Selected combinations resulted 15 combinations and parameter values D = 2,  = 1, p=2 and with various neighborhood sizes (i.e.  ) Forecasted 100 time steps into future Forecasted 100 time steps into future

Blind prediction Index 3701Index 3711 Ensemble forecast (5 th &95 th; 25 th & 75 th percentiles); Real observations; the best AR forecast