1. Models for Non-Stationary Time Series The ARIMA(p,d,q) time series

2. Many non-stationary time series can be converted to a stationary time series by taking d th order differences.

3. Let {x t |t  T} denote a time series such that {w t |t  T} is an ARMA(p,q) time series where w t =  d x t = (I – B) d x t = the d th order differences of the series x t. Then {x t |t  T} is called an ARIMA(p,d,q) time series (an integrated auto-regressive moving average time series.)

4. The equation for the time series {w t |t  T} is:  (B)w t =  +  (B)u t or  (B) x t =  +  (B)u t.. Where  (B) =  (B)  d =  (B)(I - B) d The equation for the time series {x t |t  T} is:  (B)  d x t =  +  (B)u t

5. Suppose that d roots of the polynomial  (x) are equal to unity then  (x) can be written:  (B) = (1 -  1 x -  2 x 2 -... -  p x p )(1-x) d. and  (B) could be written:  (B) = (I -  1 B -  2 B 2 -... -  p B p )(I-B) d =  (B)  d. In this case the equation for the time series becomes:  (B)x t =  +  (B)u t or  (B)  d x t =  +  (B)u t..

6. Comments: 1.The operator  (B) =  (B)  d = 1 -  1 x -  2 x 2 -... -  p+d x p+d is called the generalized autoregressive operator. (d roots are equal to 1, the remaining p roots have |r i | > 1) 2. The operator  (B) is called the autoregressive operator. (p roots with |r i | > 1) 3. The operator  (B) is called moving average operator.

7. Example – ARIMA(1,1,1) The equation: (I –  1 B)(I – B)x t =  + (I +  1 )u t (I – (1 +  1 ) B +  1 B 2 )x t =  + u t +  1 u t - 1 x t – (1 +  1 ) x t-1 +  1 x t-2 =  + u t +  1 u t – 1 or x t = (1 +  1 ) x t-1 –  1 x t-2 +  + u t +  1 u t – 1

8. Modeling of Seasonal Time Series

9. If a time series, {x t : t  T},that is seasonal we would expect observations in the same season in adjacent years to have a higher auto correlation than observations that are close in time (but in different seasons. For example for data that is monthly we would expect the autocorrelation function to look like this

10. The AR(1) seasonal model This model satisfies the equation: The autocorrelation for this model can be shown to be: This model is also an AR(12) model with  1 = … =  11 = 0

11. Graph:  (h)

12. The AR model with both seasonal and serial correlation This model satisfies the equation: The autocorrelation for this model will satisfy the equations: This model is also an AR(13) model.

13. The Yule-Walker Equations:

14. or:

15. Some solutions for  h

16. Excel file for determining Autocorrelation functionExcel

17. The general ARIMA model incorporating seasonality where

18. Prediction

19. Three Important Forms of a Non-Stationary Time Series

20. The Difference equation Form: x t =  1 x t-1 +  2 x t-2 +... +  p+d x t-p-d +  + u t +  1 u t-1 +  2 u t-2 +...+  q u t-q  (B)  d x t =  +  (B)u t

21. The Random Shock Form: x t =  (t) + u t  +  1 u t-1 +  2 u t-2 +  3 u t-3 +... x t =  (t) +  (B)u t

22. The Inverted Form: x t =  1 x t-1 +  2 x t-2 +  3 x t-3 +...  +  + u t  (B)x t =  + u t

23. Example Consider the ARIMA(1,1,1) time series (I – 0.8B)  x t = (I + 0.6B)u t (I – 0.8B) (I –B) x t = (I + 0.6B)u t (I – 1.8B + 0.8B 2 ) x t = (I + 0.6B)u t x t = 1.8 x t - 1 - 0.8 x t - 2 + u t + 0.6u t -1 Difference equation form

24. (I – 1.8B + 0.8B 2 ) x t = (I + 0.6B)u t The random shock form x t = (I – 1.8B + 0.8B 2 ) -1 (I + 0.6B)u t x t = (I + 2.4B + 3.52B 2 + 4.416B 3 + 5.1328B 4 + … )u t x t = u t + 2.4 u t - 1 + 3.52 u t - 2 + 4.416 u t - 3 + 5.1328 u t - 4 + …

25. (I – 1.8B + 0.8B 2 ) x t = (I + 0.6B)u t The Inverted form (I + 0.6B) -1 (I – 1.8B + 0.8B 2 )x t = u t (I - 2.4B + 2.24B 2 – 1.344 B 3 + 0.8064B 4 +… )x t = u t x t = 2.4 x t - 1 - 2.24 x t - 2 + 1.344 x t - 3 - 0.8064 x t - 4 + … + u t

26. Forecasting an ARIMA(p,d,q) Time Series Let P T denote {…, x T-2, x T-1, x T } = the “past” until time T. Then the optimal forecast of x T+l given P T is denoted by: This forecast minimizes the mean square error

27. Three different forms of the forecast 1.Random Shock Form 2.Inverted Form 3.Difference Equation Form Note:

28. Random Shock Form of the forecast Recall x t =  (t) + u t  +  1 u t-1 +  2 u t-2 +  3 u t-3 +... x T+l =  (T + l) + u T+l +  1 u T+l-1 +  2 u T+l-2 +  3 u T+l-3 +... or Taking expectations of both sides and using

29. To compute this forecast we need to compute {…, u T-2, u T-1, u T } from {…, x T-2, x T-1, x T }. Note: x t =  (t) + u t +  1 u t-1 +  2 u t-2 +  3 u t-3 +... Thus Which can be calculated recursively and

30. The Error in the forecast: The Mean Sqare Error in the Forecast Hence

31. Prediction Limits for forecasts (1 –  )100% confidence limits for x T+l

32. The Inverted Form:  (B)x t =  + u t or x t =  1 x t-1 +  2 x t-2 +  3 x 3 +... +  + u t where  (B) = [  (B)] -1  (B) = [  (B)] -1 [  (B)  d ] = I -  1 B -  2 B 2 -  3 B 3 -...

33. The Inverted form of the forecast x t =  1 x t-1 +  2 x t-2 +... +  + u t and for t = T+l x T+l =  1 x T+l-1 +  2 x T+l-2 +...  +  + u T+l Taking conditional Expectations Note:

34. The Difference equation form of the forecast x T+l =  1 x T+l-1 +  2 x T+l-2 +...  +  p+d x T+l-p-d +  + u T+l +  1 u T+l-1 +  2 u T+l-2 +... +  q u T+l-q Taking conditional Expectations

35. Example: ARIMA(1,1,2) The Model: x t - x t-1 =  1 (x t-1 - x t-2 ) + u t +  1 u t +  2 u t or x t = (1 +  1 )x t-1 -  1 x t-2 + u t +  1 u t +  2 u t or  (B)x t =  (B)(I-B)x t =  (B)u t where  (x) = 1 - (1 +  1 )x +  1 x 2 = (1 -  1 x)(1-x) and  (x) = 1 +  1 x +  2 x 2.

36. The Random Shock form of the model: x t =  (B)u t where  (B) = [  (B)(I-B)] -1  (B) = [  (B)] -1  (B) i.e.  (B) [  (B)] =  (B). Thus (I +  1 B +  2 B 2 +  3 B 3 +  4 B 4 +... )(I - (1 +  1 )B +  1 B 2 ) = I +  1 B +  2 B 2 Hence  1 =  1 - (1 +  1 ) or  1 = 1 +  1 +  1.  2 =  2 -  1 (1 +  1 ) +  1 or  2 =  1 (1 +  1 ) -  1 +  2. 0 =  h -  h-1 (1 +  1 ) +  h-2  1 or  h =  h-1 (1 +  1 ) -  h-2  1 for h ≥ 3.

37. The Inverted form of the model:  (B) x t = u t where  (B) = [  (B)] -1  (B)(I-B) =  B)] -1  (B) i.e.  (B) [  (B)] =  (B). Thus (I -  1 B -  2 B 2 -  3 B 3 -  4 B 4 -... )(I +  1 B +  2 B 2 ) = I - (1 +  1 )B +  1 B 2 Hence -(1 +  1 ) =  1 -  1 or  1 = 1 +  1 +  1.  1 = -  2 -  1  1 +  2 or  2 = -  1  1 -  1 +  2. 0 =  h -  h-1  1 -  h-2  2 or  h = -(  h-1  1 +  h-2  2 ) for h ≥ 3.

38. Now suppose that  1 = 0.80,  1 = 0.60 and  2 = 0.40 then the Random Shock Form coefficients and the Inverted Form coefficients can easily be computed and are tabled below:

39. The Forecast Equations

40. The Difference Form of the Forecast Equation

41. Computation of the Random Shock Series, One- step Forecasts One-step Forecasts Random Shock Computations

42. Computation of the Mean Square Error of the Forecasts and Prediction Limits Mean Square Error of the Forecasts Prediction Limits

43. Table:  MSE of Forecasts to lead time l = 12 (  2 = 2.56)

44. Raw Observations, One-step Ahead Forecasts, Estimated error, Error

45. Forecasts with 95% and 66.7% prediction Limits

46. Graph: Forecasts with 95% and 66.7% Prediction Limits