1. Models for Non-Stationary Time Series
The ARIMA(p,d,q) time series

2. The ARIMA(p,d,q) time series
Many non-stationary time series can be converted to a stationary time series by taking dth order differences.

3. Let {xt|t  T} denote a time series such that
{wt|t  T} is an ARMA(p,q) time series where
wt = Ddxt
= (I – B)dxt
= the dth order differences of the series xt.
Then {xt|t  T} is called an ARIMA(p,d,q) time series (an integrated auto-regressive moving average time series.)

4. The equation for the time series {wt|t  T} is:
b(B)wt = d + a(B)ut
The equation for the time series {xt|t  T} is:
b(B)Ddxt = d + a(B)ut or
f(B) xt = d + a(B)ut..
Where
f(B) = b(B)Dd = b(B)(I - B) d

5. Suppose that d roots of the polynomial f(x) are equal to unity then f(x) can be written:
f(B) = (1 - b1x - b2x bpxp)(1-x)d.
and f(B) could be written:
f(B) = (I - b1B - b2B bpBp)(I-B)d= b(B)Dd.
In this case the equation for the time series becomes:
f(B)xt = d + a(B)ut or
b(B)Dd xt = d + a(B)ut..

6. Comments: The operator f(B) =b(B)Dd = 1 - f1x - f2x2 -... - fp+dxp+d
is called the generalized autoregressive operator. (d roots are equal to 1, the remaining p roots have |ri| > 1)
2. The operator b(B) is called the autoregressive operator. (p roots with |ri| > 1)
3. The operator a(B) is called moving average operator.

7. Example – ARIMA(1,1,1) The equation:
(I – b1B)(I – B)xt = d + (I + a1)ut
(I – (1 + b1) B + b1B2)xt = d + ut + a1 ut - 1
xt – (1 + b1) xt-1 + b1xt-2 = d + ut + a1 ut – 1 or
xt = (1 + b1) xt-1 – b1xt-2 + d + ut + a1 ut – 1

8. Modeling of Seasonal Time Series

9. If a time series, {xt: t  T},that is seasonal we would expect observations in the same season in adjacent years to have a higher auto correlation than observations that are close in time (but in different seasons.
For example for data that is monthly we would expect the autocorrelation function to look like this

10. The AR(1) seasonal model
This model satisfies the equation:
The autocorrelation for this model can be shown to be:
This model is also an AR(12) model with
b1 = … = b11 = 0

11. Graph: r(h)

12. The AR model with both seasonal and serial correlation
This model satisfies the equation:
This model is also an AR(13) model.
The autocorrelation for this model will satisfy the equations:

13. The Yule-Walker Equations:

14. or:

15. Some solutions for rh

16. Excel file for determining Autocorrelation function

17. The general ARIMA model incorporating seasonality
where

18. Prediction

19. Three Important Forms of a Non-Stationary Time Series

20. xt = f1xt-1 + f2xt-2 +... +fp+dxt-p-d
The Difference equation Form:
xt = f1xt-1 + f2xt fp+dxt-p-d
+ d + ut +a1ut-1 + a2ut aqut-q
f (B) xt = b(B)Ddxt = d + a(B)ut
f (B) = b(B)Dd = b(B) (I - B)d

21. xt = m(t) +y(B)ut The Random Shock Form:
xt = m(t) + ut +y1ut-1 + y2ut-2 +y3ut
xt = m(t) +y(B)ut
y(B) = I + y1B + y2B
= [f(B)]-1a(B)
the coeficients) y1, y2 ... are used to calculate the MSE of forecasts

22. The Inverted Form:
xt = p1xt-1 + p2xt-2 +p3xt t + ut
p(B)xt = t + ut
p(B) = I - p1B - b2B
= [a(B)]-1 f(B)

23. Difference equation form
Example
Consider the ARIMA(1,1,1) time series
(I – 0.8B)Dxt = (I + 0.6B)ut
(I – 0.8B) (I –B) xt = (I + 0.6B)ut
(I – 1.8B + 0.8B2) xt = (I + 0.6B)ut
xt = 1.8 xt xt ut+ 0.6ut -1
Difference equation form

24. The random shock form
(I – 1.8B + 0.8B2) xt = (I + 0.6B)ut
xt = (I – 1.8B + 0.8B2)-1(I + 0.6B)ut
xt = (I + 2.4B B2 +
4.416B B4 + … )ut
xt = ut ut ut ut ut …

25. The Inverted form
(I – 1.8B + 0.8B2) xt = (I + 0.6B)ut
(I + 0.6B)-1(I – 1.8B + 0.8B2)xt = ut
(I - 2.4B B2 – B B4 +… )xt = ut
xt = 2.4 xt xt xt xt … + ut

26. Forecasting an ARIMA(p,d,q) Time Series
Let PT denote {…, xT-2, xT-1, xT} = the “past” until time T.
Then the optimal forecast of xT+l given PT is denoted by:
This forecast minimizes the mean square error

27. Three different forms of the forecast
Random Shock Form
Inverted Form
Difference Equation Form
Note:

28. Random Shock Form of the forecast
Recall
xt = m(t) + ut +y1ut-1 + y2ut-2 +y3ut or
xT+l = m(T + l) + uT+l +y1uT+l-1 + y2uT+l-2 +y3uT+l
Taking expectations of both sides and using

29. To compute this forecast we need to compute
{…, uT-2, uT-1, uT} from {…, xT-2, xT-1, xT}.
Note:
xt = m(t) + ut +y1ut-1 + y2ut-2 +y3ut
Thus
and
Which can be calculated recursively

30. The Error in the forecast:
The Mean Sqare Error in the Forecast
Hence

31. Prediction Limits for forecasts
(1 – a)100% confidence limits for xT+l

32. The Inverted Form:
p(B)xt = t + ut or
xt = p1xt-1 + p2xt-2 +p3x3+ ...
+ t + ut
where
p(B) = [a(B)]-1f(B) = [a(B)]-1[b(B)Dd]
= I - p1B - p2B2 - p3B

33. The Inverted form of the forecast
Note:
xt = p1xt-1 + p2xt t + ut
and for t = T+l
xT+l = p1xT+l-1 + p2xT+l t + uT+l
Taking conditional Expectations

34. The Difference equation form of the forecast
xT+l = f1xT+l-1 + f2xT+l fp+dxT+l-p-d
+ d + uT+l +a1uT+l-1 + a2uT+l aquT+l-q
Taking conditional Expectations

35. Three Important Forms of a Non-Stationary Time Series

36. xt = f1xt-1 + f2xt-2 +... +fp+dxt-p-d
The Difference equation Form:
xt = f1xt-1 + f2xt fp+dxt-p-d
+ d + ut +a1ut-1 + a2ut aqut-q
f (B) xt = b(B)Ddxt = d + a(B)ut
f (B) = b(B)Dd = b(B) (I - B)d

37. xt = m(t) +y(B)ut The Random Shock Form:
xt = m(t) + ut +y1ut-1 + y2ut-2 +y3ut
xt = m(t) +y(B)ut
y(B) = I + y1B + y2B
= [f(B)]-1a(B)
the coeficients) y1, y2 ... are used to calculate the MSE of forecasts

38. The Inverted Form:
xt = p1xt-1 + p2xt-2 +p3xt t + ut
p(B)xt = t + ut
p(B) = I - p1B - b2B
= [a(B)]-1 f(B)

39. Forecasting an ARIMA(p,d,q) Time Series
Let PT denote {…, xT-2, xT-1, xT} = the “past” until time T.
Then the optimal forecast of xT+l given PT is denoted by:
This forecast minimizes the mean square error

40. Three different forms of the forecast
Random Shock Form
Inverted Form
Difference Equation Form
Note:

41. Random Shock Form of the forecast
The Inverted form of the forecast
The Difference equation form of the forecast

42. Computation of white noise values

43. Prediction Limits for forecasts
(1 – a)100% confidence limits for xT+l

44. Example: ARIMA(1,1,2) The Model:
xt - xt-1 = b1(xt-1 - xt-2) + ut + a1ut + a2ut or
xt = (1 + b1)xt-1 - b1 xt-2 + ut + a1ut + a2ut
f(B)xt = b(B)(I-B)xt = a(B)ut
where
f(x) = 1 - (1 + b1)x + b1x2 = (1 - b1x)(1-x) and
a(x) = 1 + a1x + a2x2 .

45. The Random Shock form of the model:
xt =y(B)ut
where
y(B) = [b(B)(I-B)]-1a(B) = [f(B)]-1a(B)
i.e.
y(B) [f(B)] = a(B).
Thus
(I + y1B + y2B2 + y3B3 + y4B )(I - (1 + b1)B + b1B2)
= I + a1B + a2B2
Hence
a1 = y1 - (1 + b1) or y1 = 1 + a1 + b1.
a2 = y2 - y1(1 + b1) + b1 or y2 =y1(1 + b1) - b1 + a2.
0 = yh - yh-1(1 + b1) + yh-2b1
or yh = yh-1(1 + b1) - yh-2b1 for h ≥ 3.

46. The Inverted form of the model:
p(B) xt = ut
where
p(B) = [a(B)]-1b(B)(I-B) = [a(B)]-1f(B)
i.e.
p(B) [a(B)] = f(B).
Thus
(I - p1B - p2B2 - p3B3 - p4B )(I + a1B + a2B2)
= I - (1 + b1)B + b1B2
Hence
-(1 + b1) = a1 - p1 or p1 = 1 + a1 + b1.
b1 = -p2 - p1a1 + a2 or p2 = -p1a1 - b1 + a2.
0 = -ph - ph-1a1 - ph-2a2 or ph = -(ph-1a1 + ph-2a2) for h ≥ 3.

47. Now suppose that b1 = 0. 80, a1 = 0. 60 and a2 = 0
Now suppose that b1 = 0.80, a1 = 0.60 and a2 = 0.40 then the Random Shock Form coefficients and the Inverted Form coefficients can easily be computed and are tabled below:

48. The Forecast Equations

49. The Difference Form of the Forecast Equation

50. Computation of the Random Shock Series, One-step Forecasts
Random Shock Computations

51. Computation of the Mean Square Error of the Forecasts and Prediction Limits

52. Table: MSE of Forecasts to lead time l = 12 (s2 = 2.56)

53. Raw Observations, One-step Ahead Forecasts, Estimated error , Error

54. Forecasts with 95% and 66.7% prediction Limits

55. Graph: Forecasts with 95% and 66.7% Prediction Limits

56. Next: Topic: Model Building