1. Chapter 6: Forecasting/Prediction
First Note That:
Forecasting is Extrapolating
NOTE: Some slides have blank sections. They are based on a teaching style in which the corresponding blank (derivations, theorem proofs, examples, …) are worked out in class on the board or overhead projector.

2. Predict the price of a car that weighs 3500 lbs.
STAT 1301 Example
Predict the price of a car that weighs 3500 lbs.
- extrapolation would say it’s about $16,000

3. oops!!! Predict the price of a car that weighs 3500 lbs.
STAT 1301 Example
Predict the price of a car that weighs 3500 lbs.
oops!!!
- extrapolation would say it’s about $16,000

4. Note: Forecasting is Extrapolating
Lesson from regression analysis and 1301 example?
- Don’t do it
However, it’s not very important to
- “predict” the sunspot number for 2012
- “predict” sales for the previous quarter

5. Deterministic Signal-plus-Noise Models
Example Signals:
, C constant

6. Recall -- sometimes it’s not easy to tell whether a deterministic signal is present in the data
Is there a deterministic signal?

7. Realizations
- is there a deterministic signal?
Recall - Sometimes it’s not easy to tell whether a deterministic signal is present in the data.
Global Temperature Data

8. How would you predict/forecast into the future?
- depends on the model
Global Temperature Data

9. Now which forecast do you prefer?
Suppose I told you this is a Realization from the model
Other Realizations
Now which forecast do you prefer?

10. Forecasting Setting in this Chapter
Forecast future behavior of a time series given a finite realization of its past.
The use of ARMA models for this purpose has become popular in recent years.
ARMA Forecasting Vs. Curve Fitting
Curve fitting (regression):
Underlying assumption that future behavior follows some deterministic path with only random fluctuations
ARMA Forecasting:
Underlying assumption is that the future is guided only by its correlation to the past

11. Problem:
Notation:
“Best Predictor”:
Mean Square Sense

12. Prediction Background
Theorem 6.1:
for any random variable h(X) such that E[h(X)2] < 
- i.e. E[Y|X] minimizes mean square error between Y and square integrable random variables that are functions of X
- proved in Probability Theory

13. Notes: We restrict our attention to “linear predictors”
i.e. we are looking for a predictor of the form

14. Theorem 6.2: (Projection Theorem)
If X t is a stationary time series, then is the best linear predictor of (has minimum mean square error among linear predictors) if and only if
Proof:

16. Notes: 3. We let denote the best linear predictor
4. Denote coefficients of by
i.e.

17. Theorem 6.3: (proof – HW)
If Xt is a stationary process with autocovariance function gk , then are the solutions to the system

18. Note 5:
The typical application of these results is the case t0 = n = realization length. This results in a “large” system of equations to solve.

19. Box-Jenkins Approach * Assumes: In this setting:
Xt is stationary/invertible ARMA *
In this setting:
- best predictor, , is easily obtained
- results in “natural approximations” in practice
- for “moderately” large t0 , results are very similar to
- approach we will use in tswge

20. Result : (Section 6.2)

22. - may be difficult to obtain
Review:
- may be difficult to obtain
- Theorem 6.2 (Projection Theorem)

23. Note: Best forecast is LINEAR
(Stationary and invertible ARMA, m = 0)
Best Forecast is
Note: Best forecast is LINEAR
- but it is not in a form for calculation

24. Note: Using
 - step ahead forecast error
Nonzero Mean

25. Properties

27. Forecasting Based on the Difference Equation
Suppose Xt is ARMA(p,q), i.e. j(B)X t = q(B)a t

28. Recall:

29. Non-Zero Mean Form

30. Note:
its known values as of time t0 or
(2) its forecast from t0

31. Forecasts using the difference equation are more easily used for calculating than those from GLP.
However – there are still problems in implementation.
Practical Situation
(a) only have been observed
(b) model has been estimated
(c) m is unknown
(d) ak ‘s are unknown

32. In Practice:
We use difference equation form with the following modifications:
The estimated model is assumed to be the true model
(ii) m is estimated by
(iii) a k’s are estimated from the data

33. Basic Formula for Calculating Forecasts from Difference Equation

34. Example 6.1: AR(1)
(1 -.8B)(X t - 25) = a t
X 1, ... , X 80 observed

35. Example 6.2: (1-1.2B+ .6B2)(Xt – m) = (1-.5B)at
Calculate Forecasts:

37. Example 6. 2: Xt , t = 1,2,…,25 from (1-1. 2B+. 6B2)(Xt – 50) = (1-
Example 6.2: Xt , t = 1,2,…,25 from (1-1.2B+ .6B2)(Xt – 50) = (1-.5B)at t Xt 1
49.49
0.00 14
50.00
-.42 2
51.14 15
49.70
-.61 3
49.97
-1.72 16
51.73
1.77 4
49.74 17
51.88
.49 5
50.35
.42 18
50.58
-.11 6
49.57
-.81 19
48.78
-1.22 7
50.39
.70 20
49.14
.51 8
51.04
.65 21
48.10
-1.36 9
49.93
-.78 22
49.52
.59 10
49.78
.08 23
50.06
-.23 11
48.37
-1.38 24
50.66
.17 12
48.85
-.03 25
52.05
1.36 13
49.85
.22

38. Example 6.2: Calculate Forecasts

39. Eventual Forecast Function
Recall:

40. Eventual Forecast Function

41. Notes:

42. Example 6.1 (cont.) Forecasts using the AR(1) Model
What should forecasts look like?
- like rk for AR(1)
- damped exponential C1(.8) k

43. Example 6.1 (cont.) Forecasts using the AR(1) Model
What should forecasts look like?
- like rk for AR(1)
- damped exponential C1(.8) k

44. Example 6.2 (cont) Forecasts using the ARMA(2,1) Model

45. Basic Formula for Calculating Forecasts from Difference Equation

46. Probability Limits for Forecasts
Recall:

47. Note: If the white noise is normal, then

48. 100(1-a)% Probability Limits for Forecasts
Theorem 2.3
Notes: p

49. Example 6.2: (cont.) psi.weights.wge l Forecast Half-width 1 51.40
1.70 2
50.46
2.07 3
49.73
2.11 4
49.42
2.12

50. Example 6.2
Forecasts with 95% Prediction Limits

51. tswge demo To forecast from a stationary ARMA model use
fore.arma.wge(x,phi,theta,n.ahead,lastn=FALSE,plot=TRUE,limits=TRUE)
Example 6.1 Forecasts using the AR(1) Model
data(fig6.1nf)
fore.arma.wge(fig6.1nf,phi=.8,n.ahead=20, lastn=FALSE,plot=TRUE,limits=FALSE)
fore.arma.wge(fig6.1nf,phi=.8,n.ahead=20, lastn=FALSE,plot=TRUE,limits=TRUE)
fore.arma.wge(fig6.1nf,phi=.8,n.ahead=20, lastn=TRUE,plot=TRUE,limits=FALSE)

52. tswge demo - continued Example 6.2 Forecasts using the ARMA(2,1) Model
data(fig6.2nf)
fore.arma.wge(fig6.2nf,phi=c(1.2,-.6), theta=.5, n.ahead=20,lastn=FALSE,plot=TRUE,limits=FALSE)
fore.arma.wge(fig6.2nf,phi=c(1.2,-.6), theta=.5, n.ahead=20,lastn=FALSE,plot=TRUE,limits=TRUE)

53. Forecasting with ARUMA Models*
Formally, as in the stationary case

54. Notes:
(1) Consider as limit of stationary models *

55. Example 6.3:

56. Example 6.3:

57. Example 6.4:

58. Note: Forecasts follow a line determined by the last 2 data values
Example 6.5:
Note: Forecasts follow a line determined by the last 2 data values
In General:
Forecasts follow a polynomial of degree d - 1 that passes through the last d time points.

59. tswge demo (1-.8B)(1-B)Xt = at (1-B)2 Xt = at
fore.aruma.wge(x,phi,theta,d,s,lambda,n.ahead, lastn=FALSE,plot=TRUE,limits=TRUE)
Example 6.4 Forecasts using the ARUMA(1,1,0) Model
(1-.8B)(1-B)Xt = at
x=gen.aruma.wge(n=200,phi=.8,d=1)
fore.aruma.wge(x,phi=.8,d=1,n.ahead=20,lastn=FALSE,plot=TRUE,limits=TRUE)
Example 6.5 Forecasts using the ARUMA(0,2,0) Model
(1-B)2 Xt = at
x=gen.aruma.wge(n=50,d=2)
fore.aruma.wge(x,d=2,n.ahead=20,lastn=FALSE, plot=TRUE,limits=TRUE)

60. Example:

61. Example 6.6: Seasonal Model
Forecasts:
- i.e. forecasts are an exact replica of the last s values

62. More general seasonal model
forecasts follow general pattern of last s points, but not an exact replica
“finer detail” incorporated through j (B) and q (B)

63. tswge demo
fore.aruma.wge(x,phi,theta,d,s,lambda,n.ahead, lastn=FALSE,plot=TRUE,limits=TRUE)
Forecasts from the pure seasonal model
(1-B 4)Xt = at
x=gen.aruma.wge(n=20,s=4)
fore.aruma.wge(x,s=4,n.ahead=8,lastn=FALSE,plot=TRUE, limits=FALSE) #
fore.aruma.wge(x,s=4,n.ahead=8,lastn=TRUE,plot=TRUE, limits=FALSE)
Example 6.5 Forecasts using seasonal model
(1-.8B)(1-B 4)Xt = at
x=gen.aruma.wge(n=20,phi=.9,s=4)
fore.aruma.wge(x,phi=.9,s=4,n.ahead=8,lastn=FALSE, plot=TRUE,limits=FALSE)
Also lastn=TRUE

64. Example 6.8: Seasonal Model with Trend
airline model
note that full model characteristic equation has 2 roots of +1
See Table 5.5
Log airline data and forecasts from t0 = 108

65. tswge demo – log airline data
1(B)(1-B)(1-B 12)Xt = at
Where 1(B) is the 12th order operator given in (6.53) of Woodward, Gray, and Elliott (2017)
data(airlog)
phi1=c(-.36,-.05,-.14,-.11,.04,.09,-.02, .02,.17,.03,-.10,-.38)
fore.aruma.wge(airlog,phi=phi1,d=1,s=12,n.ahead=36,lastn=TRUE,plot=TRUE,limits=FALSE)

66. Example 6.9: Cyclic Models
forecasts follow a non-damping sinusoid
adaptive

67. Forecasts:
Note: It can be shown that the forecasts satisfy

68. Signal-plus-noise forecasts
Xt = st + Zt
Text considers

69. Forecasting Strategy (linear case)

70. tswge demo
fore.sigplusnoise.wge(x, linear = TRUE, freq = 0, max.p = 5, n.ahead = 10, lastn = FALSE, plot = TRUE, limits = TRUE)
x=gen.sigplusnoise.wge(n=50,b0=10,b1=.2, phi=.8) #
xfore=fore.sigplusnoise.wge(x,linear=TRUE,,n.ahead=10,lastn=FALSE,limits=FALSE)

71. Effect of starting values on ARUMA Model

72. Effect of starting values on ARUMA Model

74. Observation: Forecasts from ARMA/ARUMA model are more adaptive
Application may dictate whether this adaptability is desirable