1. (c) Martin L. Puterman1 BABS 502 Regression Based Forecasting February 28, 2011

2. (c) Martin L. Puterman2 Simple and Multiple Regression A widely used set of statistical tools that are useful for: –forecasting –data summary –adjustment for uncontrolled factors Basic idea is to fit an equation of the following form relating a dependent variable to one or more independent variables y =  0 +  1 x 1 +  2 x 2 +  3 x 3 + … It’s power is that by choosing the y and x i ’s in different ways a wide range of different effects can be taken into account. The theoretical model assumes that each observation is subject to an additive error which is normally distributed with mean zero and the same variance for every observation so that one observes the signal and noise components in aggregate. In forecasting the signal part provides the point forecast and the random part provides an accuracy measure.

3. (c) Martin L. Puterman3 Regression in forecasting - trend extrapolation Fit a trend to historical data –linear Y t = a + bt –quadratic Y t = a + bt + ct 2 –exponential Y t = ae bt or Log (Y t ) = a + bt Assumption is that the same trend occurred throughout the past and that it will persist into future Fit using multiple regression module in NCSS or spreadsheet regression function Extensive regression theory available to guide use

4. (c) Martin L. Puterman4 Trend Regression

5. (c) Martin L. Puterman5 Trend Regression Linear TrendQuadratic Trend

6. (c) Martin L. Puterman6 Dummy Variables Dummy Variables are independent variables in regression that assume the values of either 0 or 1. –A value 1 means a condition is present; a value 0 means it is not. –When an observation in regression corresponds to a condition being present, then the value of that observation is decreased or increased by a constant amount equal to the value of the coefficient of the dummy variable in the regression. If a condition has three possible values; say “high”, “medium” or “low”. We encode its value with two dummy variables. The first variable, High, equals 1 if the condition is “high” and zero otherwise and the second variable, Medium, equals 1 if the condition is “medium” and zero otherwise. When the condition is “low” both values are zero. The Baseline condition “low” is reflected in the constant in the regression equation. In time series regression, we use dummy variables for seasons, and use S-1 dummy variables if there are S seasons. We are free to chose the baseline season from which all others are measured.

7. (c) Martin L. Puterman7 Trend Regression with Seasonality My experience suggests that a quadratic trend regression plus (additive) seasonality is useful for forecasting Uses “dummy variables” for seasons Must be fit with regression software Equation with linear trend and additive monthly seasonality Y t = a + bt + dt 2 + c 2 Feb t + c 3 Mar t + … + c 12 Dec t Also enables multiple levels of seasonality such as weekly and monthly.

8. (c) Martin L. Puterman8 Trend Regression with Seasonality In previous Feb t, Mar t, … are dummy variables –they equal 1 if observation Y t is from the indicated month and 0 otherwise –Note that there is no dummy variable for January January is the baseline for comparison Examples: Y t = a + btObservation t in January Y t = a + bt + c 2 Observation t in February Y t = a + bt + c 3 Observation t in March In NCSS, declaring a variable as categorical means that NCSS will generate dummy variables automatically –But it will choose the baseline somewhat arbitrarily.

9. (c) Martin L. Puterman9 Trend Regression With Seasonality - Example Some forecasts: Jan: F 156 (1) = 189.88 + 0.36*157 = 246.40 Feb: F 156 (2) = 189.88 + 0.36*158 - 23.85 = 222.91 Mar: F 156 (3) = 189.88 + 0.36*159 - 19.96 = 227.16

10. (c) Martin L. Puterman10 Regression Example: Forecast Updating During Season Goal: Improve total sales forecasts using interim sales data Data; early forecast, interim sales and total sales data for a wide range of products. Fitted Model: Total Sales = 120 +.6 Interim Sales +.3 Early Forecast Example: Early Forecast of Total Sales = 3000; Interim Sales =1400 Revised Total Sales Forecast Total Sales = 120 +.6*1400 +.3*3000 = 1860 Forecast Standard Deviation is Regression RMSE

11. (c) Martin L. Puterman11 Regression Example: Impact of Advertising Goal: Take into account effect of advertising expenditures on sales Data; Sales and advertising expenditures in previous quarter Fitted Model: Sales t = 15 + 10 Quarter t +.8 Sales t-1 +.4 (Advertising t-1 ) 1/2 Example: Sales in last quarter = 2000 and Advertising in previous quarter = 10,000 Total Sales Forecast Sales = 15 +.8*2000 +.4*100 =1655 Forecast Standard Deviation is Regression RMSE

12. (c) Martin L. Puterman12 Some special concerns when using regression with time series data Often the usual regression assumption of uncorrelated errors is violated –This means that the residuals contain information. Case A: This is usually due to model mis-specification; i.e. omission of important variables Case B: But sometimes we have what we think is a good model and there is nothing obvious to add. Difficulty – Standard errors are underestimated so model seems better than it really is. –Concept: Since observations are not independent, there is less information in the data than you would think –Reject H o : β j = 0 when we shouldn’t. Detection –Some systematic pattern in residual plot vs. time –Durbin-Watson Test (see next slide). –(Best approach) ACF of residuals

13. (c) Martin L. Puterman13 Durbin-Watson test; comments The Durbin-Watson test is a not so good alternative to using the ACF of the residuals but it is widely used probably because of historical reasons. It is based on the Durbin-Watson test statistic D. It tests only for first order autocorrelation in the errors. –Formally it tests H 0 :  =0 vs. H a :   0 –The test is reject H 0 and conclude that there is autocorrelation in the residuals if D is well below 2 or well above 2; I suggest being imprecise here. I would worry about values less than 1.4 or greater than 2.6. In economic data, when  is not zero, it is usually positive.

14. (c) Martin L. Puterman14 Regression in the face of correlated residuals Approaches for obtaining more reliable estimates; –Add variables, such as trend squared, or use the lagged dependent variable as an explanatory variable. (See sales and advertising example on previous slide; Sales t-1 is a lagged variable.) –Use time series regression models – which except for a special case (AR1 errors) requires advanced software such as SAS or R. –Use “Multiple regression with serial correlation” under “other regression routines” in NCSS. –Difference data if lag one autocorrelation is large and software such as that above is not availabl e.

15. (c) Martin L. Puterman15 Regression with auto-correlated errors Model y t = β 0 + β 1 x 1t + + β m x mt + ε t where ε t = ρ ε t-1 + ν t and ν t ~ N(0, σ 2 ) and independent The quantity ρ is called the first order auto- correlation or serial correlation parameter and is between -1 and +1. The Corchrane-Orcutt procedure, which is coded in NCSS, estimates the regression coefficients and ρ for this model. Note that usually the regression coefficients will not change much from ordinary regression but their standard errors will be larger.

16. (c) Martin L. Puterman16 What if seasonality is multiplicative and we want to use regression? Problem; Model on nominal scale assumes additive effect of seasonal dummy variables. Solution: Do regression on the logarithmic scale. This means that we transform the dependent variable by taking logarithms (base 10 or base e) and then do regression. Why does this work? Multiplicative seasonality is additive on the log scale! Thus we can do forecasts using the model on the log scale and then transform back to the original scale by exponentiating the forecast on the log scale. –Example: If forecast on Log 10 scale is 3.4, then forecast on the nominal (original) scale is 10 3.4 = 2511.9 units. Also trends and dummy’s on the log-scale have nice interpretations. Consider the model for 4 seasons log 10 (y t ) = 2 +.014t +.19 Season2 t -.13 Season3 t +.04 Season4 t Then the value of the series is increasing 1.4% per period. The value in Season2 is about 19% above the what the trend alone predicts for that season. But predictions based on these transformations are often biased. Alternative ad hoc approach: Deseasonalize data; fit model to deseasonalized data and then multiply back by seasonal factors to get forecasts. This is how time series decomposition works.

17. (c) Martin L. Puterman17 Example – BC Incorporations Trend Regression Regression Equation Section RegressionStandardT-ValueRejectPower IndependentCoefficientErrorto test ProbH0 atof Test Variableb(i)Sb(i)H0:B(i)=0Level5%?at 5% Intercept18672.31621865.454910.0100.0000Yes1.0000 trend584.1152182.04983.2090.0059Yes0.8503 Serial Correlation of Residuals Section SerialSerialSerial LagCorrelationLagCorrelationLagCorrelation 10.74739-0.2205170.0000 20.363210-0.0183180.0000 3-0.0190110.1487190.0000 4-0.2897120.2349200.0000 5-0.4198130.0000210.0000 6-0.4632140.0000220.0000 7-0.4478150.0000230.0000 8-0.3732160.0000240.0000 Above serial correlations significant if their absolute values are greater than 0.485071 Durbin-Watson Test For Serial Correlation Did the Test Reject ParameterValueH0: Rho(1) = 0? Durbin-Watson Value0.3573 Prob. Level: Positive Serial Correlation0.0000Yes Prob. Level: Negative Serial Correlation1.0000No

18. (c) Martin L. Puterman18 Same data using serial correlation routine Run Summary Section ParameterValueParameterValue Dependent VariableBCRows Processed17 Number Ind. Variables1Rows Filtered Out0 Weight VariableNoneRows with X's Missing0 R20.1506Rows with Weight Missing0 Adj R20.0899Rows with Y Missing0 Coefficient of Variation0.4879Rows Used in Estimation17 Mean Square Error4628200Sum of Weights16.000 Square Root of MSE2151.325Completion StatusNormal Completion Ave Abs Pct Error17.034Autocorrelation (Rho)0.8523 Regression Equation Section RegressionStandardT-ValueReject IndependentCoefficientErrorto test ProbH0 at Variableb(i)Sb(i)H0:B(i)=0Level5%? Intercept10850.323712603.32580.8610.4038No trend1244.8410790.06571.5760.1374No

19. (c) Martin L. Puterman19 Same data but adding extra variables In above – dummy = 1 if year for year > 2001 and dummmyXyear allows for a shift in trends. Note there is still some autocorrelation present, lag 1 serial autocorrelation equals.32 (which is insignificant) and the Durbin-Watson Test is significant but much less so than without extra variables. The purpose of this example was to show that autocorrelation can result from the omission of independent variables. Regression Equation Section RegressionStandardT-ValueRejectPower IndependentCoefficientErrorto test ProbH0 atof Test Variableb(i)Sb(i)H0:B(i)=0Level5%?at 5% Intercept22290.50911303.638217.0990.0000Yes1.0000 dummy-37509.50917155.5803-5.2420.0002Yes0.9979 dummyXyear3036.6909518.81705.8530.0001Yes0.9997 year-73.6909192.2110-0.3830.7076No0.0646