Copyright © 2014, 2011 Pearson Education, Inc. 1 Chapter 24 Building Regression Models

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables What explanatory variables belong in a regression model for stock returns?  Initial model motivated by theory such as CAPM  Seek additional variables that improve fit and produce better predictions  The process is typically complicated by correlated explanatory variables (i.e., collinearity)

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables The Initial Model  Build a model that describes returns on Sony stock  CAPM provides a theoretical starting point: use % change for the whole stock market as an explanatory variable

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables The Initial Model – Scatterplot Association appears linear, two outliers identified.

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables The Initial Model – Timeplot of Residuals Locates outliers in time (Dec and Apr. 2003). No evidence of dependence.

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables The Initial Model – Regression Results

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables The Initial Model – Residual Plot Aside from the two outliers, residuals have similar variances.

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables The Initial Model – Check Normality Aside from the two outliers, residuals are nearly normal.

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables The Initial Model – Proceed to Inference  Estimates are consistent with CAPM.  The estimated intercept is not significantly different from zero with a p-value of  The estimated slope is highly significant with a p- value less than

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables Identifying Other Variables  Research in finance suggests other variables, should be added to the initial model.  Three of these variables are: percentage change in the DJIA (Dow % Change) and differences in performance between small and large companies (Small-Big) and between growth and value stocks (High-Low).

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables Correlation Matrix

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables Scatterplot Matrix

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables Identifying Other Variables  The correlation matrix indicates that percentage changes in the DJIA and in the whole market index are highly correlated.  The scatterplot matrix indicates that the associations between the response and these variables appear linear.

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables Adding Explanatory Variables  The data consist of 204 observations with four candidate explanatory variables.  Begin model building by including all four variables in the multiple regression model.

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables MRM with All Four Explanatory Variables

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables Residual Plot: Residuals vs. Fitted Values Outliers are still present; however, this and other residual plots show the conditions for MRM are satisfied.

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables MRM with All Four Explanatory Variables  The F-statistic is with p-value < ; this multiple regression equation explains statistically significant variation in percentage changes in the value of Sony stock.  Based on the t-statistics, only the variable Small- Big improves a regression that contains all of the other explanatory variables.

Copyright © 2014, 2011 Pearson Education, Inc Identifying Explanatory Variables MRM with All Four Explanatory Variables  Adding other explanatory variables to the initial model alters the slope for Market % Change.  This once important variable barely contributes statistically significant variation to the multiple regression.

Copyright © 2014, 2011 Pearson Education, Inc Collinearity Marginal and Partial Slopes  There is a high correlation between Market % Change and Dow % Change (r = 0.91).  This collinearity produces imprecise estimates of the partial slopes.  It explains the difference between the marginal and partial slopes for Market % Change.

Copyright © 2014, 2011 Pearson Education, Inc Collinearity Variance Inflation Factor (VIF)  Variance inflation factor: quantifies the amount of unique variation in each explanatory variable and measures the effect of collinearity.  The VIF for is

Copyright © 2014, 2011 Pearson Education, Inc Collinearity Results for Sony Stock Value Example

Copyright © 2014, 2011 Pearson Education, Inc Collinearity Results for Sony Stock Value Example  Is High-Low not statistically significant because it is redundant or simply unrelated to the response?  Because it has a VIF near 1, collinearity has little effect on this variable (not redundant).  Generally, VIF > 5 or 10 suggests redundancy.

Copyright © 2014, 2011 Pearson Education, Inc Collinearity Signs of Collinearity  R 2 increases less than we’d expect.  Slopes of correlated explanatory variables in the model change dramatically.  The F-statistic is more impressive than individual t-statistics.

Copyright © 2014, 2011 Pearson Education, Inc Collinearity Signs of Collinearity (Continued)  Standard errors for partial slopes are larger than those for marginal slopes.  Variance inflation factors increase.

Copyright © 2014, 2011 Pearson Education, Inc Collinearity Remedies for Collinearity  Remove redundant explanatory variables.  Re-express explanatory variables (e.g., use the average of Market % Change and Dow % Change as an explanatory variable).  Do nothing if the explanatory variables are significant with sensible estimates.

Copyright © 2014, 2011 Pearson Education, Inc Removing Explanatory Variables Issues  After adding several explanatory variables to a model, some of those added and some of those originally present may not be statistically significant.  Remove those variables for which both statistics and substance indicate removal (e.g., remove Dow % Change rather than Market % Change).

Copyright © 2014, 2011 Pearson Education, Inc. 27 4M Example 24.1: MARKET SEGMENTATION Motivation Within which magazine should a manufacturer of a new mobile phone advertise? One has an older audience. They collect consumer ratings on the new phone design along with consumers’ ages and reported incomes.

Copyright © 2014, 2011 Pearson Education, Inc. 28 4M Example 24.1: MARKET SEGMENTATION Method Use multiple regression with ratings as the response and age and income as the explanatory variables. Examine the correlation matrix and scatterplot matrix.

Copyright © 2014, 2011 Pearson Education, Inc. 29 4M Example 24.1: MARKET SEGMENTATION Method There is a high correlation between age and income that implies collinearity.

Copyright © 2014, 2011 Pearson Education, Inc. 30 4M Example 24.1: MARKET SEGMENTATION Method Association is linear with no outliers.

Copyright © 2014, 2011 Pearson Education, Inc. 31 4M Example 24.1: MARKET SEGMENTATION Mechanics – Estimation Results

Copyright © 2014, 2011 Pearson Education, Inc. 32 4M Example 24.1: MARKET SEGMENTATION Mechanics – Examine Plots MRM conditions are satisfied.

Copyright © 2014, 2011 Pearson Education, Inc. 33 4M Example 24.1: MARKET SEGMENTATION Mechanics The F-statistic has a p-value of < The model explains statistically significant variation in the ratings. Although collinear, both predictors (age and income) are statistically significant.

Copyright © 2014, 2011 Pearson Education, Inc. 34 4M Example 24.1: MARKET SEGMENTATION Message The manufacturer should advertise in the magazine with younger subscribers. Based on the 95% confidence interval for the slope of Age, a younger affluent audience assigns, on average, ratings that are 1 to 2 points higher (out of 10) than the older, affluent audience. Age changes sign when adjusted for differences in income. Substantively, this makes sense because younger customers with money find the new design attractive.

Copyright © 2014, 2011 Pearson Education, Inc. 35 4M Example 24.2: RETAIL PROFITS Motivation A chain of pharmacies is looking to expand into a new community. It has data for 110 cities on the following variables: income, disposable income, birth rate, social security recipients, cardiovascular deaths and percentage of local population aged 65 or more.

Copyright © 2014, 2011 Pearson Education, Inc. 36 4M Example 24.2: RETAIL PROFITS Method Use multiple regression. The response variable is profit. Examine the correlation matrix and the scatterplot matrix.

Copyright © 2014, 2011 Pearson Education, Inc. 37 4M Example 24.2: RETAIL PROFITS Method Several high correlations are present (shaded in table) and indicate the presence of collinearity.

Copyright © 2014, 2011 Pearson Education, Inc. 38 4M Example 24.2: RETAIL PROFITS Method This partial scatterplot matrix identifies communities that are distinct from others. Linearity and no lurking variables conditions are met.

Copyright © 2014, 2011 Pearson Education, Inc. 39 4M Example 24.2: RETAIL PROFITS Mechanics – Estimation Results

Copyright © 2014, 2011 Pearson Education, Inc. 40 4M Example 24.2: RETAIL PROFITS Mechanics – Examine Plots These and other plots (not shown here) indicate that all MRM conditions are satisfied.

Copyright © 2014, 2011 Pearson Education, Inc. 41 4M Example 24.2: RETAIL PROFITS Mechanics The F-statistic indicates that this collection of explanatory variables explains statistically significant variation in profits. The VIF’s indicate some explanatory variables are redundant and should be removed (one at a time) from the model.

Copyright © 2014, 2011 Pearson Education, Inc. 42 4M Example 24.2: RETAIL PROFITS Mechanics – Simplified Model This multiple regression separates the effects of birth rates from age (and income). It reveals that cities with higher birth rates produce higher profits when compared to cities with lower birth rates but comparable income and local population above 65.

Copyright © 2014, 2011 Pearson Education, Inc. 43 4M Example 24.2: RETAIL PROFITS Message Three characteristics of the local community affect estimated profits: disposable income, age and birth rates. Increases in each of these lead to higher profits. The data show that the pharmacy chain will have to trade off these characteristics in selecting a site for expansion.

Copyright © 2014, 2011 Pearson Education, Inc. 44 Best Practices  Become familiar with the substantive problem that needs to be solved.  Begin a regression analysis by looking at plots.  Use the F-statistic for the overall model and a t-statistic for each explanatory variable.  Learn to recognize the presence of collinearity.  Don’t fear collinearity – understand it.

Copyright © 2014, 2011 Pearson Education, Inc. 45 Pitfalls  Do not remove explanatory variables at the first sign of collinearity.  Don’t remove several explanatory variables from your model at once.