1. MODEL BUILDING IN REGRESSION MODELS

2. Model Building and Multicollinearity Suppose we have five factors that we feel could linearly affect y. If all 5 are included we have: y =  0 +  1 x 1 +  2 x 2 +  3 x 3 +  4 x 4 +  5 x 5 +  But while the p-value for the F-test (Significance F) might be small, one or more (if not all) of the p- values for the individual t-tests may be large. Question: Which factors make up the “best” model? –This is called model building

3. Model Building There many approaches to model building –Elimination of some (all) of the variables with high p-values is one approach Forward stepwise regression “builds” the model by adding one variable at a time. Modified F-tests can be used to test if the a certain subset of the variables should be included in the model.

4. The Stepwise Regression Approach y =  0 +  1 x 1 +  2 x 2 +  3 x 3 +  4 x 4 +  5 x 5 +  Step 1: Run five simple linear regressions: –y =  0 +  1 x 1 –y =  0 +  2 x 2 –y =  0 +  3 x 3 –y =  0 +  4 x 4 –y =  0 +  5 x 5 Check the p-values for each – –Note for simple linear regression Significance F = p-value for the t-test. Suppose this model has lowest p-value (< α)

5. Stepwise Regression Step 2: Run four 2-variable linear regressions: Check Significance F and p-values for: –y =  0 +  4 x 4 +  1 x 1 –y =  0 +  4 x 4 +  2 x 2 –y =  0 +  4 x 4 +  3 x 3 –y =  0 +  4 x 4 +  5 x 5 Suppose lowest p-values (< α) Add X3

6. Stepwise Regression Step 3: Run three 3-variable linear regressions: –y =  0 +  3 x 3 +  4 x 4 +  1 x 1 –y =  0 +  3 x 3 +  4 x 4 +  2 x 2 –y =  0 +  3 x 3 +  4 x 4 +  5 x 5 Suppose none of these models have all p-values < α -- STOP -- best model is the one with x 3 and x 4 only

7. Example

8. Regression on 5 Variables

9. Summary of Results from 1-Variable Tests

10. Performing Tests With More Than One Variable Remember the Range for X must be contiguous CUTINSERT CUT CELLSUse CUT and INSERT CUT CELLS to arrange the X columns so that they are next to each other

11. Summary of Results From 2-Variable Tests

12. Summary of Results from 3-Variable Tests

13. Summary of Results from 4-Variable Tests

14. Best Model The best model is the three-variable model that includes x 1, x 4, and x 5.

15. TESTING PARTS OF THE MODEL Sometimes we wish to see whether to keep a set of variables “as a group” or eliminate them from the model. –Example: Model might include 3 dummy variables to account for how the independent variable is affected by a particular season (or quarter) of the year. Will either keep all seasons or will keep none The general approach is to assess how much “extra value” these additional variables will add to the model. –Approach is a Modified F-test

16. Approach: Compare Two Models – The Full Model and The Reduced Model Suppose a model consists of p variables and we wish to consider whether or not to keep a set of p-q of those p variables in the model. Two models –Full model – p variables –Reduced model – q variables For notational convenience, assume the last p-q of the p variables are the ones that would be eliminated. –Sample of size n is taken

17. The Modified F-Test Modified F-Test: H 0 : β q+1 = β q+2 =..… = β p = 0 H A : At least one of these p-q β’s ≠ 0 This is an F-test of the form: Reject H 0 (Accept H A ) if: F > F α,p-q,n-p-1 # variables considered for elimination Degrees of Freedom for the Error Term of the Full Model

18. The Modified F-Statistic For this model, the F-statistic is defined by:

19. Example A housing price model (Full model) is proposed for homes in Laguna Hills that takes into account p = 5 factors: –House size, Lot Size, Age, Whether or not there is a pool, # Bedrooms A reduced model that takes into account only the first of these (q = 3) was discussed earlier. Based on a sample of n = 38 sales, can we conclude that adding these p-q = 2 additional variables (Pool, # Bedrooms) is significant?

21. The Modified F-Test For This Example Modified F-Test: H 0 : β 4 = β 5 = 0 H A : At least one of β 4 and β 5 ≠ 0 For α =.05, the test is Reject H 0 (Accept H A ) if: F > F.05,2,32 F.05,2,32 can be generated in Excel by FINV(.05,2,32) = 3.29.

22. Full Model SSE Full MSE Full DFE Full

23. Reduced Model SSE Reduced

24. The Partial F-Test =((G3-C13)/2)/D13 =FINV(.05,2,B13) SSE from Output Reduced Worksheet

25. The Modified F-Statistic For this model, the modified F-statistic is: The critical value of F = F.05,2,32 = 3.29453087 21.43522834 > 3.29453087 There is enough evidence to conclude that including Pool and Bedrooms is significant.

26. Review Stepwise regression helps determine a “best model” from a series of possible independent variables (x’s) –Approach – Step 1 – Run one variable regressions –If there is a p-value < , keep the variable with lowest p-value as a variable in the model Step 2 – Run 2-variable regressions –One of the two variables in each model is the one determined in Step 1 –Keep the one with the lowest p-values if both are <  Repeat with 3, 4, 5 variables, etc. until no model as has p-values <  Modified F-test for testing the significance of parts of the model –Compare F to F α,p-q,DFE(Full), where F= ((SSE Reduced – SSE Full )/(#terms removed))/MSE Full