1. 14 - 1 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved.

2. 14 - 2 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. When you have completed this chapter, you will be able to: Understand the importance of an appropriate model specification and multiple regression analysis Comprehend the nature and technique of multiple regression models and the concept of partial regression coefficients. Use the estimation techniques for multiple regression models. 1. 2. 3. 4. Conduct an analysis of variance of an estimated model

3. 14 - 3 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. Identify the problems raised, and the remedies thereof, by the presence of multicollinearity in the data sets 6. 7. 8. Draw inferences about the assumed (true) model though a joint test of hypothesis (F test) on the coefficients of all variables Draw inferences about the importance of the independent variables through tests of hypothesis (t-tests) 5. Explain the goodness of fit of an estimated model. Identify the problems raised, and the remedies thereof, by the presence of outliers/influential observations in the data sets 9.

4. 14 - 4 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. Comprehend the concept of partial correlations and its importance in multiple regression analysis 11. 12. Write a research report on an investigation using multiple regression analysis Use some simple remedial measures in the presence of violations of the model assumptions. 13.10. Identify the violation of model assumptions, including linearity, homoscedasticity, autocorrelation, and normality through simple diagnosic procedures.

5. 14 - 5 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. 15. 16. Apply some advanced diagnostic checks and remedies in multiple regression analysis Use qualitative variables, as well as their interactions with other independent variables through a joint test of hypothesis 14. Draw inferences about the importance of a subset of the importance in multiple regression analysis

6. 14 - 6 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. For two independent variables, the general form of the multiple regression equation is: x 1 and x 2 are the independent variables. a is the y-intercept. Multiple Regression Analysis b1 is the net change in y for each unit change in x1 holding x2 constant. It is called …a partial regression coefficient, …a net regression coefficient, or …just a regression coefficient. yabxbx  112 2

7. 14 - 7 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. The general multiple regression with k independent variables is given by: The least squares criterion is used to develop this equation. Because determining b 1, b 2, etc. is very tedious, a software package such as Excel or MINITAB is recommended. Multiple Regression Analysis yabxbx  112 2 bx  k k ...

8. 14 - 8 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. … is measured in the same units as the dependent variable Multiple Standard Error of Estimate … is a measure of the effectiveness of the regression equation … i t is difficult to determine what is a large value and what is a small value of the standard error!

9. 14 - 9 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. Multiple Regression and Correlation Assumptions Multiple Regression and Correlation Assumptions … the independent variables and the dependent variables have a linear relationship … the dependent variable must be continuous and at least interval-scale … the variation in (y - y) or residual must be the same for all values of y. When this is the case, we say the difference exhibits homoscedasticity … the residuals should follow the normal distribution with mean of 0 … successive values of the dependent variable must be uncorrelated

10. 14 - 10 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. … reports the variation in the dependent variable … the variation is divided into two components: a. … the Explained Variation is that accounted for by the set of independent variable b. … the Unexplained or Random Variation is not accounted for by the independent variables The AVOVA Table

11. 14 - 11 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. A correlation matrix is used to show all possible simple correlation coefficients among the variables … i t shows how strongly each independent variable is correlated with the dependent variable. …t he matrix is useful for locating correlated independent variables. Correlation Matrix

12. 14 - 12 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. Global Test The global test is used to investigate whether any of the independent variables have significant coefficients. 0 equal s allNot : 1  H0...: 210  H k  The hypotheses are: … continued

13. 14 - 13 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. The test statistic is the … F distribution with k (number of independent variables) and n-(k+1) degrees of freedom, where n is the sample size Global Test … continued

14. 14 - 14 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. Test for Individual Variables This test is… used to determine which independent variables have nonzero regression coefficients … the variables that have zero regression coefficients are usually dropped from the analysis … the test statistic is the t distribution with n-(k+1) degrees of freedom.

15. 14 - 15 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. A market researcher for Super Dollar Super Markets is studying the yearly amount families of four or more spend on food. Three independent variables are thought to be related to yearly food expenditures (Food). Those variables are: … total family income (Income) in $00, … size of family (Size), and … whether the family has children in college (College)

16. 14 - 16 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. … gender … the part is acceptable or unacceptable … the voter will or will not vote for the incumbent … continued Note: … the following regarding the regression equation … the variable college is called a dummy or indicator variable. (It can take only one of two possible outcomes, i.e. a child is a college student or not) Other examples of dummy variables include… We usually code one value of the dummy variable as “1” and the other “0”

17. 14 - 17 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. … continued FamilyFoodIncomeSizeStudent 1390037640 2530051551 3430051640 4490046850 5640053861 6730062671 7490054350 8530043740 9610060851 10640051361 11740049361 12580056350

18. 14 - 18 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. Use a computer software package, such as MINITAB or Excel, to develop a correlation matrix. From the analysis provided by MINITAB, write out the regression equation: … continued What food expenditure would you estimate for a family of 4, with no college students, and an income of $50,000 (which is input as 500)?

19. 14 - 19 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. The regression equation is Food = 954 + 1.09 Income + 748 Size + 565 Student Predictor Coef SE Coef T P Constant 954 1581 0.60 0.563 Income 1.092 3.153 0.35 0.738 Size 748.4 303.0 2.47 0.039 Student 564.5 495.1 1.14 0.287 S = 572.7 R-Sq = 80.4% R-Sq(adj) = 73.1% Analysis of Variance Source DF SS MS F P Regression 3 10762903 3587634 10.94 0.003 Residual Error 8 2623764 327970 Total 11 13386667 … continued y = 954 +1.09x 1 + 748x 2 + 565x 3

20. 14 - 20 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. The regression equation is Food = 954 + 1.09 Income + 748 Size + 565 Student Predictor Coef SE Coef T P Constant 954 1581 0.60 0.563 Income 1.092 3.153 0.35 0.738 Size 748.4 303.0 2.47 0.039 Student 564.5 495.1 1.14 0.287 S=572.7 R-Sq = 80.4% R-Sq(adj) = 73.1% Analysis of Variance Source DF SS MS F P Regression 3 10762903 3587634 10.94 0.003 Residual Error 8 2623764 327970 Total 11 13386667 From the regression output we note: The coefficient of determination is 80.4 percent. From the regression output we note: The coefficient of determination is 80.4 percent. … continued This means that more than 80 percent of the variation in the amount spent on food is accounted for by the variables income, family size, and student

21. 14 - 21 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. The regression equation is Food = 954 + 1.09 Income + 748 Size + 565 Student Predictor Coef SE Coef T P Constant 954 1581 0.60 0.563 Income 1.092 3.153 0.35 0.738 Size 748.4 303.0 2.47 0.039 Student 564.5 495.1 1.14 0.287 S=572.7 R-Sq = 80.4% R-Sq(adj) = 73.1% Analysis of Variance Source DF SS MS F P Regression 3 10762903 3587634 10.94 0.003 Residual Error 8 2623764 327970 Total 11 13386667 An additional family member will increase the amount spent per year on food by $748 … continued A family with a college student will spend $565 more per year on food than those without a college student

22. 14 - 22 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. … continued The correlation matrix is as follows: Food Income Size Income 0.587 Size 0.876 0.609 Student 0.773 0.491 0.743 The strongest correlation between the dependent variable (Food) and an independent variable is between family size and amount spent on food. None of the correlations among the independent variables should cause problems. All are between –.70 and.70

23. 14 - 23 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. Find the estimated food expenditure for a family of 4 with a $500 (that is $50,000) income and no college student. … continued The regression equation is… Food = 954 + 1.09 Income + 748 Size + 565 Student y = 954 + 1.09(500) + 748(4) + 565(0) = $4,491

24. 14 - 24 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. Decision: H 0 is rejected. Not all the regression coefficients are zero The regression equation is Food = 954 + 1.09 Income + 748 Size + 565 Student Predictor Coef SE Coef T P Constant 954 1581 0.60 0.563 Income 1.092 3.153 0.35 0.738 Size 748.4 303.0 2.47 0.039 Student 564.5 495.1 1.14 0.287 S=572.7 R-Sq = 80.4% R-Sq(adj) = 73.1% Analysis of Variance Source DF SS MS F P Regression 3 10762903 3587634 10.94 0.003 Residual Error 8 2623764 327970 Total 11 13386667 Conduct a global test of hypothesis to determine if any of the regression coefficients are not zero H 0 is rejected if F>4.07 …from the MINITAB output, the computed value of F is 10.94 … continued   oneleastat: 1 H  0: 3210 H

25. 14 - 25 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. … Using the 5% level of significance, reject H 0 if the P-value<.05 … continued From the MINITAB output, the only significant variable is FAMILY (family size) using the P-values (The other variables can be omitted from the model) H 12 0:  H 02 0:  The regression equation is Food = 954 + 1.09 Income + 748 Size + 565 Student Predictor Coef SECoef T P Constant 954 1581 0.60 0.563 Income 1.092 3.153 0.35 0.738 Size 748.4 303.0 2.47 0.039 Student 564.5 495.1 1.14 0.287 Conduct an individual test to determine which coefficients are not zero (This is the hypothesis for the independent variable family size)

26. 14 - 26 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. The regression equation is Food = 954 + 1.09 Income + 748 Size + 565 Student Predictor Coef SECoef 954 1581 0.60 Income 1.092 3.153 Size 748.4 303.0 Student 564.5 495.1 S=572.7 R-Sq = 80.4% R-Sq(adj) = 73.1% Regression Analysis: Food versus Size The regression equation is Food = 340 + 1031 Size Predictor Coef SECoef Constant 339.7 940.7 Size 1031.0 179.4 S = 557.7 R-Sq = 76.8% R-Sq(adj) = 74.4% … continued Rerun the analysis using only the significant independent family size …and the R-square term was reduced by only 3.6 percent...the coefficient of determination is 76.8 percent (the two independent variables are dropped) …the new regression equation is: y = 340 + 1031 X 2

27. 14 - 27 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. Residuals should be approximately normally distributed Analysis Residuals … histograms and stem-and-leaf charts are useful in checking this requirement A residual is the difference between the actual value of y and the predicted value y … a plot of the residuals and their corresponding y values is used for showing that there are no trends or patterns in the residuals

28. 14 - 28 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. Analysis Residuals 450075006000 0 -500 500 1000 y Residuals Plot

29. 14 - 29 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. Analysis Residuals -600 -200 200 600 1000 876543210876543210 Frequency Residuals Histograms

30. 14 - 30 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. Test your learning … www.mcgrawhill.ca/college/lind Click on… Online Learning Centre for quizzes extra content data sets searchable glossary access to Statistics Canada’s E-Stat data …and much more!

31. 14 - 31 Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. This completes Chapter 14