1 1 Slide © 2016 Cengage Learning. All Rights Reserved. The equation that describes how the dependent variable y is related to the independent variables x 1, x 2,... x p and an error term is: y =  0 +  1 x 1 +  2 x  p x p +  where:  0,  1,  2,...,  p are the parameters, and  is a random variable called the error term n Multiple Regression Model Chapter 13(a) - Multiple Regression

2 2 Slide © 2016 Cengage Learning. All Rights Reserved. The equation that describes how the mean value of y is related to x 1, x 2,... x p is: The equation that describes how the mean value of y is related to x 1, x 2,... x p is: Multiple Regression Equation and Estimated MRE E ( y ) =  0 +  1 x 1 +  2 x  p x p n Multiple Regression Equation Estimated Multiple Regression Equation Estimated Multiple Regression Equation y = b 0 + b 1 x 1 + b 2 x b p x p A simple random sample is used to compute sample statistics b 0, b 1, b 2,..., b p that are used as the point estimators of the parameters  0,  1,  2,...,  p.

3 3 Slide © 2016 Cengage Learning. All Rights Reserved. Estimation Process Multiple Regression Model E ( y ) =  0 +  1 x 1 +  2 x  p x p +  Multiple Regression Equation E ( y ) =  0 +  1 x 1 +  2 x  p x p Unknown parameters are  0,  1,  2,...,  p Sample Data: x 1 x 2... x p y.... Estimated Multiple Regression Equation Sample statistics are b 0, b 1, b 2,..., b p b 0, b 1, b 2,..., b p b 0, b 1, b 2,..., b p provide estimates of  0,  1,  2,...,  p

4 4 Slide © 2016 Cengage Learning. All Rights Reserved. Two variable model Y X1X1 X2X2 Slope for variable X 1 Slope for variable X 2 Multiple Regression Equation

5 5 Slide © 2016 Cengage Learning. All Rights Reserved. Least Squares Method n Least Squares Criterion n Computation of Coefficient Values The formulas for the regression coefficients The formulas for the regression coefficients b 0, b 1, b 2,... b p involve the use of matrix algebra. We will rely on computer software packages to perform the calculations.

6 6 Slide © 2016 Cengage Learning. All Rights Reserved. The years of experience, score on the aptitude test test, and corresponding annual salary ($1000s) for a sample of 20 programmers is shown on the next slide. n Example: Programmer Salary Survey Multiple Regression Model A software firm collected data for a sample of 20 A software firm collected data for a sample of 20 computer programmers. A suggestion was made that regression analysis could be used to determine if salary was related to the years of experience and the score on the firm’s programmer aptitude test.

7 7 Slide © 2016 Cengage Learning. All Rights Reserved Exper.(Yrs.) TestScore TestScore Exper.(Yrs.) Salary($000s) Salary($000s) Multiple Regression Model

8 8 Slide © 2016 Cengage Learning. All Rights Reserved. Suppose we believe that salary ( y ) is related to the years of experience ( x 1 ) and the score on the programmer aptitude test ( x 2 ) by the following regression model: Suppose we believe that salary ( y ) is related to the years of experience ( x 1 ) and the score on the programmer aptitude test ( x 2 ) by the following regression model: Multiple Regression Model where y = annual salary ($000) y = annual salary ($000) x 1 = years of experience x 1 = years of experience x 2 = score on programmer aptitude test x 2 = score on programmer aptitude test y =  0 +  1 x 1 +  2 x 2 + 

9 9 Slide © 2016 Cengage Learning. All Rights Reserved. Solving for the Estimates of  0,  1,  2 Input Data Least Squares Output x 1 x 2 y C omputer Package for Solving MultipleRegressionProblems b 0 = b 0 = b 1 = b 2 = R 2 = etc.

10 Slide © 2016 Cengage Learning. All Rights Reserved. Excel’s Regression Equation Output Note: Columns F-I are not shown. Solving for the Estimates of  0,  1,  2 SALARY = (EXPER) (SCORE) Note: Predicted salary will be in thousands of dollars.

11 Slide © 2016 Cengage Learning. All Rights Reserved. Interpreting the Coefficients In multiple regression analysis, we interpret each In multiple regression analysis, we interpret each regression coefficient as follows: regression coefficient as follows: b i represents an estimate of the change in y b i represents an estimate of the change in y corresponding to a 1-unit increase in x i when all corresponding to a 1-unit increase in x i when all other independent variables are held constant. other independent variables are held constant. b 1 = Salary is expected to increase by $1,404 for each additional year of experience (when the variable score on programmer attitude test is held constant). Salary is expected to increase by $1,404 for each additional year of experience (when the variable score on programmer attitude test is held constant). b 2 = Salary is expected to increase by $251 for each additional point scored on the programmer aptitude test (when the variable years of experience is held constant). Salary is expected to increase by $251 for each additional point scored on the programmer aptitude test (when the variable years of experience is held constant).

12 Slide © 2016 Cengage Learning. All Rights Reserved. Multiple Coefficient of Determination n Relationship Among SST, SSR, SSE where: SST = total sum of squares SST = total sum of squares SSR = sum of squares due to regression SSR = sum of squares due to regression SSE = sum of squares due to error SSE = sum of squares due to error SST = SSR + SSE = +

13 Slide © 2016 Cengage Learning. All Rights Reserved. Excel’s ANOVA Output Multiple Coefficient of Determination SSR SST R 2 = SSR/SST R 2 = / =.83418

14 Slide © 2016 Cengage Learning. All Rights Reserved. Adjusted Multiple Coefficient of Determination The coefficient of determination R 2 is the proportion of variability in a data set that is accounted for by a statistical model. In this definition, the term "variability" is defined as the sum of squares. Adjusted R-square is a modification of R-square that adjusts for the number of terms in a model. R-square always increases when a new term is added to a model, but adjusted R-square increases only if the new term improves the model more than would be expected by chance. decomposition.

15 Slide © 2016 Cengage Learning. All Rights Reserved. Testing for Significance: Multicollinearity The term multicollinearity refers to the correlation The term multicollinearity refers to the correlation among the independent variables. among the independent variables. The term multicollinearity refers to the correlation The term multicollinearity refers to the correlation among the independent variables. among the independent variables. When the independent variables are highly correlated When the independent variables are highly correlated (say, | r | >.7), it is not possible to determine the (say, | r | >.7), it is not possible to determine the separate effect of any particular independent variable separate effect of any particular independent variable on the dependent variable. on the dependent variable. When the independent variables are highly correlated When the independent variables are highly correlated (say, | r | >.7), it is not possible to determine the (say, | r | >.7), it is not possible to determine the separate effect of any particular independent variable separate effect of any particular independent variable on the dependent variable. on the dependent variable. If the estimated regression equation is to be used only If the estimated regression equation is to be used only for predictive purposes, multicollinearity is usually for predictive purposes, multicollinearity is usually not a serious problem. not a serious problem. If the estimated regression equation is to be used only If the estimated regression equation is to be used only for predictive purposes, multicollinearity is usually for predictive purposes, multicollinearity is usually not a serious problem. not a serious problem. Every attempt should be made to avoid including Every attempt should be made to avoid including independent variables that are highly correlated. independent variables that are highly correlated. Every attempt should be made to avoid including Every attempt should be made to avoid including independent variables that are highly correlated. independent variables that are highly correlated.

16 Slide © 2016 Cengage Learning. All Rights Reserved. In many situations we must work with categorical In many situations we must work with categorical independent variables such as gender (male, female), independent variables such as gender (male, female), method of payment (cash, check, credit card), etc. method of payment (cash, check, credit card), etc. In many situations we must work with categorical In many situations we must work with categorical independent variables such as gender (male, female), independent variables such as gender (male, female), method of payment (cash, check, credit card), etc. method of payment (cash, check, credit card), etc. For example, x 2 might represent gender where x 2 = 0 For example, x 2 might represent gender where x 2 = 0 indicates male and x 2 = 1 indicates female. indicates male and x 2 = 1 indicates female. For example, x 2 might represent gender where x 2 = 0 For example, x 2 might represent gender where x 2 = 0 indicates male and x 2 = 1 indicates female. indicates male and x 2 = 1 indicates female. Categorical Independent Variables In this case, x 2 is called a dummy or indicator variable. In this case, x 2 is called a dummy or indicator variable.

17 Slide © 2016 Cengage Learning. All Rights Reserved. The years of experience, the score on the programmer aptitude test, whether the individual has a relevant graduate degree, and the annual salary ($000) for each of the sampled 20 programmers are shown on the next slide. Categorical Independent Variables Example: Programmer Salary Survey As an extension of the problem involving the computer programmer salary survey, suppose that management also believes that the annual salary is related to whether the individual has a graduate degree in computer science or information systems.

18 Slide © 2016 Cengage Learning. All Rights Reserved Exper.(Yrs.) TestScoreTestScoreExper.(Yrs.)Salary($000s) Salary($000s) Degr. No NoYes YesYesYes Yes Degr. Yes Yes No NoYes Yes Yes Categorical Independent Variables

19 Slide © 2016 Cengage Learning. All Rights Reserved. Estimated Regression Equation ^where: y = annual salary ($1000) y = annual salary ($1000) x 1 = years of experience x 1 = years of experience x 2 = score on programmer aptitude test x 2 = score on programmer aptitude test x 3 = 0 if individual does not have a graduate degree x 3 = 0 if individual does not have a graduate degree 1 if individual does have a graduate degree 1 if individual does have a graduate degree x 3 is a dummy variable y = b 0 + b 1 x 1 + b 2 x 2 + b 3 x 3 ^

20 Slide © 2016 Cengage Learning. All Rights Reserved. Excel’s Regression Statistics Categorical Independent Variables

21 Slide © 2016 Cengage Learning. All Rights Reserved. Excel’s ANOVA Output Categorical Independent Variables

22 Slide © 2016 Cengage Learning. All Rights Reserved. Excel’s Regression Equation Output Categorical Independent Variables Not significant

23 Slide © 2016 Cengage Learning. All Rights Reserved. More Complex Categorical Variables If a categorical variable has k levels, k - 1 dummy If a categorical variable has k levels, k - 1 dummy variables are required, with each dummy variable variables are required, with each dummy variable being coded as 0 or 1. being coded as 0 or 1. If a categorical variable has k levels, k - 1 dummy If a categorical variable has k levels, k - 1 dummy variables are required, with each dummy variable variables are required, with each dummy variable being coded as 0 or 1. being coded as 0 or 1. For example, a variable with levels A, B, and C could For example, a variable with levels A, B, and C could be represented by x 1 and x 2 values of (0, 0) for A, (1, 0) be represented by x 1 and x 2 values of (0, 0) for A, (1, 0) for B, and (0,1) for C. for B, and (0,1) for C. For example, a variable with levels A, B, and C could For example, a variable with levels A, B, and C could be represented by x 1 and x 2 values of (0, 0) for A, (1, 0) be represented by x 1 and x 2 values of (0, 0) for A, (1, 0) for B, and (0,1) for C. for B, and (0,1) for C. Care must be taken in defining and interpreting the Care must be taken in defining and interpreting the dummy variables. dummy variables. Care must be taken in defining and interpreting the Care must be taken in defining and interpreting the dummy variables. dummy variables.

24 Slide © 2016 Cengage Learning. All Rights Reserved. For example, a variable indicating level of education could be represented by x 1 and x 2 values as follows: For example, a variable indicating level of education could be represented by x 1 and x 2 values as follows: More Complex Categorical Variables Highest Degree x 1 x 2 Bachelor’s00 Master’s10 Ph.D.01