Chapter 6: Multiple Regression I Ayona Chatterjee Spring 2008 Math 4813/5813

Need for Several Predictor Variables Suppose you want to study operating cost of a branch office of a consumer finance chain. Predictor variables can be. –Number of new loan applications –Number of outstanding loans Study peak plasma growth hormone in children. Predictors can be: –Age –Gender –Height –Weight

The Need for Multiple Regression At times single variable regression approach may be imprecise. More variables in the model help in better predictions. Multiple regression is very useful in an experimental set up where the experimenter can control a variety of predictor variables simultaneously.

First-Order Model with Two Predictor Variables When there are two predictor variables X 1 and X 2, the first-order regression model with two variables is given as: As before we have E{  i }=0. E{Y}=  0 +  1 X 1 +  2 X 2

Note For simple linear regression, the regression function is a line. For multiple regression, the regression function is a plane. The mean response E{Y} corresponds to a combination of levels of X 1 and X 2. The regression function is called the regression surface or the response surface.

Meaning of Regression Coefficients The value of  0 represents the the mean response of E{Y} when X 1 =0 and X 2 =0. In some models  0 may not have any inherent meaning. The parameter  1 indicates the change in the mean response E{Y} per unit increase in X 1 when X 2 is held constant. Similarly we can define  2.

Additive Effects When the effect of X 1 on the mean response does not depend on the level of X 2, and correspondingly the effect of X 2 does not depend on the level of X 1, the two predictor variables are said to have additive effects or not to interact.

Example A regression model relation test market sales (Y) to point-of- sale expenditure(X 1 ) and TV expenditures (X 2 ) is E{Y}=10+2X 1 +5X 2 Here  1 =2, if point-of- sale expenditure is increased by 1 unit while TV expenditures are held constant, then expected sales Y is increased by 2 units.

First-Order Model with more than Two Predictor Variables Suppose there are p – 1 predictor variables X 1, ……, X p-1. The first-order regression model can be written as below: –

Note This response function is a hyperplane, a plane in more than two-dimensions. The meaning of the parameters is analogous to the case of two predictor variables. The parameter  k indicates the change in the mean response E{Y} with a unit increase in X k, when all other predictor variables are held constant.

General Linear Regression Model The variables X 1, ……, X p-1 do not have to represent different predictor variables. We define the general linear regression model, with normal error terms, simply in terms of X variables: –Here  0,  1, ….,  p-1 are parameters. –X i1, X i2, …, X ip-1 are known constants. –  i are independent N(0,  2 ) –i = 1, …, n

Qualitative Predictor Variable The general linear model allows for both quantitative and qualitative variables. Consider a regression analysis to predict the length of hospital stay (Y) based in the age (X 1 ) and gender (X 2 ) of the patient. We define X 2 as follows: This is called an indicator function

Polynomial Regression Special cases of the general linear regression model. The following is a polynomial regression model with one predictor variable: Polynomials with higher-degree polynomial response functions and transformed variables are also particular cases of the general linear regression model.

Interaction Effects When the effects of the predictor variables on the response variable are not additive, the non-additive regression model with two predictor variables is: This can also be written as a GLRM.

GLRM in Matrix Terms Lets write it out by hand! Matrix notation hides the enormous computational complexities. We will perform matrix manipulations on computers.