Chapter 8: Regression Models for Quantitative and Qualitative Predictors Ayona Chatterjee Spring 2008 Math 4813/5813

Polynomial Regression Models When the true curvilinear response function is indeed a polynomial. When the true curvilinear response function is unknown but the polynomial is a good approximation to the true function.

One Predictor Variable – Second Order Let us consider a polynomial model with one variable raised to the first and second order. This polynomial is called a second-order model with one predictor.

One Predictor Variable – Second Order Note the second order regression equation in one variable represents a parabola. Here β 1 is called the linear effect coefficient and β 11 is called the quadratic effect coefficient.

One Predictor Variable – Third Order The third-order model with one predictor variable is given as

Two Predictor Variables – Second Order The regression model This is a second-order model with two predictor variables. The equation represents a conic section.

Example of a Quadratic Response Surface

Hierarchical Approach to Fitting The norm is to fit a second-order or a third-order polynomial and explore if a lower order model is adequate. For example if we have a third-order model in one variable, we may want to test of β 111 =0, or whether or not both β 11 and β 111 equal zero. We use extra sums of squares to do the test.

Extra Sums of Squares To test if β 111 =0 we would use SSR(x 3 |x, x 2 ). If we want to test if both β 11 and β 111 equal zero then we would use SSR(x 2, x 3 |x). Note SSR(x 2, x 3 | x) = SSR(x 2 |x) + SSR(x 3 |x 2, x). If a polynomial of a given order is retained, then all related terms of lower-order are also retained.

Regression Function in Terms of X To revert back to the original scale, and un do the centering of the predictor variables, we use the following transformations.

Example A researcher studied the effects of the charge rate and temperature on the life of a new type of power cell in a small-scale experiment. The charge rate (X 1 ) was controlled at 3 level, and so was the ambient temperature (X 2 ). The life of the power cell was the response (Y). The researcher decided to fit a second- order polynomial regression model.

Data Set - Power Cells Example Power of cells Y Charge Rate X_1 Temperature X_ Scale the units and fit the second order polynomial regression model. Obtain the correlation between the new variables x and original X. Has the transformation reduced collinearity?

Test of Fit An F test to test the goodness of fit of the model to the data. Define If F* is greater than F table value, the model is not a good fit.

Partial F Test Suppose for the given data you want to test if a first-order model is sufficient. Here H 0 :  11 =  22 =  12 =0 The F statistics

Interaction Regression Models A regression model with p-1 variables contains additive effects if the response function can be written as –E{Y} = f 1 (X 1 )+f 2 (X 2 )+……… + f p-1 (X p-1 ) Note all functions need not be simple. If a response function cannot be written as above, then the model is not additive and interaction terms are present.

Interpretation of Interaction Regression Models In presence of interaction term, the regression coefficients cannot be interpreted as before. For a first-order model with interaction term, the change in the mean response with a unit increase in X 1 when X 2 is held constant is  1 +  3 X 2 and not just  1.

Reinforcement Effects When the regression coefficients are positive, we say the interaction effect between the two quantitative variables is of a reinforcement or synergistic when the slope of the response function against one of the predictor variables increases for higher levels of the predictor variables. That is when  3 is positive.

Interference Effects When the regression coefficients are positive, we say the interaction effect between the two quantitative variables is of an interference or antagonistic type when the slope of the response function against one of the predictor variables decreases for higher levels of the predictor variables. That is when  3 is negative.

Implementing an Interaction Model There are two points to keep in mind: –High multicollinearity may exist between some predictors and hence centering the variables may help in reducing this problem. –If there are larger number of predictors, then we have a large choice for possible interaction terms. Choose only the terms that you think will influence the response.

Qualitative Predictors Example: Y is speed at which an insurance innovation is adopted, X 1 is the size of the firm, and another predictor variable to identify type of firm. Here let the firm types be stock or mutual company. Thus we can define

Principle A qualitative variable with c classes will be represented by c-1 indicator variable, each taking on the values 0 and 1. We modify the previous example as

Qualitative Predictor with More than Two Classes Suppose the regression on tool wear (Y) on tool speed (X 1 ) and tool model. Tool model is a qualitative variables with M1, M2, M3 and M4 possible models.

Indicator Variables versus Allocated Codes An alternative to using indicator variables is to use allocated codes. Consider, for instance the predictor variable “frequency of product use”, which has three classes. –Frequent user – 3 –Occasional user – 2 –Nonuser - 1 Here we have Y i =  0 +  1 X i1 +error. This coding implies that the mean response changes by the same amount when going from a nonuser to an occasional user as when going from occasional user to frequent user.

Why indicator variables? Indicator variables make no assumptions about the pacing of the classes. They reply on data to show the differential effects. Alternative model Y i =  0 +  1 X i1 +  2 X i2 +error –Here X1 = 1 for frequent user –X2 =1 for occasional user –All other cases we have zero.

Quantitative to Qualitative Sometimes we may convert quantitative data to qualitative data, for example ages can be grouped and we can use indicator variables to denote the age groups. An alternative coding is to use 1 and –1 for the two levels of a qualitative factor.

Comparison of Two or More Regression Functions-Example We can compare regression functions using hypothesis testing and see if two functions represent the same response function or now. Examples.

Comparison of Two or More Regression Functions-Example A company operates two production lines for making soap bars. For each line, the relation between the speed of the line and the amount of the scrap for the day was studied. A scatter plot of the data for the two production lines suggest that the regression relation between production line speed and amount of scarp is linear but not the same for the two production lines. The slopes appear same but the heights of the regression lines differ. A formal test is desired to determine if the two regression lines are identical.

Soap Production line - Example First fit separate regression models for both production lines. Next combine all the data and using an indicator variable fit a first-order regression model with interaction. Identity of the regression functions for the two production lines is tested by considering the alternatives –H 0 :  2 =  3 =0 and H 0 :  3 =0