12b - 1 © 2000 Prentice-Hall, Inc. Statistics Multiple Regression and Model Building Chapter 12 part II
12b - 2 © 2000 Prentice-Hall, Inc. Learning Objectives 1.Describe Curvilinear Regression Models 2.Summarize Interaction Models 3.Explain Models with Qualitative Variables 4.Evaluate Portions of Regression Models 5.Describe Stepwise Regression Analysis
12b - 3 © 2000 Prentice-Hall, Inc. Types of Regression Models
12b - 4 © 2000 Prentice-Hall, Inc. Models With a Single Quantitative Variable
12b - 5 © 2000 Prentice-Hall, Inc. Types of Regression Models
12b - 6 © 2000 Prentice-Hall, Inc. First-Order Model With 1 Independent Variable
12b - 7 © 2000 Prentice-Hall, Inc. First-Order Model With 1 Independent Variable 1.Relationship Between 1 Dependent & 1 Independent Variable Is Linear
12b - 8 © 2000 Prentice-Hall, Inc. First-Order Model With 1 Independent Variable 1.Relationship Between 1 Dependent & 1 Independent Variable Is Linear 2.Used When Expected Rate of Change in Y Per Unit Change in X Is Stable
12b - 9 © 2000 Prentice-Hall, Inc. First-Order Model With 1 Independent Variable 1.Relationship Between 1 Dependent & 1 Independent Variable Is Linear 2.Used When Expected Rate of Change in Y Per Unit Change in X Is Stable 3.Used With Curvilinear Relationships If Relevant Range Is Linear
12b - 10 © 2000 Prentice-Hall, Inc. First-Order Model Relationships 1 < 0 1 > 0 Y X 1 Y X 1
12b - 11 © 2000 Prentice-Hall, Inc. First-Order Model Worksheet Run regression with Y, X 1
12b - 12 © 2000 Prentice-Hall, Inc. Types of Regression Models
12b - 13 © 2000 Prentice-Hall, Inc. Second-Order Model With 1 Independent Variable 1.Relationship Between 1 Dependent & 1 Independent Variables Is a Quadratic Function 2.Useful 1 St Model If Non-Linear Relationship Suspected
12b - 14 © 2000 Prentice-Hall, Inc. Second-Order Model With 1 Independent Variable 1.Relationship Between 1 Dependent & 1 Independent Variables Is a Quadratic Function 2.Useful 1 St Model If Non-Linear Relationship Suspected 3.Model Linear effect Curvilinear effect
12b - 15 © 2000 Prentice-Hall, Inc. Second-Order Model Relationships 2 > 0 2 < 0
12b - 16 © 2000 Prentice-Hall, Inc. Second-Order Model Worksheet Create X 1 2 column. Run regression with Y, X 1, X 1 2.
12b - 17 © 2000 Prentice-Hall, Inc. Types of Regression Models
12b - 18 © 2000 Prentice-Hall, Inc. Third-Order Model With 1 Independent Variable 1.Relationship Between 1 Dependent & 1 Independent Variable Has a ‘Wave’ 2.Used If 1 Reversal in Curvature
12b - 19 © 2000 Prentice-Hall, Inc. Third-Order Model With 1 Independent Variable 1.Relationship Between 1 Dependent & 1 Independent Variable Has a ‘Wave’ 2.Used If 1 Reversal in Curvature 3.Model Linear effect Curvilinear effects
12b - 20 © 2000 Prentice-Hall, Inc. Third-Order Model Relationships 3 < 0 3 > 0
12b - 21 © 2000 Prentice-Hall, Inc. Third-Order Model Worksheet Multiply X 1 by X 1 to get X 1 2. Multiply X 1 by X 1 by X 1 to get X 1 3. Run regression with Y, X 1, X 1 2, X 1 3.
12b - 22 © 2000 Prentice-Hall, Inc. Models With Two or More Quantitative Variables
12b - 23 © 2000 Prentice-Hall, Inc. Types of Regression Models
12b - 24 © 2000 Prentice-Hall, Inc. First-Order Model With 2 Independent Variables 1.Relationship Between 1 Dependent & 2 Independent Variables Is a Linear Function 2.Assumes No Interaction Between X 1 & X 2 Effect of X 1 on E(Y) Is the Same Regardless of X 2 Values Effect of X 1 on E(Y) Is the Same Regardless of X 2 Values
12b - 25 © 2000 Prentice-Hall, Inc. First-Order Model With 2 Independent Variables 1.Relationship Between 1 Dependent & 2 Independent Variables Is a Linear Function 2.Assumes No Interaction Between X 1 & X 2 Effect of X 1 on E(Y) Is the Same Regardless of X 2 Values Effect of X 1 on E(Y) Is the Same Regardless of X 2 Values 3.Model
12b - 26 © 2000 Prentice-Hall, Inc. No Interaction
12b - 27 © 2000 Prentice-Hall, Inc. No Interaction E(Y) X1X1X1X E(Y) = 1 + 2X 1 + 3X 2
12b - 28 © 2000 Prentice-Hall, Inc. No Interaction E(Y) X1X1X1X E(Y) = 1 + 2X 1 + 3X 2 E(Y) = 1 + 2X 1 + 3(0) = 1 + 2X 1
12b - 29 © 2000 Prentice-Hall, Inc. No Interaction E(Y) X1X1X1X E(Y) = 1 + 2X 1 + 3X 2 E(Y) = 1 + 2X 1 + 3(1) = 4 + 2X 1 E(Y) = 1 + 2X 1 + 3(0) = 1 + 2X 1
12b - 30 © 2000 Prentice-Hall, Inc. No Interaction E(Y) X1X1X1X E(Y) = 1 + 2X 1 + 3(2) = 7 + 2X 1 E(Y) = 1 + 2X 1 + 3X 2 E(Y) = 1 + 2X 1 + 3(1) = 4 + 2X 1 E(Y) = 1 + 2X 1 + 3(0) = 1 + 2X 1
12b - 31 © 2000 Prentice-Hall, Inc. No Interaction E(Y) X1X1X1X E(Y) = 1 + 2X 1 + 3(2) = 7 + 2X 1 E(Y) = 1 + 2X 1 + 3X 2 E(Y) = 1 + 2X 1 + 3(1) = 4 + 2X 1 E(Y) = 1 + 2X 1 + 3(0) = 1 + 2X 1 E(Y) = 1 + 2X 1 + 3(3) = X 1
12b - 32 © 2000 Prentice-Hall, Inc. No Interaction Effect (slope) of X 1 on E(Y) does not depend on X 2 value E(Y) X1X1X1X E(Y) = 1 + 2X 1 + 3(2) = 7 + 2X 1 E(Y) = 1 + 2X 1 + 3X 2 E(Y) = 1 + 2X 1 + 3(1) = 4 + 2X 1 E(Y) = 1 + 2X 1 + 3(0) = 1 + 2X 1 E(Y) = 1 + 2X 1 + 3(3) = X 1
12b - 33 © 2000 Prentice-Hall, Inc. First-Order Model Relationships
12b - 34 © 2000 Prentice-Hall, Inc. First-Order Model Worksheet Run regression with Y, X 1, X 2
12b - 35 © 2000 Prentice-Hall, Inc. Types of Regression Models
12b - 36 © 2000 Prentice-Hall, Inc. Interaction Model With 2 Independent Variables 1.Hypothesizes Interaction Between Pairs of X Variables Response to One X Variable Varies at Different Levels of Another X Variable Response to One X Variable Varies at Different Levels of Another X Variable
12b - 37 © 2000 Prentice-Hall, Inc. Interaction Model With 2 Independent Variables 1.Hypothesizes Interaction Between Pairs of X Variables Response to One X Variable Varies at Different Levels of Another X Variable Response to One X Variable Varies at Different Levels of Another X Variable 2.Contains Two-Way Cross Product Terms
12b - 38 © 2000 Prentice-Hall, Inc. 1.Hypothesizes Interaction Between Pairs of X Variables Response to One X Variable Varies at Different Levels of Another X Variable Response to One X Variable Varies at Different Levels of Another X Variable 2.Contains Two-Way Cross Product Terms 3.Can Be Combined With Other Models Example: Dummy-Variable Model Example: Dummy-Variable Model Interaction Model With 2 Independent Variables
12b - 39 © 2000 Prentice-Hall, Inc. Effect of Interaction
12b - 40 © 2000 Prentice-Hall, Inc. Effect of Interaction 1.Given:
12b - 41 © 2000 Prentice-Hall, Inc. Effect of Interaction 1.Given: 2.Without Interaction Term, Effect of X 1 on Y Is Measured by 1
12b - 42 © 2000 Prentice-Hall, Inc. Effect of Interaction 1.Given: 2.Without Interaction Term, Effect of X 1 on Y Is Measured by 1 3.With Interaction Term, Effect of X 1 on Y Is Measured by 1 + 3 X 2 Effect Increases As X 2i Increases Effect Increases As X 2i Increases
12b - 43 © 2000 Prentice-Hall, Inc. Interaction Model Relationships
12b - 44 © 2000 Prentice-Hall, Inc. Interaction Model Relationships E(Y) X1X1X1X E(Y) = 1 + 2X 1 + 3X 2 + 4X 1 X 2
12b - 45 © 2000 Prentice-Hall, Inc. Interaction Model Relationships E(Y) X1X1X1X E(Y) = 1 + 2X 1 + 3X 2 + 4X 1 X 2 E(Y) = 1 + 2X 1 + 3(0) + 4X 1 (0) = 1 + 2X 1
12b - 46 © 2000 Prentice-Hall, Inc. Interaction Model Relationships E(Y) X1X1X1X E(Y) = 1 + 2X 1 + 3X 2 + 4X 1 X 2 E(Y) = 1 + 2X 1 + 3(1) + 4X 1 (1) = 4 + 6X 1 E(Y) = 1 + 2X 1 + 3(0) + 4X 1 (0) = 1 + 2X 1
12b - 47 © 2000 Prentice-Hall, Inc. Interaction Model Relationships Effect (slope) of X 1 on E(Y) does depend on X 2 value E(Y) X1X1X1X E(Y) = 1 + 2X 1 + 3X 2 + 4X 1 X 2 E(Y) = 1 + 2X 1 + 3(1) + 4X 1 (1) = 4 + 6X 1 E(Y) = 1 + 2X 1 + 3(0) + 4X 1 (0) = 1 + 2X 1
12b - 48 © 2000 Prentice-Hall, Inc. Interaction Model Worksheet Multiply X 1 by X 2 to get X 1 X 2. Run regression with Y, X 1, X 2, X 1 X 2
12b - 49 © 2000 Prentice-Hall, Inc. Types of Regression Models
12b - 50 © 2000 Prentice-Hall, Inc. Second-Order Model With 2 Independent Variables 1.Relationship Between 1 Dependent & 2 or More Independent Variables Is a Quadratic Function 2.Useful 1 St Model If Non-Linear Relationship Suspected
12b - 51 © 2000 Prentice-Hall, Inc. Second-Order Model With 2 Independent Variables 1.Relationship Between 1 Dependent & 2 or More Independent Variables Is a Quadratic Function 2.Useful 1 St Model If Non-Linear Relationship Suspected 3.Model
12b - 52 © 2000 Prentice-Hall, Inc. Second-Order Model Relationships 4 + 5 > 0 4 + 5 < 0 3 2 > 4 4 5
12b - 53 © 2000 Prentice-Hall, Inc. Second-Order Model Worksheet Multiply X 1 by X 2 to get X 1 X 2 ; then X 1 2, X 2 2. Run regression with Y, X 1, X 2, X 1 X 2, X 1 2, X 2 2.
12b - 54 © 2000 Prentice-Hall, Inc. Testing Model Portions
12b - 55 © 2000 Prentice-Hall, Inc. 1.Tests the Contribution of a Set of X Variables to the Relationship With Y 2.Null Hypothesis H 0 : g+1 =... = k = 0 Variables in Set Do Not Improve Significantly the Model When All Other Variables Are Included Variables in Set Do Not Improve Significantly the Model When All Other Variables Are Included 3.Used in Selecting X Variables or Models 4.Part of Most Computer Programs Testing Model Portions
12b - 56 © 2000 Prentice-Hall, Inc. Models With One Qualitative Independent Variable
12b - 57 © 2000 Prentice-Hall, Inc. Types of Regression Models
12b - 58 © 2000 Prentice-Hall, Inc. Dummy-Variable Model 1.Involves Categorical X Variable With 2 Levels e.g., Male-Female; College-No College e.g., Male-Female; College-No College 2.Variable Levels Coded 0 & 1 3.Number of Dummy Variables Is 1 Less Than Number of Levels of Variable 4.May Be Combined With Quantitative Variable (1 st Order or 2 nd Order Model)
12b - 59 © 2000 Prentice-Hall, Inc. Dummy-Variable Model Worksheet X 2 levels: 0 = Group 1; 1 = Group 2. Run regression with Y, X 1, X 2
12b - 60 © 2000 Prentice-Hall, Inc. Interpreting Dummy- Variable Model Equation
12b - 61 © 2000 Prentice-Hall, Inc. Interpreting Dummy- Variable Model Equation Given: Starting s alary of c ollege gra d' s s GPA i i if Female f Male Y Y X X X X Y Y X X X X i i i i i i R R S S T T
12b - 62 © 2000 Prentice-Hall, Inc. Interpreting Dummy- Variable Model Equation Given: Starting s alary of c ollege gra d' s s GPA i i if Female Males ( f Male ): Y Y X X X X Y Y X X X X Y Y X X X X i i i i i i i i i i i i X X R R S S T T a a f f
12b - 63 © 2000 Prentice-Hall, Inc. Interpreting Dummy- Variable Model Equation Same slopes
12b - 64 © 2000 Prentice-Hall, Inc. Dummy-Variable Model Relationships Y X1X1X1X1 0 0 Same Slopes 1 0000 0 + 2 ^ ^ ^ ^ Females Males
12b - 65 © 2000 Prentice-Hall, Inc. Dummy-Variable Model Example
12b - 66 © 2000 Prentice-Hall, Inc. Dummy-Variable Model Example Computer O utput: f Male if Female i i Y Y X X X X X X i i i i i i R R S S T T
12b - 67 © 2000 Prentice-Hall, Inc. Dummy-Variable Model Example Computer O utput: f Male if Female Males ( i i ): Y Y X X X X X X Y Y X X X X i i i i i i i i i i i i X X R R S S T T a a f f
12b - 68 © 2000 Prentice-Hall, Inc. Dummy-Variable Model Example Same slopes
12b - 69 © 2000 Prentice-Hall, Inc. Selecting Variables in Model Building
12b - 70 © 2000 Prentice-Hall, Inc. Selecting Variables in Model Building A Butterfly Flaps its Wings in Japan, Which Causes It to Rain in Nebraska. -- Anonymous Use Theory Only! Use Computer Search!
12b - 71 © 2000 Prentice-Hall, Inc. Model Building with Computer Searches 1.Rule: Use as Few X Variables As Possible 2.Stepwise Regression Computer Selects X Variable Most Highly Correlated With Y Computer Selects X Variable Most Highly Correlated With Y Continues to Add or Remove Variables Depending on SSE Continues to Add or Remove Variables Depending on SSE 3.Best Subset Approach Computer Examines All Possible Sets Computer Examines All Possible Sets
12b - 72 © 2000 Prentice-Hall, Inc. Conclusion 1.Described Curvilinear Regression Models 2.Summarized Interaction Models 3.Explained Models with Qualitative Variables 4.Evaluated Portions of Regression Models 5.Described Stepwise Regression Analysis
End of Chapter Any blank slides that follow are blank intentionally.