Korelasi Parsial dan Pengontrolan Parsial Pertemuan 14 Matakuliah : I0174 – Analisis Regresi Tahun : Ganjil 2007/2008 Korelasi Parsial dan Pengontrolan Parsial Pertemuan 14

Chapter Topics The Multiple Regression Model Residual Analysis Testing for the Significance of the Regression Model Inferences on the Population Regression Coefficients Testing Portions of the Multiple Regression Model Dummy-Variables and Interaction Terms Bina Nusantara

The Multiple Regression Model Relationship between 1 dependent & 2 or more independent variables is a linear function Population Y-intercept Population slopes Random error Dependent (Response) variable Independent (Explanatory) variables Bina Nusantara

Multiple Regression Model Bivariate model Bina Nusantara

Multiple Regression Equation Bivariate model Multiple Regression Equation Bina Nusantara

Multiple Regression Equation Too complicated by hand! Ouch! Bina Nusantara

Interpretation of Estimated Coefficients Slope (bj ) Estimated that the average value of Y changes by bj for each 1 unit increase in Xj , holding all other variables constant (ceterus paribus) Example: If b1 = -2, then fuel oil usage (Y) is expected to decrease by an estimated 2 gallons for each 1 degree increase in temperature (X1), given the inches of insulation (X2) Y-Intercept (b0) The estimated average value of Y when all Xj = 0 Bina Nusantara

Multiple Regression Model: Example (0F) Develop a model for estimating heating oil used for a single family home in the month of January, based on average temperature and amount of insulation in inches. Bina Nusantara

Multiple Regression Equation: Example Excel Output For each degree increase in temperature, the estimated average amount of heating oil used is decreased by 5.437 gallons, holding insulation constant. For each increase in one inch of insulation, the estimated average use of heating oil is decreased by 20.012 gallons, holding temperature constant. Bina Nusantara

Multiple Regression in PHStat PHStat | Regression | Multiple Regression … Excel spreadsheet for the heating oil example Bina Nusantara

Venn Diagrams and Explanatory Power of Regression Variations in Oil explained by the error term Variations in Temp not used in explaining variation in Oil Oil Variations in Oil explained by Temp or variations in Temp used in explaining variation in Oil Temp Bina Nusantara

Venn Diagrams and Explanatory Power of Regression (continued) Oil Temp Bina Nusantara

Venn Diagrams and Explanatory Power of Regression Variation NOT explained by Temp nor Insulation Overlapping variation in both Temp and Insulation are used in explaining the variation in Oil but NOT in the estimation of nor Oil Temp Insulation Bina Nusantara

Coefficient of Multiple Determination Proportion of Total Variation in Y Explained by All X Variables Taken Together Never Decreases When a New X Variable is Added to Model Disadvantage when comparing among models Bina Nusantara

Venn Diagrams and Explanatory Power of Regression Oil Temp Insulation Bina Nusantara

Adjusted Coefficient of Multiple Determination Proportion of Variation in Y Explained by All the X Variables Adjusted for the Sample Size and the Number of X Variables Used Penalizes excessive use of independent variables Smaller than Useful in comparing among models Can decrease if an insignificant new X variable is added to the model Bina Nusantara

Coefficient of Multiple Determination Excel Output Adjusted r2 reflects the number of explanatory variables and sample size is smaller than r2 Bina Nusantara

Interpretation of Coefficient of Multiple Determination 96.56% of the total variation in heating oil can be explained by temperature and amount of insulation 95.99% of the total fluctuation in heating oil can be explained by temperature and amount of insulation after adjusting for the number of explanatory variables and sample size Bina Nusantara

Simple and Multiple Regression Compared The slope coefficient in a simple regression picks up the impact of the independent variable plus the impacts of other variables that are excluded from the model, but are correlated with the included independent variable and the dependent variable Coefficients in a multiple regression net out the impacts of other variables in the equation Hence, they are called the net regression coefficients They still pick up the effects of other variables that are excluded from the model, but are correlated with the included independent variables and the dependent variable Bina Nusantara

Simple and Multiple Regression Compared: Example Two Simple Regressions: Multiple Regression: Bina Nusantara

Simple and Multiple Regression Compared: Slope Coefficients Bina Nusantara

Simple and Multiple Regression Compared: r2 = Bina Nusantara

Example: Adjusted r2 Can Decrease Adjusted r 2 decreases when k increases from 2 to 3 Color is not useful in explaining the variation in oil consumption. Bina Nusantara

Using the Regression Equation to Make Predictions Predict the amount of heating oil used for a home if the average temperature is 300 and the insulation is 6 inches. The predicted heating oil used is 278.97 gallons. Bina Nusantara

Predictions in PHStat PHStat | Regression | Multiple Regression … Check the “Confidence and Prediction Interval Estimate” box Excel spreadsheet for the heating oil example Bina Nusantara

Residual Plots Residuals Vs Residuals Vs Time May need to transform Y variable May need to transform variable May need to transform variable Residuals Vs Time May have autocorrelation Bina Nusantara

Residual Plots: Example Maybe some non-linear relationship No Discernable Pattern Bina Nusantara

Testing for Overall Significance Shows if Y Depends Linearly on All of the X Variables Together as a Group Use F Test Statistic Hypotheses: H0: 1 = 2 = … = k = 0 (No linear relationship) H1: At least one i  0 ( At least one independent variable affects Y ) The Null Hypothesis is a Very Strong Statement The Null Hypothesis is Almost Always Rejected Bina Nusantara

Testing for Overall Significance (continued) Test Statistic: Where F has k numerator and (n-k-1) denominator degrees of freedom Bina Nusantara

Test for Overall Significance Excel Output: Example p-value k = 2, the number of explanatory variables n - 1 Bina Nusantara

Test for Overall Significance: Example Solution H0: 1 = 2 = … = k = 0 H1: At least one j  0  = .05 df = 2 and 12 Critical Value: Test Statistic: Decision: Conclusion:  F 168.47 (Excel Output) Reject at  = 0.05.  = 0.05 There is evidence that at least one independent variable affects Y. F 3.89 Bina Nusantara

Test for Significance: Individual Variables Show If Y Depends Linearly on a Single Xj Individually While Holding the Effects of Other X’s Fixed Use t Test Statistic Hypotheses: H0: j = 0 (No linear relationship) H1: j  0 (Linear relationship between Xj and Y) Bina Nusantara

t Test Statistic Excel Output: Example t Test Statistic for X1 (Temperature) t Test Statistic for X2 (Insulation) Bina Nusantara

t Test : Example Solution Does temperature have a significant effect on monthly consumption of heating oil? Test at  = 0.05. Test Statistic: Decision: Conclusion: H0: 1 = 0 H1: 1  0 df = 12 Critical Values: t Test Statistic = -16.1699 Reject H0 at  = 0.05. Reject H Reject H There is evidence of a significant effect of temperature on oil consumption holding constant the effect of insulation. .025 .025 t -2.1788 2.1788 Bina Nusantara

Venn Diagrams and Estimation of Regression Model Only this information is used in the estimation of Only this information is used in the estimation of Oil This information is NOT used in the estimation of nor Temp Insulation Bina Nusantara

Confidence Interval Estimate for the Slope Provide the 95% confidence interval for the population slope 1 (the effect of temperature on oil consumption). -6.169  1  -4.704 We are 95% confident that the estimated average consumption of oil is reduced by between 4.7 gallons to 6.17 gallons per each increase of 10 F holding insulation constant. We can also perform the test for the significance of individual variables, H0: 1 = 0 vs. H1: 1  0, using this confidence interval. Bina Nusantara

Contribution of a Single Independent Variable Let Xj Be the Independent Variable of Interest Measures the additional contribution of Xj in explaining the total variation in Y with the inclusion of all the remaining independent variables Bina Nusantara

Contribution of a Single Independent Variable From ANOVA section of regression for From ANOVA section of regression for Measures the additional contribution of X1 in explaining Y with the inclusion of X2 and X3. Bina Nusantara

Coefficient of Partial Determination of Measures the proportion of variation in the dependent variable that is explained by Xj while controlling for (holding constant) the other independent variables Bina Nusantara

Coefficient of Partial Determination for (continued) Example: Model with two independent variables Bina Nusantara

Venn Diagrams and Coefficient of Partial Determination for Oil = Temp Insulation Bina Nusantara

Coefficient of Partial Determination in PHStat PHStat | Regression | Multiple Regression … Check the “Coefficient of Partial Determination” box Excel spreadsheet for the heating oil example Bina Nusantara

Contribution of a Subset of Independent Variables Let Xs Be the Subset of Independent Variables of Interest Measures the contribution of the subset Xs in explaining SST with the inclusion of the remaining independent variables Bina Nusantara

Contribution of a Subset of Independent Variables: Example Let Xs be X1 and X3 From ANOVA section of regression for From ANOVA section of regression for Bina Nusantara

Testing Portions of Model Examines the Contribution of a Subset Xs of Explanatory Variables to the Relationship with Y Null Hypothesis: Variables in the subset do not improve the model significantly when all other variables are included Alternative Hypothesis: At least one variable in the subset is significant when all other variables are included Bina Nusantara

Testing Portions of Model (continued) One-Tailed Rejection Region Requires Comparison of Two Regressions One regression includes everything Another regression includes everything except the portion to be tested Bina Nusantara

Partial F Test for the Contribution of a Subset of X Variables Hypotheses: H0 : Variables Xs do not significantly improve the model given all other variables included H1 : Variables Xs significantly improve the model given all others included Test Statistic: with df = m and (n-k-1) m = # of variables in the subset Xs Bina Nusantara

Partial F Test for the Contribution of a Single Hypotheses: H0 : Variable Xj does not significantly improve the model given all others included H1 : Variable Xj significantly improves the model given all others included Test Statistic: with df = 1 and (n-k-1 ) m = 1 here Bina Nusantara

Testing Portions of Model: Example Test at the  = .05 level to determine if the variable of average temperature significantly improves the model, given that insulation is included. Bina Nusantara

Testing Portions of Model: Example H0: X1 (temperature) does not improve model with X2 (insulation) included H1: X1 does improve model  = .05, df = 1 and 12 Critical Value = 4.75 (For X1 and X2) (For X2) Conclusion: Reject H0; X1 does improve model. Bina Nusantara

Testing Portions of Model in PHStat PHStat | Regression | Multiple Regression … Check the “Coefficient of Partial Determination” box Excel spreadsheet for the heating oil example Bina Nusantara

Do We Need to Do This for One Variable? The F Test for the Contribution of a Single Variable After All Other Variables are Included in the Model is IDENTICAL to the t Test of the Slope for that Variable The Only Reason to Perform an F Test is to Test Several Variables Together Bina Nusantara

Dummy-Variable Models Categorical Explanatory Variable with 2 or More Levels Yes or No, On or Off, Male or Female, Use Dummy-Variables (Coded as 0 or 1) Only Intercepts are Different Assumes Equal Slopes Across Categories The Number of Dummy-Variables Needed is (# of Levels - 1) Regression Model Has Same Form: Bina Nusantara

Dummy-Variable Models (with 2 Levels) Given: Y = Assessed Value of House X1 = Square Footage of House X2 = Desirability of Neighborhood = Desirable (X2 = 1) Undesirable (X2 = 0) 0 if undesirable 1 if desirable Same slopes Bina Nusantara

Dummy-Variable Models (with 2 Levels) (continued) Y (Assessed Value) Same slopes Desirable Location b0 + b2 Undesirable Intercepts different b0 X1 (Square footage) Bina Nusantara

Interpretation of the Dummy-Variable Coefficient (with 2 Levels) Example: : Annual salary of college graduate in thousand $ 0 non-business degree : GPA : 1 business degree With the same GPA, college graduates with a business degree are making an estimated 6 thousand dollars more than graduates with a non-business degree, on average. Bina Nusantara

Dummy-Variable Models (with 3 Levels) Bina Nusantara

Interpretation of the Dummy-Variable Coefficients (with 3 Levels) With the same footage, a Split-level will have an estimated average assessed value of 18.84 thousand dollars more than a Condo. With the same footage, a Ranch will have an estimated average assessed value of 23.53 thousand dollars more than a Condo. Bina Nusantara

Regression Model Containing an Interaction Term Hypothesizes Interaction between a Pair of X Variables Response to one X variable varies at different levels of another X variable Contains a Cross-Product Term Can Be Combined with Other Models E.g., Dummy-Variable Model Bina Nusantara

Effect of Interaction Given: Without Interaction Term, Effect of X1 on Y is Measured by 1 With Interaction Term, Effect of X1 on Y is Measured by 1 + 3 X2 Effect Changes as X2 Changes Bina Nusantara

Effect (slope) of X1 on Y depends on X2 value Interaction Example Y = 1 + 2X1 + 3X2 + 4X1X2 Y Y = 1 + 2X1 + 3(1) + 4X1(1) = 4 + 6X1 12 8 Y = 1 + 2X1 + 3(0) + 4X1(0) = 1 + 2X1 4 X1 0.5 1 1.5 Effect (slope) of X1 on Y depends on X2 value Bina Nusantara

Interaction Regression Model Worksheet Case, i Yi X1i X2i X1i X2i 1 3 2 4 8 5 40 6 30 : Multiply X1 by X2 to get X1X2 Run regression with Y, X1, X2 , X1X2 Bina Nusantara

Interpretation When There Are 3+ Levels MALE = 0 if female and 1 if male MARRIED = 1 if married; 0 if not DIVORCED = 1 if divorced; 0 if not MALE•MARRIED = 1 if male married; 0 otherwise = (MALE times MARRIED) MALE•DIVORCED = 1 if male divorced; 0 otherwise = (MALE times DIVORCED) Bina Nusantara

Interpretation When There Are 3+ Levels (continued) Bina Nusantara

Interpreting Results MALE Single: Married: Divorced: Difference FEMALE Single: Married: Divorced: MALE Single: Married: Divorced: Difference Main Effects : MALE, MARRIED and DIVORCED Interaction Effects : MALE•MARRIED and MALE•DIVORCED Bina Nusantara

Evaluating the Presence of Interaction with Dummy-Variable Suppose X1 and X2 are Numerical Variables and X3 is a Dummy-Variable To Test if the Slope of Y with X1 and/or X2 are the Same for the Two Levels of X3 Model: Hypotheses: H0: 4 = 5 = 0 (No Interaction between X1 and X3 or X2 and X3 ) H1: 4 and/or 5  0 (X1 and/or X2 Interacts with X3) Perform a Partial F Test Bina Nusantara

Evaluating the Presence of Interaction with Numerical Variables Suppose X1, X2 and X3 are Numerical Variables To Test If the Independent Variables Interact with Each Other Model: Hypotheses: H0: 4 = 5 = 6 = 0 (no interaction among X1, X2 and X3 ) H1: at least one of 4, 5, 6  0 (at least one pair of X1, X2, X3 interact with each other) Perform a Partial F Test Bina Nusantara

Chapter Summary Developed the Multiple Regression Model Discussed Residual Plots Addressed Testing the Significance of the Multiple Regression Model Discussed Inferences on Population Regression Coefficients Addressed Testing Portions of the Multiple Regression Model Discussed Dummy-Variables and Interaction Terms Bina Nusantara