Spring 2007 Lecture 9Slide #1 More on Multivariate Regression Analysis Multivariate F-Tests Multicolinearity The EVILS of Stepwise Regression Intercept Dummies Interaction Effects –Interaction Dummies –Slope Dummies

Spring 2007 Lecture 9Slide #2 F-Tests F-tests are used to test for the statistical significance of the overall model fit. Normally, that’s redundant given that you already have the t-stats for the individual b’s. This amounts to a ratio of the explained variance by the residual variance, correcting for the number of observations and parameters. The F value is compared to the F-distribution, just like a t-distribution, to obtain a p-value. F-tests become useful when we need to assess nested models.

Spring 2007 Lecture 9Slide #3 F-Tests, continued The question is whether a more complex model adds to the explanatory power over a simpler model. E.g., using the Guns data, does including a set of gender/age (pm1024) and race (pb1064) variables improve the prediction of the violent crime rate (vio) variable after we’ve included average income (avginc) and population density (density)? To find out, we calculate an F-statistic for the model improvement: Where the complex model has K parameters, and the simpler model has K-H parameters. RSS{K-H} is the residual sum of squares for the simpler model RSS{K} is the residual sum of squares for the more complex model

Spring 2007 Lecture 9Slide #4 F-Testing a Nested Model Simpler Model: vio = b 0 + b 1 (avginc) + b 2 (density) More Complex Model: vio = b 0 + b 1 (avginc) + b 2 (density) + b 3 (pm1024) + b 4 (pb1064) Given the models, K = 5, K - H = 3, and n = Calculating the RSS’s involves running the two models, obtaining the RSS from each. For these models, RSS K = 60,820,479.3 and RSS K-H = 68,295, So: Given that df 1 = H (2) and df 2 = n-K (1168), the p-value of the model improvement shown in Appendix Table 5C (p. 762) is <0.01.

Spring 2007 Lecture 9Slide #5 In-Class Homework -- Set 1 Test for the additional explanatory power of the age/gender and race varables (pm1024 and pb1064) when modeling the murder and robbery rates (mur and rob). Use the simple model from the prior lecture exercise (avginc and density) as your base of comparison. Discuss the theoretical meaning of your results.

Spring 2007 Lecture 9Slide #6 Multicolinearity If any X i is a linear combination of other X’s in the model, b i cannot be estimated –Remember that the partial regression coefficient strips both the X’s and Y of the overlapping covariation: –If an X is perfectly predicted by the other X’s, then:

Spring 2007 Lecture 9Slide #7 Multicolinearity, Continued We rarely find perfect multicolinearity in practice, but high multicolinearity results in loss of statistical resolution: –Large standard errors –Low t-stats, high p-values Erodes hypothesis tests –Enormous sensitivity to small changes in: Data Model specification

Spring 2007 Lecture 9Slide #8 Detecting Multicolinearity Use of VIF measures: –Variance Inflation Factor Degree to which other coefficients’ variance is increased due to the inclusion of the specified variable 1/VIF is equivalent to: (AKA “Tolerance”) Acceptable tolerance? Partly a function of n-size: If n < 50, tolerance should exceed 0.7 If n < 300, tolerance should exceed 0.5 If n < 600, tolerance should exceed 0.3 If n < 1000, tolerance should exceed 0.1 Calculating Tolerance in Stata (using VIF)

Spring 2007 Lecture 9Slide #9 Addressing Multicolinearity Drop one or other of co-linear variables Caution: May result in model mis-specification Use theory as a guide Add new data Special samples may maximize independent variation –E.g.: Elite samples may disentangle income, education Data Scaling Highly correlated variables may represent multiple indicators of an underlying scale Data reduction through factor analysis More on this later

Spring 2007 Lecture 9Slide #10 In-Class Homework: Set 2 Run a VIF on the “more complex” model from the nested F-test exercise (from lecture slide #5) Now add the variable pw1064 (percent of the population white, ages 10-64). Run VIF again. Using the pm1024 and pb1064 variables –F-Test will show significant improvement (over the reduced model from earlier) –But not all individual coefficients were statistically significant What gives? –Correlate pb1064, pw1064 and pm1029 –Strategy? Drop either pw1064 or pb1064 (use theory to decide which) Re-run the model

Spring 2007 Lecture 9Slide #11 Evils of Mindless Regression Stata permits a number of mechanical “search strategies” –Stepwise regression (forward, backward, upside down) How they work: use of sequential F-tests These methods pose serious problems: –If X’s are strongly related, one or the other will tend to be excluded –Susceptible to inclusion of spuriously related variables –Obliterates the meaning of classical statistical tests Hypothesis tests depend on prior hypothesis identification Example: The infamous Swedish “EMF study” Hundreds of diseases regressed on proximity to EMF sources –about 5% found to be “statistically significant” Enormous policy implications -- but method was buried in report

Spring 2007 Lecture 9Slide #12 Dummy Intercept Variables Dummy variables allow for tests of the differences in overall value of the Y for different nominal groups in the data (akin to a difference of means) Coding: 0 and 1 values (e.g., men versus women) Y X1X1 X 2,0 X 2,1

Spring 2007 Lecture 9Slide #13 Modeling Robbery as a function of Population Density and “Shall Issue” laws Source | SS df MS Number of obs = F( 2, 1170) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = rob | Coef. Std. Err. t P>|t| [95% Conf. Interval] density | shall | _cons | Robbery rates are systematically lower in “Shall Issue” states (is this necessarily causal??)

Spring 2007 Lecture 9Slide #14 Dummy Variable Applications Implies a comparison (the omitted group) –Be clear about the “comparison category” Multinomial Dummies –When categories exceed 2 Importance of specifying the base category Examples of Category Variables –Experimental treatment groups –Race and ethnicity –Region of residence –Type of education –Religious affiliation –“Seasonality” Adds to modeling flexibility

Spring 2007 Lecture 9Slide #15 Interaction Effects Interactions occur when the effect of one X is dependent on the value of another Modeling interactions: –Use Dummy variables (requires categories) –Use multiplicative interaction effect Multiply an interval scale times a dummy (also known as a “slope dummy”) Example: the effect of Population Density (density) on Robbery Rate (rob) may be affected by whether the respondent in in a Shall Issue state or not –Re-code the interaction; run it.

Spring 2007 Lecture 9Slide #16 Modeling Robbery with a Dummy Slope Variable: Density * Shall Source | SS df MS Number of obs = F( 3, 1169) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = rob | Coef. Std. Err. t P>|t| [95% Conf. Interval] density | shall | i | _cons |

Spring 2007 Lecture 9Slide #17 Illustration of Slope Interaction

Spring 2007 Lecture 9Slide #18 Coming Up... Assumptions of OLS Revisited Visual Diagnostic Techniques Non-linearity Non-normality and Heteroscedasticity Outliers and Case Statistics Aren’t we having fun now?