Research Methods I Lecture 10: Regression Analysis on SPSS

Introduction Lecture 8: looked at descriptive statistics; relationships between variables: correlation; cross-tabulation This lecture: regression Focus on how regression can be done on SPSS Focus on OLS, although SPSS is capable of WLS and logistic regression

Regression: fundamentals Relationship between two variables lead to prediction of values of a dependent variable Prediction of Y i based on Y(bar), model (effect of X on Y) and residual Regression equation: Y^ = a^ + b^X + e^

Regression: fundamentals Actual values of X will predict values of Y^ based upon intercept term (a^), slope coefficient (b^) and residual (e^) Ordinary Least Squares finds values of a^ and b^ which minimise sum of squared residuals from the regression Under certain assumptions, OLS is BLUE

Evaluating the regression SPSS allows various ways to evaluate the generated regression equation: Goodness of fit Individual significance of regressors Tests of classical assumptions (necessary for inference)

Goodness of fit R 2 =ESS/TSS F-test: F=ESS/TSS Check outliers via standardised residuals or studentised residuals Can check for significant outliers, i.e., ones which would affect the value of the regressors

Significant outliers Cook’s distance (critical>1) Leverage values (critical: > 2(k+1)/n Mahalanobis distance Covariance ratio: if CVR i >/<1 + [3(k+1)/n], deleting case damages/improves parameter precision Casewise diagnostics CI of regression coefficients

Methods of regression Hierarchical: when there is a good statistical or theoretical reason for including one variable first Forward (specific to general) or Backward (general to specific): based on statistical criteria Stepwise: forward + removal test Use Enter command to do usual regression

Regression equation Individual coefficients: unstandardised b and standardised b SPSS will give t-statistics, s.e.(b), and p-value for significance of b If p<0.05, coefficient significant at 5% This is all OK for description but for inference, require specific assumptions: need to test these

“Diagnostic” tests No direct specification test (no equivalent of RESET in Microfit) Variation in ind. variables (  2 ≠0): test by prior observation SPSS provides many tests for multicollinearity: e.g. VIF (critical value=1), tolerance (=1/VIF), covariance matrix, partial correlations between regressors

“Diagnostic” tests Autocorrelation: DW test (DW(2) not directly available) Linearity and homoscedasticity can be tested via scatter plots of standardised residuals versus standardised predicted residuals (from the assump) Normality: ask SPSS for histogram and P-P plot of the regression residuals (can also do K-S and S-W tests)

Conclusions SPSS is capable of doing regression analysis of various types SPSS offers a range of tests of specification, fit and underlying assumptions of regression Tests are more extensive but less user- friendly than a package like Microfit