B AD 6243: Applied Univariate Statistics Multiple Regression Professor Laku Chidambaram Price College of Business University of Oklahoma

BAD 6243: Applied Univariate Statistics 2 Basics of Multiple Regression Multiple regression examines the relationship between one interval/ratio level variable and two or more interval/ratio (or dichotomous) variables As in simple regression, the dependent (or criterion) variable is y and the other variables are the independent (or predictor) variables x i The intent of the regression model is to find a linear combination of x’s that best correlate with y The model is expressed as: Y =  0 +  1 X i +  2 X 2 … +  n X n +  I

BAD 6243: Applied Univariate Statistics 3 A Graphical Representation Objective: To graphically represent the equation Y =  0 +  1 Exp_X 1 +  2 RlExp_X 2 +  I

BAD 6243: Applied Univariate Statistics 4 Selecting Predictors Rely on theory to inform selection Examine correlation matrix to determine strength of relationships with Y Use variables based on your knowledge Let the computer decide based on data set

BAD 6243: Applied Univariate Statistics 5 Selecting Method of Inclusion Enter Enter – Block Stepwise –Forward selection –Backward elimination –Stepwise

BAD 6243: Applied Univariate Statistics 6 What to Look For? b-values vs. standardized beta weights (β) R: represents correlation between observed values and predicted values of Y R-squared: represents the amount of variance shared between Y and all the predictors combined Adjusted R-squared

BAD 6243: Applied Univariate Statistics 7 First Order Assumptions Continuous variables (also see next slide) Linear relationships between Y and Xs Sufficient variance in values of predictors Predictors uncorrelated with external variables

BAD 6243: Applied Univariate Statistics 8 Including Categorical Variables Dichotomous variables: e.g., Gender –Coded as 0 or 1 Dummy variables: e.g., Political affiliation –Create d - 1 dummy variables, where d is the number of categories –So, with four categories, you need three dummy variables Variable/ Category D1D2D3 Democrat100 Republican010 Libertarian001 Other000

BAD 6243: Applied Univariate Statistics 9 Second Order Assumptions Independence of independent variables Equality of variance Normal distribution of error terms Independence of observations

BAD 6243: Applied Univariate Statistics 10 Violations of Assumptions PROBLEMDEFINITIONDETECTION Multicollinearity Predictor variables are highly correlated High inter-correlations Examine VIFs and tolerances Heteroskedasticity Error terms do not have a constant variance Scatter plot of residuals Split file to examine variances Outliers Error terms not normally distributed Cook’s distance Mahalanobis’ distance Residual plots Autocorrelation Residuals are correlated Durbin-Watson  2 (If < 2, then + correlation If > 2, then – correlation)

BAD 6243: Applied Univariate Statistics 11 Multicollinearity High correlations among predictors Can result in: –Lower value of R –Difficulty of judging relative importance of predictors –Increases instability of model Possible solutions: –Examine correlation matrices, VIFs and tolerances to judge if predictor(s) need to be dropped –Rely on computer assisted means –Other options

BAD 6243: Applied Univariate Statistics 12 Heteroskedasticity Systematic increase or decrease in variance Can result in: –Confidence intervals being too wide or narrow –Unstable estimates Possible solutions: –Transform data –Other options

BAD 6243: Applied Univariate Statistics 13 Outliers Undue influence of extreme values Can result in: –Incorrect estimates and inaccurate confidence intervals Possible solutions: –Identify and eliminate value(s), but … –Transform data –Other options

BAD 6243: Applied Univariate Statistics 14 Autocorrelation Observations are not independent (typically, observations over time) Can result in: –Lower standard error of estimate –Lower standardized beta values Possible solutions: –Search for key “missing” variables –Cochrane-Orcutt Procedure –Other options

Results of Analysis

Results of Analysis (contd.)

BAD 6243: Applied Univariate Statistics 17 A Graphical Representation