Multivariate Data Analysis Chapter 4 – Multiple Regression

What is Multiple Regression Analysis? An Example of Simple and Multiple Regression Setting a Baseline: Prediction Without an Independent Variable Prediction Using A Single Independent Variable – Simple Regression The Role of the Correlation Coefficient Specifying the Simple Regression Equation Establishing a Confidence Interval for the Prediction Assessing Prediction Accuracy

What is Multiple Regression Analysis? Prediction Using Several Independent Variables – Multiple Regression The Impact of Multicollinearity The Multiple Regression Equation Adding a Third Independent Variable Summary: simple and straightforward dependence technique

A Decision Process for Multiple Regression Analysis Stage 1: Objectives of Multiple Regression Research Problems Appropriate for Multiple Regression Prediction with Multiple Regression Explanation with Multiple Regression Specifying a Statistical Relationship Selection of Dependent and Independent Variables

A Decision Process for Multiple Regression Analysis (Cont.) Stage 2: Research Design of a Multiple Regression Analysis Sample Size Statistical Power and Sample Size Generalizability and Sample Size Fixed Versus Random Effects Predictors Creating Additional Variables Incorporating Nonmetric Data with Dummy Variables Representing Curvilinear Effects with Polynomials Representing Interaction or Moderator Effects

A Decision Process for Multiple Regression Analysis (Cont.) Stage 3: Assumptions in Multiple Regression Analysis Assessing Individual Variables Versus the Variate Linearity of the Phenomenon Constant Variance of the Error Term Independence of the Error Terms Normality of the Error Term Distribution

A Decision Process for Multiple Regression Analysis (Cont.) Stage 4: Estimating the Regression Model and Assessing Overall Fit General Approaches to Variables Selection Confirmatory Specification Sequential Search Methods Combinational Approach Overview of the Model Selection Approaches Testing the Regression Variate for Meeting the Regression Assumptions

A Decision Process for Multiple Regression Analysis (Cont.) Stage 4: Estimating the Regression Model and Assessing Overall Fit (Cont.) Examining the Statistical Significance of Our Model Significance of the Overall Model: The Coefficient of Determination Significance Tests of Regression Coefficients Identifying Influential Observations

A Decision Process for Multiple Regression Analysis (Cont.) Stage 5: Interpreting the Regression Variate Using the Regression Coefficients Standardizing the Regression Coefficients: Beta Coefficients Assessing Multicollinearity The Effect of Multicollinearity Identifying Multicollinearity Remedies for Multicollinearity

A Decision Process for Multiple Regression Analysis (Cont.) Stage 6: Validation of the Results Additional or Split Samples Calculating the PRESS Statistics Comparing Regression Models Predicting with the Model

Chapter 4A Assessing Multicollinearity A Two-Part Process An Illustration of Assessing Multicollinearity

Chapter 4a: Identifying Influential Observations Step 1: Examining Residuals Analysis of Residuals Partial regression plots Step 2: Identifying Leverage Points from the Predictors Hat Matrix Mahalanobis distance (D^2)

Identifying Influential Observations (Cont.) Step 3: Single-Case Diagnostics Identifying Influential Observations Influences on individual coefficients Overall influence measures Step 4: Selecting and Accommodating Influential Observations