Multiple Regression

Outline Purpose and logic : page 3 Purpose and logic : page 3 Parameters estimation : page 9 Parameters estimation : page 9 R-square : page 13 R-square : page 13 Hypothesis testing : page 17 Hypothesis testing : page 17 Partial and semi-partial regression : page 21 Partial and semi-partial regression : page 21 Confidence intervals : page 26 Confidence intervals : page 26

Multiple Regression There are more than one predictor There are more than one predictor Example with 2 predictors Example with 2 predictors

Multiple Regression Because we have two predictors, it is possible to illustrate the situation using a 3-dimensional scatter plot Because we have two predictors, it is possible to illustrate the situation using a 3-dimensional scatter plot

Multiple Regression The different relations can be illustrated by a bivariate scatter plots. The different relations can be illustrated by a bivariate scatter plots. x1x1 x1x1 x2x2 y x2x2 y

Multiple Regression As well as the bivariate correlations (see SSCP). As well as the bivariate correlations (see SSCP).

Multiple Regression From the regression line to the regression hyperplane From the regression line to the regression hyperplane

Multiple Regression Graphics Graphics It is not possible to illustrates vectors in 5 dimensions. However, the computations will be the same.

Multiple Regression How the regression coefficients can be obtained ? Universal formula, it does not change whatever the number of predictors (it is a special case of multivariate regression).

Multiple Regression The b 0 coefficient can also be directly obtained if we include the unity vector 1 as a variable.

Multiple Regression How the regression coefficients can be obtained ?

Standardized Regression Coefficients It allow to measure the « importance » of the predictors, since they all have a variability of 1 and a mean of 0. Therefore, an increase of one unit by z 1, will increase 0.74 standard deviation by y Z. ^ Or

R-Square Like in simple regression, in multiple regression we will use the R-square coefficient (R 2 ); also named coefficient of determination. This R 2 have the same interpretation as the one in simple regression: percentage of explained variance given by all the predictors. Sum of Squares and Cross Product Matrix (SSCP)

R-Square Recall: By dividing the SSCP matrix by the corresponding degrees of freedom, we get the variance-covariance matrix. We can also get the bivariate correlations

R-Square In addition, the SSCP matrix can be partitioned in function of the independent (predictors) and dependent variables (criterion). Scp Spp Spc Scc

R-Square R 2 is obtained by: R 2 adj is an unbiased estimation of the population variability given a sample Scp Spp Spc Scc Number of participantsNumber of predictors (independent variables)

Hypothesis testing The hypothesis is that the R-square between the predictors and the criterion is null. In other words, we are trying to know if the X and y variables are linearly independent. If we reject that hypothesis, then the two populations are not independent, there is a linear relation between the two.

Hypothesis testing Using confidence intervals. Using observed statistics.

Hypothesis testing The hypothesis is that the R-Square between the predictors and the criterion is null. In other words, we are trying to know if the X and y variables are linearly independent. If we reject that hypothesis, then the two populations are not independent, there is a linear relation between the two. Because the F obs >F crit ( >19.00), we reject H 0 and we therefore accept H 1. The 2 populations are linearly dependant. Explained variabilityUnexplained variability

ANOVA Table The hypothesis is that the R-Square between the predictors and the criterion is null. Because F(2,2)= , p.<0.05, we reject H 0 and we accept H 1. The 2 populations are linearly dependant. =

Partial and semi-partial correlations The idea is to put forward the effect of one predictor on the dependant variable by controlling the effects of the other predictors. Squared semi partial correlation coefficient It is the total variability (R 2 ) minus the total variability excluding the studied predictor from the data set. It is the portion of variability that is unique to this particular predictor Squared partial correlation coefficient It is the variability proportion associated with one predictor but not with the others. In other words, it is the unestimated variability by the other predictors that is estimated by the predictor studied.

Partial and semi-partial correlations a b c e Y x1x1 x2x2

Example 39 % of the y variability is explained solely by the first predictor. 9% of the y variability is explained solely by the second predictor. 90% of the unexplained y variability by the second predictor is explained by the first predictor. 67% of the unexplained y variability by the first predictor is explained by the second predictor. x1x1 x2x2

Partial and semi-partial correlations Signification testing x1x1 x2x2 The various parameters (pr i, b i, B i ) are directly related from the portion of explained variability by the semi partial coefficient. Therefore, if this last coefficient is statistically significant, then all the other parameters will be.

Standard errors associated with the regression parameters Standard error associated with the regression coefficients Standard errors associated with the standardized regression coefficients

Confidance intervals associated with the regression parameters Confidence intervals associated with the regression coefficients Confidence intervals associated with the standardized regression coefficients