3.2. SIMPLE LINEAR REGRESSION Design and Data Analysis in Psychology II SalvadorChacón Moscoso Susana Sanduvete Chaves

1. Introduction Specifying regression equation is useful to: Describe variable relations in a simple and precise way. Predict values of one variable as a function of another one. Only linear relationship can be considered. Simple linear regression just specifies the relationship between two variables; the simplest possible relationship between variables.

VARIABLES TERMINOLOGY 1. Introduction VARIABLES TERMINOLOGY X Y Predictor, regressor criterion explanatory explained predetermined reply independent dependent exogenous endogenous (explains variability in the other variable) (its variability is explained by the other variable)

2. SCATTER PLOT INTERPRETATION Scatter plot provides information about: Linear relationship between variables. Type (shape) of relationship (linear or another one). Level of intensity of relationship (depending on how close group of scores are represented). Outliers can confound existing relation. Group of scores can or not be uniformed represented (homoscedasticity vs. heteroscedasticity).

3. SPECIFYING THE SIMPLE LINEAR REGRESSION MODEL x

3. SPECIFYING THE SIMPLE LINEAR REGRESSION MODEL : Is usually labeled: Error Perturbation Residual Basically, it depends on: Inaccurate variable measurement. Other variables influence not considered in the model.

3. SPECIFYING THE SIMPLE LINEAR REGRESSION MODEL

3.1. Model Assumptions Statistical conditions: Linearity. Homoscedasticity : Y variances in each X value are ‘equal’ (statistically homogeneous). Lack of autocorrelation: Y variables are uncorrelated (important issue in longitudinal studies). Normal distribution.

3.1. Model Assumptions Features of the descriptive model: Adequate model specification: Do not exclude relevant independent variables. Do not include irrelevant independent variables [‘INUS’; insufficient but necessary useful variables]. Independent variable must not have any measurement error.

4. Parameters Estimation α y β. Minimum Least squares (MLS) Using raw scores :

4. Parameters Estimation Using deviation scores : ‘b’ is the same in the formula for raw and deviation scores. Using standard scores :

4. An example of parameters estimation Based on data from previous example (lesson 3.1.), calculate the regression equation in raw, deviation and standard scores. X 2 4 6 8 10 12 14 16 18 20 Y 1 13 22

4. An example of parameters estimation Regression equation for raw scores :

4. An example of parameters estimation Regression equation with deviation scores: Regression equation with standard scores:

5. REGRESSION MODEL: INTERPRETATION In the linear regression model We distinguish the following elements: e  error or residuals: randomized part; what is not explained by the model.

5. REGRESSION MODEL: INTERPRETATION  predicted score: mean value anticipated for all the participants that obtained a value of Xi in the variable X. b  slope: change in Y for each unit of change in X. a  constant: mean value of Y when X=0.

5. REGRESSION MODEL: INTERPRETATION (EXAMPLE) We have the following regression equation: Where X is the number of years of professional experience, and Y is the salary. 1. Interpret a and b. 2. Supposing that a person presents 3 years of professional experience, what salary will this person obtain? Interpret the result. 3. If one person with 3 years of professional experience obtained a salary of 1700 €, which would the error be? Interpret the result.

5. REGRESSION MODEL: INTERPRETATION (EXAMPLE) Interpret a and b. b=300  change in Y for each unit of change in X. In each change of professional experience, the salary increases 300 €. a=600  mean value of Y when X=0. Mean salary of those people without any professional experience.

5. REGRESSION MODEL: INTERPRETATION (EXAMPLE) 2. Supposing that a person presents 3 years of professional experience, what salary will this person obtain? Interpret the result.  predicted score: mean value anticipated for all participants that obtained in the variable X a value of Xi. People that present 3 years of professional experience, have a mean salary of 1500 €.

5. REGRESSION MODEL: INTERPRETATION (EXAMPLE) 3. If one person with 3 years of professional experience obtained a salary of 1700 €, which would the error be? Interpret the result. The model estimates a salary of 1500 € for a person with 3 years of professional experience. If his real salary is 1700 €, this difference of 200 € is the error; the part the model does not explain.

6. COMPONENTS OF VARIATION

6. COMPONENTS OF VARIATION Total sum of squares = explained sum of squares + residual sum of squares Total variation = explained variation + residual variation

6. COMPONENTS OF VARIATION: EXAMPLE Determine the components of variation from the data used in the first example.

6. COMPONENTS OF VARIATION: EXAMPLE Total sum of squares calculation:

6. COMPONENTS OF VARIATION: EXAMPLE Explained sum of squares calculation:

6. COMPONENTS OF VARIATION: EXAMPLE Residual sum of squares calculation:

6. COMPONENTS OF VARIATION: EXAMPLE Checkout: TotalSS= ExplainedSS+ResidualSS

7. GOODNESS OF FIT - Coincides with the coefficient of determination. The proportion of unexplained variability = 1-R2

7. GOODNESS OF FIT

7. GOODNESS OF FIT : EXAMPLE Calculate the goodness of fit (with the three proposed formulas) and the proportion of unexplained variability.

7. GOODNESS OF FIT : EXAMPLE

8. MODEL VALIDATION How can I tell if my data fit a model? Sources of variation Sums of squares Degrees of Freedom Variances F Regression or explained k Residual or unexplained N-k-1 Total N-1

8. MODEL VALIDATION  Null hypothesis is rejected. The variables are related. The model is valid.  Null hypothesis is accepted. The variables are not related. The model is not valid. (k = number of independent variables)

8. MODEL VALIDATION Other possible formulas of ‘F’ : ‘F’ calculation with raw scores:

8. MODEL VALIDATION Using variances: Using coefficient of determination (R2) :

8. MODEL VALIDATION: EXAMPLE With the above data, calculate the ‘F’ (using the 4 proposed formulas) and conclude on the validity of the model.

8. MODEL VALIDATION: EXAMPLE Sources of variation Sums of squares Degrees of Freedom Variances F Regression or explained 183.158 1 19.025 19 Residual or unexplained 77.018 8 9.627 Total (approx.) 260.176 9 28.908

8. MODEL VALIDATION: EXAMPLE  Conclusion: Null hypothesis is rejected. The variables X and Y are related. The model is valid.

8. MODEL VALIDATION: EXAMPLE Other formulas:

8. MODEL VALIDATION: EXAMPLE

8. MODEL VALIDATION: EXAMPLE Which is the final conclusion? Significant effect Non-significant effect High effect size (≥ 0.67) The effect probably exists The non-significance can be due to low statistical power Low effect size (≤ 0.18) The statistical significance can be due to an excessive high statistical power The effect probably does not exist

8. MODEL VALIDATION: EXAMPLE Which is the final conclusion? Significant effect Non-significant effect High effect size 0.8392 = 0.704 (≥ 0.67) The effect probably exists The non-significance can be due to low statistical power Low effect size (≤ 0.18) The statistical significance can be due to an excessive high statistical power The effect probably does not exist

9. SIGNIFICANCE OF REGRESSION PARAMETERS b study (related to the independent variable). In simple linear regression, test of significance equivalent to F and to the significance of rXY More interesting in multiple linear regression, where in spite of obtaining a significant global F some of the regression parameters may not be significant.

9. SIGNIFICANCE OF REGRESSION PARAMETERS Hypothesis: H0: β = 0 H1: β = 0

9. SIGNIFICANCE OF REGRESSION PARAMETERS

9. SIGNIFICANCE OF REGRESSION PARAMETERS  Null hypothesis is rejected. The model is valid. The slope is statistically different from 0. There is, therefore, relationship between variables.  Null hypothesis is accepted. The model is not valid. The slope is statistically equal to 0. There is not, therefore, relationship between variables.

9. SIGNIFICANCE OF REGRESSION PARAMETERS: EXAMPLE Using previous data, determine ‘b’ parameter significance.

9. SIGNIFICANCE OF REGRESSION PARAMETERS: EXAMPLE Conclusion: reject the null hypothesis. The model is valid. The slope is statistically different from 0. There is, therefore, relationship between variables.

10. PREDICTION A specific value: Which Y value will have a person with X = 4?

10. PREDICTION Whithin a confidence interval:

10. PREDICTION: EXAMPLE Which will be the score interval Y of the person who obtained X = 4?

10. PREDICTION: EXAMPLE

10. PREDICTION: EXAMPLE Conclusion: There is a probability of 0.95 that a person having a value of X = 4, obtains a score between -2.808 and 13.178.

10. PREDICTION: LIMITATIONS Do not extrapolate values beyond the observational data. What would happen if there were a quadratic relationship? Predicted score Yield Yield Real score Anxiety Anxiety