1. Statistics for the Social Sciences
Psychology 340
Spring 2005
Prediction

2. Outline (for week)
Simple bi-variate regression, least-squares fit line
The general linear model
Residual plots
Using SPSS
Multiple regression
Comparing models, (?? Delta r2)

3. Regression
Last time: with correlation, we examined whether variables X & Y are related
This time: with regression, we try to predict the value of one variable given what we know about the other variable and the relationship between the two.

4. Regression
Last time: “it doesn’t matter which variable goes on the X-axis or the Y-axis”
The variable that you are predicting goes on the Y-axis (criterion variable)
Predicted variable
For regression this is NOT the case Y X 1 2 3 4 5 6
Quiz performance
Predicting variable
The variable that you are making the prediction based on goes on the X-axis (predictor variable)
Hours of study

5. Regression Last time: “Imagine a line through the points”
But there are lots of possible lines Y X 1 2 3 4 5 6
One line is the “best fitting line”
Today: learn how to compute the equation corresponding to this “best fitting line”
Quiz performance
Hours of study

6. The equation for a line A brief review of geometry
Y = intercept, when X = 0 Y X 1 2 3 4 5 6
Y = (X)(slope) + (intercept)
2.0

7. The equation for a line A brief review of geometryX 1 2 3 4 5 6
Y = (X)(slope) + (intercept)
0.5
2.0 1 2
Change in Y
Change in X
= slope

8. The equation for a line A brief review of geometryX 1 2 3 4 5 6
Y = (X)(slope) + (intercept)
Y = (X)(0.5) + 2.0

9. Regression A brief review of geometry Consider a perfect correlation
X = 5
Y = ? Y X 1 2 3 4 5 6
Y = (X)(0.5) + (2.0)
Y = (5)(0.5) + (2.0)
Y = = 4.5
4.5
Can make specific predictions about Y based on X

10. Regression Consider a less than perfect correlation
The line still represents the predicted values of Y given X
X = 5
Y = ? Y X 1 2 3 4 5 6
Y = (X)(0.5) + (2.0)
Y = (5)(0.5) + (2.0)
Y = = 4.5
4.5

11. Regression
The “best fitting line” is the one that minimizes the error (differences) between the predicted scores (the line) and the actual scores (the points) Y X 1 2 3 4 5 6
Rather than compare the errors from different lines and picking the best, we will directly compute the equation for the best fitting line

12. Regression The linear model Y = intercept + slope (X) + error
Beta’s () are sometimes called parameters
Come in two types:
standardized
unstanderdized
Now let’s go through an example computing these things

13. Scatterplot Using the dataset from our correlation lecture Y X Y 6 6 6 5 4 3 2 1 X 1 2 3 4 5 6

14. From the Computing Pearson’s r lecture
mean
3.6
4.0
2.4
-2.6
1.4
-0.6
0.0
2.0
-2.0
4.8
5.2
2.8
1.2
5.76
6.76
1.96
0.36
X Y
14.0
15.20
16.0
SSY
SSX SP

15. Computing regression line (with raw scores)
X Y
mean
3.6
4.0
14.0
15.20
16.0
SSY
SSX SP

16. Computing regression line (with raw scores)Y X 1 2 3 4 5 6
X Y
mean
3.6
4.0

17. Computing regression line (with raw scores)Y
X Y 6 5 4
The two means will be on the line 3 2 1
mean
3.6
4.0 X 1 2 3 4 5 6

18. Computing regression line (standardized, using z-scores)
Sometimes the regression equation is standardized.
Computed based on z-scores rather than with raw scores
X Y
2.4
5.76
2.0
4.0
1.38
1.1
-2.6
6.76
-2.0
4.0
-1.49
-1.1
1.4
1.96
2.0
4.0
0.8
1.1
-0.6
0.36
0.0
0.0
- 0.34
0.0
-0.6
0.36
-2.0
4.0
- 0.34
-1.1
Mean
3.6
4.0
0.0
15.20
0.0
16.0
0.0
0.0
1.74
1.79
Std dev

19. Computing regression line (standardized, using z-scores)
Sometimes the regression equation is standardized.
Computed based on z-scores rather than with raw scores
Prediction model
Predicted Z score (on criterion variable) = standardized regression coefficient multiplied by Z score on predictor variable
Formula
1.38
1.1
-1.49
-1.1
0.8
1.1
- 0.34
0.0
- 0.34
-1.1
The standardized regression coefficient (β)
In bivariate prediction, β = r
0.0
0.0

20. Computing regression line (with z-scores)ZY 2 1
1.38
1.1 ZX
-1.49
-1.1 -2 -1 1 2 -1
0.8
1.1
- 0.34
0.0 -2
- 0.34
-1.1
0.0
0.0
mean

21. Regression The linear equation isn’t the whole thing
Y = intercept + slope (X)
+ error
The linear equation isn’t the whole thing
Also need a measure of error
Y = X(.5) + (2.0) + error
Y = X(.5) + (2.0) + error
Same line, but different relationships (strength difference) Y X 1 2 3 4 5 6 Y X 1 2 3 4 5 6

22. Regression Error Actual score minus the predicted score
Measures of error
r2 (r-squared)
Proportionate reduction in error
Note: Total squared error when predicting from the mean = SSTotal=SSY
Squared error using prediction model = Sum of the squared residuals = SSresidual= SSerror

23. R-squared r2 represents the percent variance in Y accounted for by X1 2 3 4 5 6 Y X 1 2 3 4 5 6
64% variance explained
25% variance explained

24. Computing Error around the line
Sum of the squared residuals = SSresidual = SSerror
Compute the difference between the predicted values and the observed values (“residuals”) Y X 1 2 3 4 5 6
Square the differences
Add up the squared differences

25. Computing Error around the line
Sum of the squared residuals = SSresidual = SSerror
X Y
mean
3.6
Predicted values of Y (points on the line)
4.0

26. Computing Error around the line
Sum of the squared residuals = SSresidual = SSerror
X Y
6.2
= (0.92)(6)+0.688
mean
3.6
Predicted values of Y (points on the line)
4.0

27. Computing Error around the line
Sum of the squared residuals = SSresidual = SSerror
X Y
6.2
= (0.92)(6)+0.688
1.6
= (0.92)(1)+0.688
5.3
= (0.92)(5)+0.688
3.45
= (0.92)(3)+0.688
3.45
= (0.92)(3)+0.688
mean
3.6
4.0

28. Computing Error around the line
Sum of the squared residuals = SSresidual = SSerror
X Y Y X 1 2 3 4 5 6
6.2
6.2
1.6
5.3
3.45
1.6
5.3
3.45
3.45

29. Computing Error around the line
Sum of the squared residuals = SSresidual = SSerror
residuals
X Y
6.2 =
-0.20
1.6 =
0.40
5.3 =
0.70
3.45 =
0.55
3.45
-1.45 =
mean
3.6
4.0
Quick check
0.00

30. Computing Error around the line
Sum of the squared residuals = SSresidual = SSerror
X Y
6.2
-0.20
0.04
1.6
0.40
0.16
5.3
0.70
0.49
3.45
0.55
0.30
3.45
-1.45
2.10
mean
3.6
4.0
0.00
3.09
SSERROR

31. Computing Error around the line
Sum of the squared residuals = SSresidual = SSerror
4.0
0.0
16.0
SSY
X Y
6.2
-0.20
0.04
1.6
0.40
0.16
5.3
0.70
0.49
3.45
0.55
0.30
3.45
-1.45
2.10
mean
3.6
4.0
0.00
3.09
SSERROR

32. Computing Error around the line
Sum of the squared residuals = SSresidual = SSerror
Proportionate reduction in error
Also (like r2) represents the percent variance in Y accounted for by X
In fact, it is mathematically identical to r2
16.0
3.09
SSY
SSERROR

33. Seeing patterns in the error
Residual plots
The sum of the residuals should always equal 0 (as should the mean).
the least squares regression line splits the data in half, half of the error is above the line and half is below the line.
In addition to summing to zero, we also want there the residuals to be randomly distributed.
That is, there should be no pattern to the residuals.
If there is a pattern, it may suggest that there is more than a simple linear relationship between the two variables.
Residual plots are very useful tools to examine the relationship even further.
These are basically scatterplots of the residuals () against the Explanatory (X) variable
(note: the examples actually plot the residuals that have transformed into z-scores).

34. Seeing patterns in the error
Scatter plot
Residual plot
The scatterplot shows a nice linear relationship.
The residual plot shows that the residuals fall randomly above and below the line. Critically there doesn't seem to be a discernable pattern to the residuals.

35. Seeing patterns in the error
Scatter plot
Residual plot
The residual plot shows that the residuals get larger as X increases.
This suggests that the variability around the line is not constant across values of X.
This is referred to as a violation of homogeniety of variance.
The scatterplot also shows a nice linear relationship.

36. Seeing patterns in the error
Scatter plot
Residual plot
The scatterplot shows what may be a linear relationship.
The residual plot suggests that a non-linear relationship may be more appropriate (see how a curved pattern appears in the residual plot).

37. Regression in SPSS Running the analysis is pretty easy
Analyze: Regression: Linear
Predictor variables go into the ‘independent variable’ field
(Predicted variable) goes into the “dependent variable’ field
You get a lot of output

38. Regression in SPSS The variables in the model r r2
We’ll get back to these numbers in a few weeks
Unstandardized coefficients
Slope (indep var name)
Intercept (constant)
Standardized coefficients

39. Multiple Regression Multiple regression prediction models “fit”
“residual”

40. Prediction in Research Articles
Bivariate prediction models rarely reported
Multiple regression results commonly reported