1. REGRESSION
G&W p

2. THIS IS A TABLE YOU SHOULD KNOW BEFORE ANALYZING YOUR DATA AND THE FINAL
Dependent Variable
Independent Variable(s)
Method
Nominal/Ordinal
Chi-Square test
Interval/Ratio
T-test (2 groups) or ANOVA (3+ groups)
Multifactorial ANOVA (2+ IVs)
Regression
Interval/Ratio AND
ANCOVA/GLM

3. Basics of bivariate regression – When do we need it?
We have a dependent variable that is interval or ratio level.
We have an independent variable that is interval or ratio (there are ways of incorporating ordinal and nominal variables).
We have a posited directional association between the two variables.
We have a theory that supports the posited directional association.

4. Technique: Algebra of the straight line25
y = 2x + 3 20
(6,15)
Regression: y = bx + c
b = slope
= 6/3 = 2
c = y-intercept
= 3 15
6.0
(3,9) 10
3.0 5 1 2 3 4 5 6 7 8 9 10 x

5. Regression line – we have multiple data points
y : Reading Scores
x : Family Income
Assumptions about deviations from the line?

6. Technique: Statistics of the straight line
We have multiple data points (not just two)
We need to find a line that best fits them
Define “best fit” – Minimum squared deviations
slope
intercept
Independent variable
Error
Dependent variable
Regression coefficients

7. Technique: Statistics of the straight line
The line that “best fits” has the following slope:
Standard deviation of y
Standard deviation of x
Regression coefficient of y on x.
Correlation coefficient of x and y
The line that “best fits” has the following intercept:

8. Things to know
A significant regression coefficient does not “prove” causality
A regression can provide “predicted” values
Predicted values are “point” estimates
Confidence intervals around the point estimates provide very important information

9. Basics of linear regression
Hypothesis: a variable(s) x (x1, x2, x3, …) cause(s) another variable y
Corollary: x can be used to partially predict y
Mathematical implication:
This is mimimal and random
This is linear

10. Example

11. Example: Regression approach
Error in prediction is minimal, and random

12. The way these assumptions look
Child’s IQ = *Mother’s IQ

13. Prediction Child’s IQ = 20.99 + 0.78*Mother’s IQ
Predicted Case 3 IQ = *110
Predicted Case 3 IQ =
Actual Case 3 IQ = 102

14. Example

15. Example

16. Example

17. How are the regression coefficients computed?
MINIMIZE SQUARED DEVIATIONS BETWEEN ACTUAL AND PREDICTED VALUES
ε is “minimal” and random

18. Interpreting coefficients

19. Error in estimation of b
The estimate of b will differ from sample to sample.
There is sampling error in the estimate of b.
b is not equal to the population value of the slope (B).
If we take many many simple random samples and estimate b many many times ….

20. Standard error of b

21. R2 = + 1 = + 1- R2 = Sum of squared errors of regression
Sum of squared deviations from the mean only
Variance due to regression
Total variance
Error variance = +
Proportion of variance due to regression
Proportion of variance due to error
1 = +