1. Regression

2. Remembering the basics of linear regression
Hypothesis: a variable(s) x (x1, x2, x3, …) cause(s) another variable y
Corollary: x can be used to partially predict y
Mathematical implication:
This is mimimal and random
This is linear

3. Example

4. Example: Regression approach
Error in prediction is minimal, and random

5. The way these assumptions look
Child’s IQ = *Mother’s IQ

6. Prediction Child’s IQ = 20.99 + 0.78*Mother’s IQ
Predicted Case 3 IQ = *110
Predicted Case 3 IQ =
Actual Case 3 IQ = 102

7. Example

8. Example

9. Example

10. How are the regression coefficients computed?
MINIMIZE SQUARED DEVIATIONS BETWEEN ACTUAL AND PREDICTED VALUES
ε is “minimal” and random

11. Interpreting coefficients

12. Error in estimation of b
The estimate of b will differ from sample to sample.
There is sampling error in the estimate of b.
b is not equal to the population value of the slope (B).
If we take many many simple random samples and estimate b many many times ….

13. Standard error of b

14. R2 = + 1 = + 1- Sum of squared errors of regression
Sum of squared deviations from the mean only
Variance due to regression
Total variance
Error variance = +
Proportion of variance due to regression
Proportion of variance due to error
1 = +

15. Multiple Regression

16. Multiple regression Purpose: Include “relevant” predictors
Reduce error in prediction
Test hypotheses about related independent variables

17. Matrices

18. Questions about matrices
If A is a 3*4 matrix, what is the dimensions of A’ (A transposed)?
If A is a 4*5 matrix and B is a column vector with 5 elements (5*1), what are the dimensions of C=A*B?
If A is a 3*2 matrix, what are the dimensions of A-1?

19. Question

20. Answer

21. Vectors and Matrices

22. Vectors and Matrices

23. Multiple regression – OLS estimation

24. Multiple regression – OLS estimation
k*1
n*1
n*1
n*k

25. Multiple regression – OLS estimation
(k*1)  {(k*n)*(n*k)} {(k*n)*(n*1)}
(k*1)  {(k*k)} * {(k*1)}

26. Multiple regression – OLS estimation
Information matrix : each element
is a cross-product term

27. OLS estimation: important points
Information matrix is inverted
Information matrix cannot be “singular.”
Information matrix cannot have any two rows or columns that are (almost) identical.
Every regression coefficient depends on the entire information matrix.
Every regression coefficient depends on the covariances of all Xs with Y.

28. Assumptions of multiple regression
Equal probability of selection (SRS)
Linearity
Independence of observations:
Errors are uncorrelated
The mean of error term is ALWAYS zero:
Mean does not depend on x.
Normality
Homoskedasticity
Variance does not depend on x.
No multicollinearity