Regression Review Evaluation Research (8521) Prof. Jesse Lecy Lecture 1 1

Regression Review: Regression Error and the Standard Error of the Regressors

What should be clear in my mind? Variance and standard deviation. What is a sampling distribution? Difference between the standard deviation and standard error. What role the standard error plays in the confidence interval. Definitions of covariance and correlation. The “intuitive” regression formula.

The Road Map Variance  St. Dev  St. Error  Confidence Intervals Mean: Slope: (of x) (of the residual) (of the slope) (of the mean)

VARIANCE

KevinLarry Who travels the furthest?

KevinLarry

STD. DEVIATION VS. STD. ERROR

You first need a reference point in order to calculate average distance. You can’t ask “how far have I traveled if I am in Chicago?” without knowing where you started from. Use the mean as the starting point for each distance. Variance is a measure of average dispersion. So you add up all the distances. The problem is that when you use the mean then the sum of all distances from the mean will always be zero. As a result, you must square them first. Divide by N so you have an average squared distance from the mean. It is actually N-1 for reasons we will not discuss. This is variance. We want to reconcile the units so that the number is meaningful. We squared everything, so we take the square root. This is the standard deviation. Intuitively, it is the “average” amount each point must travel to reach the mean. What is a standard deviation? 1-22

What is the standard error? In this case the “error” means how far, on average, we come from the true mean. Note! This is the standard error of the mean.

What is the standard error? This is the standard error of the slope.

CONFIDENCE INTERVALS

Recall that a distribution can reflect the observed characteristics of a population, but it can also reflect the sampling variation of a parameter.

How sure are we about the mean? If we were sure of ourselves we wouldn’t need a margin of error! Because we only have a sample, though, we can’t be certain. The population parameters are never known, so we use t-stats and the formula for the sample standard error. (CI of the mean)

How often are we wrong? μ Chose an alpha-level, which determines the size of the confidence interval. This example uses alpha=0.05. We would expect five samples in one-hundred to result in confidence intervals that do not contain the true mean. We see 3 in 50 draws here, which is consistent with expectations.

COVARIANCE

What is Covariance? (+,+) (+,-)(-,-) (-,+) Classroom size and test scores XYX-XbarY-Ybar(X-Xbar)(Y-Ybar) (-)5-3 (+)(-)(+) = (-) (-)2-3 (-)(-)(-) = (+) (+)2-3 (-)(+)(-) = (-)

Negative Covariance Example High negative correlation, large negative b 1 in a regression Class Size Test Scores Class size and student performance

Positive Covariance Example Hours of study and test scores High positive correlation, large positive b 1 in a regression

Small Covariance Low correlation, small b 1 in a regression

What is Correlation? The correlation is the covariance in simple units: -1 < cor(x,y) < +1

THE REGRESSION SLOPE

Intuitive Regression Formula Cov(x,y) Var(x) We want to interpret the slope causally, meaning the outcome is going to change b amount because of a one-unit change in X.

The Road Map Variance  St. Dev  St. Error  Confidence Intervals Mean: Slope: (of x) (of the residual) (of the slope) (of the mean)

Exercise 25

Which model is the “right” one? To answer this we need to understand both slopes and standard errors.

xy z xy z xy z xy xy

xy z Example #1

xy z Example #2

xy z Example #3

xy xy