1. Simple Regression
Mary M. Whiteside, PhD

2. Overview Model Data Least Squares Criterion
Interpreting b0, b1, and R2
Partitioning of the Sum of Squares
LINE Assumptions
Inferences
Diagnostics
Caveats

3. Model Y = b0 + b1X + e m(Y|x) = b0 + b1X
where Y and e are random variables
X is a variable with fixed values
b0, b1 are unknown parameters

4. Data n independent observations of Y for fixed values of X
bi-variate pairs (xi,yi) i = 1, 2, …, n.
Data comprise a random sample from a finite population or independent observations of a random variable
Example: (female literacy, infant mortality rate)

5. Least Squares Criterion
Minimize sum of squared error from the point (y) to the line (yhat)
Unique analytical solution obtained by differentiating the sum of squared error with respect to b0 and b1

6. Interpreting b0, b1 and R2
bo is only interpretable if zero, (0) is in the range of the X data. Then, it is the estimated expected value of Y when x = 0.
b1 is the estimated change in the expected value of Y when X increases by 1 (change scale for context)
R2 is the reduction in the squared error of Y associated with the linear relationship with X

7. Partitioning of the total sum of squares TSS of Y
(mean adjusted)TSS = MSS + SSE
Total (mean adjusted) sum of squares = model sum of squares + sum of squared error
S[y - ȳ]2 = S[ŷ – ȳ]2 + S[y – ŷ]2
R2 = 1 – (SSE/TSS)

8. LINE Assumptions
Linearity - check scatterplot of data or residuals vs. X
Independence - check scatterplots of residuals vs. Yhat, X
Normality – histograms of residuals
Equal Variance - scatterplots of residuals by Yhat, X

9. Confidence interval b0, b1 (2,3) Prediction interval Ynext (4)
Inferences
Tests of H0: b1=0 (r=0) (1)
Confidence interval b0, b1 (2,3)
Prediction interval Ynext (4)
Confidence interval mY|x (5)

10. Caveats
Outliers
Extrapolations
Outliers