1. REGRESSION/LS FORMULAS
Simplest case.
S() =  (Yi - )2
dS/d = -2(Yi - )
Normal equation. A  1 =  Yi
A = Y-bar
S(A) =  (Yi - A)2 =  Ei2
Anova identity.
 Yi2 =  (Y-bar + Yi - Y-bar)2
=  (Y-bar2) + (Yi - Y-bar)2 + (Y-bar)(Yi - Y-bar)
= nY-bar2 + (Yi - Y-bar)2

2. Simple regression.
(,) = (Yi -  -  Xi)2
S/  = (-2)(Yi -  -  Xi)
S/  = (-2)(Yi -  -  Xi)Xi
Normal equations.
An + B  Xi =  Yi (1)
A Xi + B  Xi2 =  XiYi (2)
A = Y-bar - B X-bar
B =  (Yi - Y-bar)(Xi - X-bar)/(Xi - X-bar)2
Residuals. Ei = Yi - A - BXi
From (1)  Ei = 0
From (2) Xi(Yi - A - BXi) = 0 =  Xi Ei

3. Fitted values. Yi-hat = A + B Xi
 Ei Yi-hat = 0
Anova identity.
(Yi - Y-bar)2 = (Yi-hat - Y-bar)2 + (Yi - Yi-hat)2
TSS = RegSS + RSS
Yi - Y-bar = Yi-hat - Y-bar + Yi - Yi-hat
(Yi-hat - Y-bar)(Yi - Yi-hat) = (Yi-hat - Y-bar)Ei = 0
S(A,B) = Ei2