1. Discrete Least Squares Approximation
Sec:8.1
Discrete Least Squares Approximation

2. Sec:8.1 Discrete Least Squares Approximation
Find a relationship between x and y.
From this graph, it appears that the actual relationship between x and y is linear.
From this graph, it appears that the actual relationship between x and y is linear.
The likely reason that no line precisely fits the data is because of errors in the data.
The likely reason that no line precisely fits the data is because of errors in the data.
𝒑 𝟐 (𝒙) = 𝒂 𝟎 + 𝒂 𝟏 𝒙+ 𝒂 𝟐 𝒙 𝟐
𝒑 𝟏 (𝒙)= 𝒂 𝟎 + 𝒂 𝟏 𝒙

3. Sec:8.1 Discrete Least Squares Approximation
𝒀=𝑿 ∗𝒂
Fit the data in Table with the discrete least squares polynomial of degree at most 2.
𝑿 𝑻 ∗𝒀= 𝑿 𝑻 ∗ 𝑿 ∗𝒂
This is a linear system of 3 equations in 3 unknowns
𝒁 = 𝑩 ∗𝒂
𝟑×𝟑
3×𝟏
𝟑×𝟏
𝒑 𝟐 (𝒙) = 𝒂 𝟎 + 𝒂 𝟏 𝒙+ 𝒂 𝟐 𝒙 𝟐
𝒂 𝟎 𝒂 𝟏 𝒂 𝟐
𝟏.𝟎 = 𝒂 𝟎 + 𝟎∙𝒂 𝟏 + 𝟎 𝟐 ∙𝒂 𝟐
𝑋 𝑇
1.284 = 𝒂 𝟎 + 𝟎.𝟐𝟓𝒂 𝟏 + 𝟎.𝟐𝟓 𝟐 𝒂 𝟐 =
= 𝒂 𝟎 + 𝟎.𝟓𝒂 𝟏 + 𝟎.𝟓 𝟐 𝒂 𝟐
2.117 = 𝒂 𝟎 + 𝟎.𝟕𝟓𝒂 𝟏 + 𝟎.𝟕𝟓 𝟐 𝒂 𝟐
8.7680
𝒂 𝟎 𝒂 𝟏 𝒂 𝟐
= 𝒂 𝟎 + 𝟏𝒂 𝟏 + 𝟏 𝟐 𝒂 𝟐
normal equations =
Matrix Form
𝒁 = 𝑩 ∗ 𝒂
= 𝒂 𝟎 𝒂 𝟏 𝒂 𝟐
𝒂 𝟎 𝒂 𝟏 𝒂 𝟐
0.8418
0.1106
0.0527 = 𝒂 𝒀 𝑿
𝒑 𝟐 (𝒙) = 𝒙 𝒙 𝟐
𝟓×𝟏
𝟓×𝟑
𝟑×𝟏

4. Sec:8.1 Discrete Least Squares Approximation
Fit the data in Table with the discrete least squares polynomial of degree at most 2.
𝒑 𝟐 (𝒙) = 𝒙 𝒙 𝟐 𝒙 𝒚
𝒑 𝟐
𝑬𝒓𝒓𝒐𝒓

5. Sec:8.1 Discrete Least Squares Approximation
EXAMPLE:
Use least-squares regression to fit a straight line to
x [ ]
y [ ]
Plot the data and the regression line.
𝒑 𝟏 (𝒙)= 𝒙
clear
x = [ ]';
Y = [ ]';
n = length(x)
X = [ones(n,1) x ]; % Y = X * a
% B = X'*X; Z = X' * Y; a = B\Z
a = X\Y
xx=[0:0.1:19]';
yy=a(1)+a(2)*xx;
plot(x,Y,'xr',xx,yy,'-b'); grid on
𝒆 𝒊 = 𝒚 𝒊 −( 𝒂 𝟎 + 𝒂 𝟏 𝒙 𝒊 )

6. Sec:8.1 Discrete Least Squares Approximation
Derivation for the formula:
EXAMPLE:
One strategy for fitting a “best” line through the data would be to minimize the sum of the absolute value of residual errors for all the available data
Use least-squares regression to fit a straight line to
x [ ]
y [ ]
𝒊=𝟏 𝟏𝟎 𝒆 𝒊 = 𝒊=𝟏 𝟏𝟎 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊
𝒆 𝒊 = 𝒚 𝒊 −( 𝒂 𝟎 + 𝒂 𝟏 𝒙 𝒊 )
find values of 𝒂 𝟎 and 𝒂 𝟏
min 𝒂 𝟎 , 𝒂 𝟏 𝒊=𝟏 𝟏𝟎 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊
𝒑 𝟏 (𝒙)= 𝒙
To minimize a function of two variables, we need to set its partial derivatives to zero and simultaneously solve the resulting equations.
𝒆 𝟏 = 𝒚 𝟏 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝟏
𝒆 𝟐 = 𝒚 𝟐 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝟐
⋮ ⋮
The problem is that the absolute-value function is not differentiable at zero, and we might not be able to find solutions to this pair of equations.
𝒆 𝟏𝟎 = 𝒚 𝟏𝟎 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝟏𝟎

7. Sec:8.1 Discrete Least Squares Approximation
EXAMPLE:
Least Squares
Use least-squares regression to fit a straight line to
The least squares approach to this problem involves determining the best approximating line when the error involved is the sum of the squares
x [ ]
y [ ]
𝒆 𝒊 = 𝒚 𝒊 −( 𝒂 𝟎 + 𝒂 𝟏 𝒙 𝒊 )
min 𝒂 𝟎 , 𝒂 𝟏 𝒊=𝟏 𝟏𝟎 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 𝟐
The least squares method is the most convenient procedure for determining best linear approximations
𝒑 𝟏 (𝒙)= 𝒙
𝑬 𝒂 𝟎 , 𝒂 𝟏 = 𝒊=𝟏 𝟏𝟎 𝒆 𝟐
For a minimum to occur, we need both
= 𝒊=𝟏 𝟏𝟎 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 𝟐
𝝏𝑬 𝝏 𝒂 𝟎 =𝟎
𝝏𝑬 𝝏 𝒂 𝟏 =𝟎

8. Sec:8.1 Discrete Least Squares Approximation
𝒊=𝟏 𝒎 −𝟐 𝒙 𝒊 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 =𝟎
𝝏𝑬 𝝏 𝒂 𝟏 =𝟎
= 𝒊=𝟏 𝒎 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 𝟐
𝑬 𝒂 𝟎 , 𝒂 𝟏
− 𝒊=𝟏 𝒎 𝒙 𝒊 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 =𝟎
For a minimum to occur, we need both
𝝏𝑬 𝝏 𝒂 𝟎 =𝟎
𝝏𝑬 𝝏 𝒂 𝟏 =𝟎
− 𝒊=𝟏 𝒎 𝒙 𝒊 𝒚 𝒊 + 𝒊=𝟏 𝒎 𝒂 𝟎 𝒙 𝒊 + 𝒊=𝟏 𝒎 𝒂 𝟏 𝒙 𝒊 𝟐 =𝟎
𝒊=𝟏 𝒎 −𝟐 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 =𝟎
𝝏𝑬 𝝏 𝒂 𝟎 =𝟎
𝒊=𝟏 𝒎 𝒙 𝒊 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒂 𝟏 = 𝒊=𝟏 𝒎 𝒙 𝒊 𝒚 𝒊
− 𝒊=𝟏 𝒎 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 =𝟎
+ 𝒊=𝟏 𝒎 𝒙 𝒊 𝒂 𝟏 = 𝒊=𝟏 𝒎 𝒚 𝒊
𝒎 𝒂 𝟎
− 𝒊=𝟏 𝒎 𝒚 𝒊 + 𝒊=𝟏 𝒎 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒂 𝟏 𝒙 𝒊 =𝟎
𝒊=𝟏 𝒎 𝒙 𝒊 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒂 𝟏 = 𝒊=𝟏 𝒎 𝒙 𝒊 𝒚 𝒊
+ 𝒊=𝟏 𝒎 𝒙 𝒊 𝒂 𝟏 = 𝒊=𝟏 𝒎 𝒚 𝒊
𝒎 𝒂 𝟎
These are called the normal equations.

9. Sec:8.1 Discrete Least Squares Approximation
+ 𝒊=𝟏 𝒎 𝒙 𝒊 𝒂 𝟏 = 𝒊=𝟏 𝒎 𝒚 𝒊
𝑦 1 ⋮ 𝑦 𝑚 = 1 𝑥 1 ⋮ ⋮ 1 𝑥 𝑚 𝒂 𝟎 𝒂 𝟏
𝒎 𝒂 𝟎
Least Squares
Use least-squares regression to fit a straight line to
𝒊=𝟏 𝒎 𝒙 𝒊 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒂 𝟏 = 𝒊=𝟏 𝒎 𝒙 𝒊 𝒚 𝒊 𝒀 𝑿 𝒂
𝑥 1 𝑦 1 𝑥 2 𝑦 2 ⋮ ⋮ 𝑥 𝑚 𝑦 𝑚 𝑥 𝑦
𝒎×𝟏
𝒎×𝟐
𝟐×𝟏
These are called the normal equations.
𝒀=𝑿 ∗𝒂
𝑿 𝑻 ∗𝒀= 𝑿 𝑻 ∗ 𝑿 ∗𝒂
Matrix Form
𝒑 𝟏 (𝒙)= 𝒂 𝟎 + 𝒂 𝟏 𝒙
𝑚 𝑥 𝑖 𝑥 𝑖 𝑥 𝑖 𝒂 𝟎 𝒂 𝟏 = 𝑦 𝑖 𝑥 𝑖 𝑦 𝑖
1 ⋯ 1 𝑥 1 ⋯ 𝑥 𝑚 𝑦 1 ⋮ 𝑦 𝑚 = 1 ⋯ 1 𝑥 1 ⋯ 𝑥 𝑚 𝑥 1 ⋮ ⋮ 1 𝑥 𝑚 𝒂 𝟎 𝒂 𝟏
𝑦 1 +⋯+ 𝑦 𝑚 𝑥 1 𝑦 1 +⋯+ 𝑥 𝑚 𝑦 𝑚 = 1+⋯+1 𝑥 1 +⋯+ 𝑥 𝑚 𝑥 1 +⋯+ 𝑥 𝑚 𝑥 1 2 +⋯+ 𝑥 𝑚 𝒂 𝟎 𝒂 𝟏

10. Derivation for the formula: (polynomial of degree 2)
Sec:8.1 Discrete Least Squares Approximation
Derivation for the formula:
(polynomial of degree 2)

11. Derivation for the formula: (polynomial of degree 2)
Sec:8.1 Discrete Least Squares Approximation
Least Squares
= 𝒊=𝟏 𝒎 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 − 𝒂 𝟐 𝒙 𝒊 𝟐 𝟐
𝑥 1 𝑦 1 𝑥 2 𝑦 2 ⋮ ⋮ 𝑥 𝑚 𝑦 𝑚 𝑥 𝑦
𝑬 𝒂 𝟎 , 𝒂 𝟏
Use least-squares regression to fit a polynomial of degree 2 to
For a minimum to occur, we need both
𝒑 𝟏 𝒙 = 𝒂 𝟎 + 𝒂 𝟏 𝒙+ 𝒂 𝟐 𝒙 𝟐
𝝏𝑬 𝝏 𝒂 𝟎 =𝟎
𝝏𝑬 𝝏 𝒂 𝟏 =𝟎
𝝏𝑬 𝝏 𝒂 𝟐 =𝟎
𝒊=𝟏 𝒎 −𝟐 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 − 𝒂 𝟐 𝒙 𝒊 𝟐 =𝟎
𝝏𝑬 𝝏 𝒂 𝟎 =𝟎
Derivation for the formula:
(polynomial of degree 2)
− 𝒊=𝟏 𝒎 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 − 𝒂 𝟐 𝒙 𝒊 𝟐 =𝟎
− 𝒊=𝟏 𝒎 𝒚 𝒊 + 𝒊=𝟏 𝒎 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒂 𝟏 𝒙 𝒊 + 𝒊=𝟏 𝒎 𝒂 𝟐 𝒙 𝒊 𝟐 =𝟎
+ 𝒊=𝟏 𝒎 𝒙 𝒊 𝒂 𝟏 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒂 𝟐 = 𝒊=𝟏 𝒎 𝒚 𝒊
𝒎 𝒂 𝟎

12. Sec:8.1 Discrete Least Squares Approximation
= 𝒊=𝟏 𝒎 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 − 𝒂 𝟐 𝒙 𝒊 𝟐 𝟐
𝑥 1 𝑦 1 𝑥 2 𝑦 2 ⋮ ⋮ 𝑥 𝑚 𝑦 𝑚 𝑥 𝑦
𝑬 𝒂 𝟎 , 𝒂 𝟏
Use least-squares regression to fit a polynomial of degree 2 to
For a minimum to occur, we need both
𝒑 𝟏 𝒙 = 𝒂 𝟎 + 𝒂 𝟏 𝒙+ 𝒂 𝟐 𝒙 𝟐
𝝏𝑬 𝝏 𝒂 𝟎 =𝟎
𝝏𝑬 𝝏 𝒂 𝟏 =𝟎
𝝏𝑬 𝝏 𝒂 𝟐 =𝟎
𝒊=𝟏 𝒎 −𝟐 𝒙 𝒊 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 − 𝒂 𝟐 𝒙 𝒊 𝟐 =𝟎
𝝏𝑬 𝝏 𝒂 𝟏 =𝟎
− 𝒊=𝟏 𝒎 𝒙 𝒊 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 − 𝒂 𝟐 𝒙 𝒊 𝟐 =𝟎
− 𝒊=𝟏 𝒎 𝒙 𝒊 𝒚 𝒊 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒂 𝟏 𝒙 𝒊 𝟐 + 𝒊=𝟏 𝒎 𝒂 𝟐 𝒙 𝒊 𝟑 =𝟎
𝒊=𝟏 𝒎 𝒙 𝒊 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒂 𝟏 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟑 𝒂 𝟐 = 𝒊=𝟏 𝒎 𝒙 𝒊 𝒚 𝒊

13. Sec:8.1 Discrete Least Squares Approximation
= 𝒊=𝟏 𝒎 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 − 𝒂 𝟐 𝒙 𝒊 𝟐 𝟐
𝑥 1 𝑦 1 𝑥 2 𝑦 2 ⋮ ⋮ 𝑥 𝑚 𝑦 𝑚 𝑥 𝑦
𝑬 𝒂 𝟎 , 𝒂 𝟏
Use least-squares regression to fit a polynomial of degree 2 to
For a minimum to occur, we need both
𝒑 𝟏 𝒙 = 𝒂 𝟎 + 𝒂 𝟏 𝒙+ 𝒂 𝟐 𝒙 𝟐
𝝏𝑬 𝝏 𝒂 𝟎 =𝟎
𝝏𝑬 𝝏 𝒂 𝟏 =𝟎
𝝏𝑬 𝝏 𝒂 𝟐 =𝟎
𝒊=𝟏 𝒎 −𝟐 𝒙 𝒊 𝟐 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 − 𝒂 𝟐 𝒙 𝒊 𝟐 =𝟎
𝝏𝑬 𝝏 𝒂 𝟐 =𝟎
− 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 − 𝒂 𝟐 𝒙 𝒊 𝟐 =𝟎
− 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒚 𝒊 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒂 𝟏 𝒙 𝒊 𝟑 + 𝒊=𝟏 𝒎 𝒂 𝟐 𝒙 𝒊 𝟒 =𝟎
𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟑 𝒂 𝟏 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟒 𝒂 𝟐 = 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒚 𝒊

14. Sec:8.1 Discrete Least Squares Approximation
= 𝒊=𝟏 𝒎 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 − 𝒂 𝟐 𝒙 𝒊 𝟐 𝟐
𝑥 1 𝑦 1 𝑥 2 𝑦 2 ⋮ ⋮ 𝑥 𝑚 𝑦 𝑚 𝑥 𝑦
𝑬 𝒂 𝟎 , 𝒂 𝟏
Use least-squares regression to fit a polynomial of degree 2 to
For a minimum to occur, we need both
𝒑 𝟐 𝒙 = 𝒂 𝟎 + 𝒂 𝟏 𝒙+ 𝒂 𝟐 𝒙 𝟐
𝝏𝑬 𝝏 𝒂 𝟎 =𝟎
𝝏𝑬 𝝏 𝒂 𝟏 =𝟎
𝝏𝑬 𝝏 𝒂 𝟐 =𝟎
normal equations
normal equations in Matrix Form
+ 𝒊=𝟏 𝒎 𝒙 𝒊 𝒂 𝟏 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒂 𝟐 = 𝒊=𝟏 𝒎 𝒚 𝒊
𝒎 𝒂 𝟎
𝒊=𝟏 𝒎 𝒙 𝒊 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒂 𝟏 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟑 𝒂 𝟐 = 𝒊=𝟏 𝒎 𝒙 𝒊 𝒚 𝒊
𝑛 𝒙 𝒊 𝑥 𝑖 𝒙 𝒊 𝑥 𝑖 𝑥 𝑖 𝑥 𝑖 𝑥 𝑖 𝑥 𝑖 𝒂 𝟎 𝒂 𝟏 𝒂 𝟐 = 𝒚 𝒊 𝒙 𝒊 𝒚 𝒊 𝑥 𝑖 2 𝒚 𝒊
𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟑 𝒂 𝟏 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟒 𝒂 𝟐 = 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒚 𝒊

15. Sec:8.1 Discrete Least Squares Approximation
𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟑 𝒂 𝟏 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟒 𝒂 𝟐 = 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒚 𝒊
𝒊=𝟏 𝒎 𝒙 𝒊 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒂 𝟏 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟑 𝒂 𝟐 = 𝒊=𝟏 𝒎 𝒙 𝒊 𝒚 𝒊
𝒎 𝒂 𝟎 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝒂 𝟏 + 𝒊=𝟏 𝒎 𝒙 𝒊 𝟐 𝒂 𝟐 = 𝒊=𝟏 𝒎 𝒚 𝒊
𝑦 1 ⋮ 𝑦 𝑚 = 1 𝑥 1 𝑥 1 2 ⋮ ⋮ ⋮ 1 𝑥 𝑚 𝑥 𝑚 𝒂 𝟎 𝒂 𝟏 𝒂 𝟐
Least Squares
Use least-squares regression to fit a poly of deg 2 to 𝒀
𝑥 1 𝑦 1 𝑥 2 𝑦 2 ⋮ ⋮ 𝑥 𝑚 𝑦 𝑚 𝑥 𝑦 𝑿 𝒂
𝒎×𝟏
𝒎×𝟑
𝟑×𝟏
𝒀=𝑿 ∗𝒂
normal equations.
𝑿 𝑻 ∗𝒀= 𝑿 𝑻 ∗ 𝑿 ∗𝒂
Matrix Form
𝒑 𝟐 𝒙 = 𝒂 𝟎 + 𝒂 𝟏 𝒙+ 𝒂 𝟐 𝒙 𝟐
𝑚 𝒙 𝒊 𝑥 𝑖 𝒙 𝒊 𝑥 𝑖 𝑥 𝑖 𝑥 𝑖 𝑥 𝑖 𝑥 𝑖 𝒂 𝟎 𝒂 𝟏 𝒂 𝟐 = 𝒚 𝒊 𝒙 𝒊 𝒚 𝒊 𝑥 𝑖 2 𝒚 𝒊
1 ⋯ 1 𝑥 1 ⋯ 𝑥 𝑚 𝑥 1 2 ⋯ 𝑥 𝑚 𝑦 1 ⋮ 𝑦 𝑚 = 1 ⋯ 1 𝑥 1 ⋯ 𝑥 𝑚 𝑥 1 2 ⋯ 𝑥 𝑚 𝑥 1 𝑥 1 2 ⋮ ⋮ ⋮ 1 𝑥 𝑚 𝑥 𝑚 𝒂 𝟎 𝒂 𝟏 𝒂 𝟐
𝑦 1 +⋯+ 𝑦 𝑚 𝑥 1 𝑦 1 +⋯+ 𝑥 𝑚 𝑦 𝑚 𝑥 1 2 𝑦 1 +⋯+ 𝑥 𝑚 2 𝑦 𝑚 = 1+⋯+1 𝑥 1 +⋯+ 𝑥 𝑚 𝑥 1 2 +⋯+ 𝑥 𝑚 2 𝑥 1 +⋯+ 𝑥 𝑚 𝑥 1 2 +⋯+ 𝑥 𝑚 2 𝑥 1 3 +⋯+ 𝑥 𝑚 3 𝑥 1 2 +⋯+ 𝑥 𝑚 2 𝑥 1 3 +⋯+ 𝑥 𝑚 3 𝑥 1 4 +⋯+ 𝑥 𝑚 𝒂 𝟎 𝒂 𝟏 𝒂 𝟐

16. 𝒎 𝒏 Sec:8.1 Discrete Least Squares Approximation
= 𝒊=𝟏 𝒎 𝒚 𝒊 − 𝒂 𝟎 − 𝒂 𝟏 𝒙 𝒊 −⋯− 𝒂 𝒏 𝒙 𝒊 𝒏 𝟐
𝑥 1 𝑦 1 𝑥 2 𝑦 2 ⋮ ⋮ 𝑥 𝑚 𝑦 𝑚 𝑥 𝑦
𝑬 𝒂 𝟎 , 𝒂 𝟏
Use least-squares regression to fit a polynomial of degree n to
For a minimum to occur, we need both
𝝏𝑬 𝝏 𝒂 𝟎 =𝟎
𝝏𝑬 𝝏 𝒂 𝒏 =𝟎
𝝏𝑬 𝝏 𝒂 𝟏 =𝟎 ⋯⋯
𝒑 𝟐 𝒙 = 𝒂 𝟎 + 𝒂 𝟏 𝒙+⋯+ 𝒂 𝒏 𝒙 𝒏
normal equations in Matrix Form
𝑚 𝒙 𝒊 ⋯ 𝑥 𝑖 𝑛 𝒙 𝒊 𝑥 𝑖 ⋯ 𝑥 𝑖 𝑛 ⋮ ⋮ ⋱ ⋮ 𝑥 𝑖 𝑛 𝑥 𝑖 𝑛 ⋯ 𝑥 𝑖 2𝑛 𝒂 𝟎 𝒂 𝟏 ⋮ 𝒂 𝒏 = 𝒚 𝒊 𝒙 𝒊 𝒚 𝒊 ⋮ 𝑥 𝑖 𝑛 𝒚 𝒊
𝑥 𝑖 0 𝒎
Number of data points 𝒏
Degree of polynomial
(𝑛+1)×(𝑛+1)
(𝑛+1)×1

17. Sec:8.1 Discrete Least Squares Approximation