Scientific Computing General Least Squares

Polynomial Least Squares Polynomial Least Squares: We assume that the class of functions is the class of all possible polynomials of degree less than or equal to n. We want to minimize

Matrix Formulation of Polynomial Least Squares To minimize Let Then, we want to find the vector c that minimizes the length squared of the error vector Ac-y (or y – Ac) That is, minimize

Matrix Formulation of Polynomial Least Squares By a similar calculation that we did for the Linear Least Squares we get the Normal Equation for A t Ac = A t y The solution c=[c 0 c 1... c n ] gives the constants for the best polynomial fit f * to the data:

Matrix Formulation of Polynomial Least Squares Definition: The matrix is called a Vandermonde Matrix. A Vandermonde matrix is a matrix whose columns (or rows) are successive powers of an independent variable.

Polynomial Least Squares Example Problem: Find the best quadratic polynomial fit to the data Normal Equation: A t Ac = A t y where x y

Polynomial Least Squares Example x y

Polynomial Least Squares Example A t y =

Polynomial Least Squares Example So, A t Ac = A t y becomes We could use Gaussian Elimination or use Matlab: D = [ ; ; ]; y = [ ]‘; c = D \ y c =

Polynomial Least Squares Example So, the best degree fit to the data is the polynomial x x 2.

Class Project Write a Matlab function that will take a vector of x values and a vector of y-values and will return the vector of coefficients for the best quadratic fit to the data.

Class Project 2 Exercise 9.15 in Pav: Write a Matlab function that will find the coefficients (a,b) for the function a e x + b e -x that best approximates a set of data {x i, y i }. Your function should have as input the x and y vectors and should output the vector c=(a,b). Use the Normal equation to solve this problem. Test your method on the data x = ( ), y = (1.194, 0.43, 0.103, 0.322, 1.034) Graph the data and your best fit function.

QR and Least Squares Given the normal equation A t Ac = A t y it is often the case that solving this directly can be unstable numerically. For example, consider the matrix Then the matrix becomes singular if eps is less than the square root of the machine epsilon.

QR and Least Squares To resolve this problem, assume we have carried out a QR factorization of A (this can be done for any matrix – square or rectangular). Then, A t Ac = A t y -> Note: where R n is an nxn matrix. Claim: To solve the least squares problem, it is enough to find the solution c to: R n c = Q t y

QR and Least Squares Proof: The least squares solution is to find the minimum value of For we have QED Note: Once R n and Q are determined, it is relatively straight- forward to solve R n c = Q t y for c. (Back-substitution)