Curve Fitting Filling in the gaps

Copyright G. A. Tagliarini, PhD Basic Problem Given sample points x1 < x2 < …< xn and sample data values y1, y2,…, yn corresponding to the xi, for 1 ≤ i ≤ n Find a function f such that y1 = f(x1), y2 = f(x2), …, yn = f(xn) Typically an approximate match is sought Additional design constraints, such as continuity or differentiability may apply 5/14/2019 Copyright G. A. Tagliarini, PhD

Least Squares Approximation 5/14/2019 Copyright G. A. Tagliarini, PhD

Copyright G. A. Tagliarini, PhD Estimating Values Interpolation is estimating the value of the data generating function for x ε [x1, xn] and x ≠ xi. Extrapolation is estimating the value of the data generating function for x outside the interval [x1, xn]. 5/14/2019 Copyright G. A. Tagliarini, PhD

Recall from Analytic Geometry (x2, y2) (x1, y1) (x2, y1) 5/14/2019 Copyright G. A. Tagliarini, PhD

Piece-wise Linear Interpolation (x2, y2) (xn, yn) (x1, y1) (x3, y3) 5/14/2019 Copyright G. A. Tagliarini, PhD

Polynomial Interpolation A polynomial p(x) is a function of the form p(x) = anxn + an-1xn-1+…+a1x+a0 If n is the largest power of x for which an ≠ 0, then p(x) is of degree n The n+1 coefficients ai are the undetermined values Assume n+1 samples to find a polynomial of degree n that passes through the sample points…Does one exist? How to find it? 5/14/2019 Copyright G. A. Tagliarini, PhD

Polynomial Approximation Polynomials are universal approximators for continuous functions on closed intervals Weierstrass Approximation Theorem Given F(x) continuous on [a, b] and any e>0 no matter how small There exists a polynomial f(x) such that |F(x)-f(x)|<e for all x in [a, b] This is an existence theorem. 5/14/2019 Copyright G. A. Tagliarini, PhD

Polynomial Interpolation Process 5/14/2019 Copyright G. A. Tagliarini, PhD

Re-writing as a Matrix Equation 5/14/2019 Copyright G. A. Tagliarini, PhD

Conditions for a Solution Given the linear system y = X a from the previous slide X-1 exists so that we can find a = X-1 y if the xi are all distinct This condition was met by our initial assumptions For many curve fitting problems (e.g., regression) this assumption is unmet X rapidly becomes ill-conditioned as n increases 5/14/2019 Copyright G. A. Tagliarini, PhD

Example: Polynomial Fit Find a cubic passing through (-1, -11), (0, -5), (1, -5), and (2, 1) Graph System 5/14/2019 Copyright G. A. Tagliarini, PhD

Example: Polynomial Fit (continued) 5/14/2019 Copyright G. A. Tagliarini, PhD

Example: Polynomial Fit (continued) 5/14/2019 Copyright G. A. Tagliarini, PhD

Copyright G. A. Tagliarini, PhD Lagrange Polynomials Expansion using the natural basis 1, x, x2, …, xn,… may give rise to numerical difficulties from searching for the inverse of an ill-conditioned matrix An alternative basis may provide a better representation 5/14/2019 Copyright G. A. Tagliarini, PhD

Lagrange Polynomial Definition 5/14/2019 Copyright G. A. Tagliarini, PhD

Example Lagrange Polynomials 5/14/2019 Copyright G. A. Tagliarini, PhD

Example Lagrange Polynomials 5/14/2019 Copyright G. A. Tagliarini, PhD

An Orthogonality Property Qk(xj) = 1 if k=j Qk(xj) = 0 if k≠j 5/14/2019 Copyright G. A. Tagliarini, PhD

The Approximation Formula 5/14/2019 Copyright G. A. Tagliarini, PhD

Plotting the Lagrange Polynomials and Their Sum 5/14/2019 Copyright G. A. Tagliarini, PhD

Lagrange Example Finished For the data points (-1, -11), (0, -5), (1, -5), and (2, 1) The polynomial f is given by f(x) = -11Q1(x)+ -5Q2(x) + -5Q3(x) + 1Q4(x) = 2x3 – 3x2 + 1x – 5 which by now should look familiar! (Can you show this?) 5/14/2019 Copyright G. A. Tagliarini, PhD

Copyright G. A. Tagliarini, PhD Linear Correlation Data may be represented in a scattergram Positive correlation results when the dependent variable’s (y) values increase as the independent variable’s (x) values increase. Negative correlation results when the dependent variable’s (y) values decrease as the independent variable’s (x) values increase. 5/14/2019 Copyright G. A. Tagliarini, PhD

Copyright G. A. Tagliarini, PhD Sample Scattergrams 5/14/2019 Copyright G. A. Tagliarini, PhD

Copyright G. A. Tagliarini, PhD Notation X and Y represent the distributions of independent and dependent variable values Critical values of r depend upon the significance level a of the test (typically, a=0.05 or a=0.01), the type of test (one tail or two tail), and the number of data pairs n 5/14/2019 Copyright G. A. Tagliarini, PhD

Pearson’s r: The Coefficient of Correlation 5/14/2019 Copyright G. A. Tagliarini, PhD

Copyright G. A. Tagliarini, PhD Prediction Equation 5/14/2019 Copyright G. A. Tagliarini, PhD

Copyright G. A. Tagliarini, PhD Cubic Splines 5/14/2019 Copyright G. A. Tagliarini, PhD