Environmental Data Analysis with MatLab 2 nd Edition Lecture 22: Linear Approximations and Non Linear Least Squares

Lecture 01Using MatLab Lecture 02Looking At Data Lecture 03Probability and Measurement Error Lecture 04Multivariate Distributions Lecture 05Linear Models Lecture 06The Principle of Least Squares Lecture 07Prior Information Lecture 08Solving Generalized Least Squares Problems Lecture 09Fourier Series Lecture 10Complex Fourier Series Lecture 11Lessons Learned from the Fourier Transform Lecture 12Power Spectra Lecture 13Filter Theory Lecture 14Applications of Filters Lecture 15Factor Analysis Lecture 16Orthogonal functions Lecture 17Covariance and Autocorrelation Lecture 18Cross-correlation Lecture 19Smoothing, Correlation and Spectra Lecture 20Coherence; Tapering and Spectral Analysis Lecture 21Interpolation Lecture 22Linear Approximations and Non Linear Least Squares Lecture 23Adaptable Approximations with Neural Networks Lecture 22 Hypothesis testing Lecture 23 Hypothesis Testing continued; F-Tests Lecture 24 Confidence Limits of Spectra, Bootstraps SYLLABUS

Goals of the lecture learn how to make linear approximations of non-linear functions apply liner approximations to error estimation apply liner approximations to least squares

Taylor Series and Linear Approximations

polynomial approximation to a function y(t) in the neighborhood of a point t 0

polynomial approximation to a function y(t) in the neighborhood of a point t 0 find coefficients by taking derivatives

polynomial approximation to a function y(t) in the neighborhood of a point t 0 find coefficients by taking deriatives evaluate at t

polynomial approximation to a function y(t) in the neighborhood of a point t 0

polynomial approximation to a function y(t) in the neighborhood of a point t 0 Taylor series

Taylor Series Linear approximation ≈

example

Linear approximation

r (λ 1,L 1 ) (λ 2,L 2 ) example: distances on a sphere measured in terms of central angle, r

exact formula: 6 trig functions approximate formula: 1 trig function and 1 square root

(λ 2,L 2 =0) (λ 1 =0,L 1 =0) linear quadratic

application to estimates of variance

spectral analysis scenario measure angular frequency, m want confidence bounds on corresponding period, T

exact (but difficult) method assume m is Normally-distributed, p(m) work out the distribution p(T) compute its mean and variance by integration

approximate (and easy) method assume m is Normally-distributed with mean m est work out a linear approximation of T in neighborhood of m est use formula for error propagation for a linear functions

consider small fluctuations about the estimated angular frequency so T est

application to least squares

Goal Solve non-linear problems of the form by generalized least squares

Taylor series of predicted data

Taylor expansion of predicted data with and

Taylor expansion of predicted data with and linearized equation

Taylor expansion of total error

gradient vector curvature matrix

linearized least squares

minimize error

linearized least squares minimize error

linearized least squares minimize error linear theory

linearized least squares minimize error linear theory

linearized least squares minimize error linear theory

linearized least squares minimize error linear theory

linearized least squares guess for the solution

linearized least squares trial solution deviation of data from prediction of trial solution

linearized least squares trial solution deviation of data from prediction of trial solution linearized data kernel

linearized least squares trial solution deviation of data from prediction of trial solution linearized data kernel correction to solution

linearized least squares trial solution deviation of data from prediction of trial solution linearized data kernel correction to solution updated solution

linearized least squares repeat

prior information = written in terms of the unknown

modification of generalized least squares

example of generalized least squares

sinusoid of unknown amplitude & frequency superimposed on a constant background level

example of generalized least squares normalized unknowns, so m i ≈1 sinusoid of unknown amplitude & frequency superimposed on a constant background level amplitude background level frequency

compute derivatives & evaluate in neighborhood of a guess m ω a