Environmental Data Analysis with MatLab

Slides:



Advertisements
Similar presentations
You have been given a mission and a code. Use the code to complete the mission and you will save the world from obliteration…
Advertisements

Substitution.
Feichter_DPG-SYKL03_Bild-01. Feichter_DPG-SYKL03_Bild-02.
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. *See PowerPoint Lecture Outline for a complete, ready-made.
1 Copyright © 2013 Elsevier Inc. All rights reserved. Appendix 01.
1 Copyright © 2013 Elsevier Inc. All rights reserved. Chapter 38.
Chapter 1 Image Slides Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Jeopardy Q 1 Q 6 Q 11 Q 16 Q 21 Q 2 Q 7 Q 12 Q 17 Q 22 Q 3 Q 8 Q 13
Jeopardy Q 1 Q 6 Q 11 Q 16 Q 21 Q 2 Q 7 Q 12 Q 17 Q 22 Q 3 Q 8 Q 13
Overview of Lecture Partitioning Evaluating the Null Hypothesis ANOVA
Lecture 2 ANALYSIS OF VARIANCE: AN INTRODUCTION
Assumptions underlying regression analysis
Modeling of Data. Basic Bayes theorem Bayes theorem relates the conditional probabilities of two events A, and B: A might be a hypothesis and B might.
The Maximum Likelihood Method
1 STA 536 – Experiments with a Single Factor Regression and ANOVA.
Chapter 4: Basic Estimation Techniques
Environmental Data Analysis with MatLab Lecture 10: Complex Fourier Series.
Lecture 32 APPLICATIONS OF BIVARIATE OPTIMIZATION.
15. Oktober Oktober Oktober 2012.
Solving Equations How to Solve Them
Squares and Square Root WALK. Solve each problem REVIEW:
We are learning how to read the 24 hour clock
The x- and y-Intercepts
Lecture 1 Describing Inverse Problems. Syllabus Lecture 01Describing Inverse Problems Lecture 02Probability and Measurement Error, Part 1 Lecture 03Probability.
25 seconds left…...
We will resume in: 25 Minutes.
Chapter Twelve Multiple Regression and Model Building McGraw-Hill/Irwin Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved.
Lecture 20 Continuous Problems Linear Operators and Their Adjoints.
Simple Linear Regression Analysis
Multiple Regression and Model Building
16. Mean Square Estimation
Murach’s OS/390 and z/OS JCLChapter 16, Slide 1 © 2002, Mike Murach & Associates, Inc.
Environmental Data Analysis with MatLab Lecture 21: Interpolation.
Lecture 10 Nonuniqueness and Localized Averages. Syllabus Lecture 01Describing Inverse Problems Lecture 02Probability and Measurement Error, Part 1 Lecture.
Environmental Data Analysis with MatLab Lecture 15: Factor Analysis.
Lecture 14 Nonlinear Problems Grid Search and Monte Carlo Methods.
Environmental Data Analysis with MatLab Lecture 8: Solving Generalized Least Squares Problems.
Lecture 13 L1 , L∞ Norm Problems and Linear Programming
Lecture 15 Orthogonal Functions Fourier Series. LGA mean daily temperature time series is there a global warming signal?
Environmental Data Analysis with MatLab Lecture 9: Fourier Series.
Environmental Data Analysis with MatLab
Environmental Data Analysis with MatLab Lecture 13: Filter Theory.
Environmental Data Analysis with MatLab Lecture 16: Orthogonal Functions.
Environmental Data Analysis with MatLab Lecture 23: Hypothesis Testing continued; F-Tests.
Environmental Data Analysis with MatLab Lecture 11: Lessons Learned from the Fourier Transform.
Environmental Data Analysis with MatLab
Environmental Data Analysis with MatLab Lecture 12: Power Spectral Density.
Lecture 2 Probability and Measurement Error, Part 1.
Environmental Data Analysis with MatLab Lecture 17: Covariance and Autocorrelation.
Lecture 4 The L 2 Norm and Simple Least Squares. Syllabus Lecture 01Describing Inverse Problems Lecture 02Probability and Measurement Error, Part 1 Lecture.
Lecture 9 Inexact Theories. Syllabus Lecture 01Describing Inverse Problems Lecture 02Probability and Measurement Error, Part 1 Lecture 03Probability and.
Lecture 6 Resolution and Generalized Inverses. Syllabus Lecture 01Describing Inverse Problems Lecture 02Probability and Measurement Error, Part 1 Lecture.
Environmental Data Analysis with MatLab Lecture 5: Linear Models.
Lecture 7 Advanced Topics in Least Squares. the multivariate normal distribution for data, d p(d) = (2  ) -N/2 |C d | -1/2 exp{ -1/2 (d-d) T C d -1 (d-d)
Lecture 5 Probability and Statistics. Please Read Doug Martinson’s Chapter 3: ‘Statistics’ Available on Courseworks.
Environmental Data Analysis with MatLab Lecture 3: Probability and Measurement Error.
Lecture 8 The Principle of Maximum Likelihood. Syllabus Lecture 01Describing Inverse Problems Lecture 02Probability and Measurement Error, Part 1 Lecture.
Environmental Data Analysis with MatLab Lecture 24: Confidence Limits of Spectra; Bootstraps.
Lecture 11 Vector Spaces and Singular Value Decomposition.
Lecture 3: Inferences using Least-Squares. Abstraction Vector of N random variables, x with joint probability density p(x) expectation x and covariance.
Environmental Data Analysis with MatLab Lecture 7: Prior Information.
Environmental Data Analysis with MatLab Lecture 20: Coherence; Tapering and Spectral Analysis.
Environmental Data Analysis with MatLab Lecture 10: Complex Fourier Series.
NON-LINEAR REGRESSION Introduction Section 0 Lecture 1 Slide 1 Lecture 6 Slide 1 INTRODUCTION TO Modern Physics PHYX 2710 Fall 2004 Intermediate 3870 Fall.
ESTIMATION METHODS We know how to calculate confidence intervals for estimates of  and  2 Now, we need procedures to calculate  and  2, themselves.
Environmental Data Analysis with MatLab 2 nd Edition Lecture 14: Applications of Filters.
Environmental Data Analysis with MatLab 2 nd Edition Lecture 22: Linear Approximations and Non Linear Least Squares.
Probability Theory and Parameter Estimation I
CH 5: Multivariate Methods
Environmental Data Analysis with MatLab
Presentation transcript:

Environmental Data Analysis with MatLab Lecture 6: The Principle of Least Squares

SYLLABUS Lecture 01 Using MatLab Lecture 02 Looking At Data Lecture 03 Probability and Measurement Error Lecture 04 Multivariate Distributions Lecture 05 Linear Models Lecture 06 The Principle of Least Squares Lecture 07 Prior Information Lecture 08 Solving Generalized Least Squares Problems Lecture 09 Fourier Series Lecture 10 Complex Fourier Series Lecture 11 Lessons Learned from the Fourier Transform Lecture 12 Power Spectra Lecture 13 Filter Theory Lecture 14 Applications of Filters Lecture 15 Factor Analysis Lecture 16 Orthogonal functions Lecture 17 Covariance and Autocorrelation Lecture 18 Cross-correlation Lecture 19 Smoothing, Correlation and Spectra Lecture 20 Coherence; Tapering and Spectral Analysis Lecture 21 Interpolation Lecture 22 Hypothesis testing Lecture 23 Hypothesis Testing continued; F-Tests Lecture 24 Confidence Limits of Spectra, Bootstraps

estimate model parameters using the principle of least-squares purpose of the lecture estimate model parameters using the principle of least-squares Least-squares is a standard way to solve a linear model.

the least squares estimation of model parameters and their covariance part 1 the least squares estimation of model parameters and their covariance Emphasize that calculating covariance is an integral part of the process. A answer is of no value without information about its accuracy. All estimates of model parameters based on noisy data are inherently uncertain. The key question is how uncertain.

motivates us to define an error vector, e the prediction error motivates us to define an error vector, e This slide and the next three are review.

prediction error in straight line case dipre ei data, d diobs Review the distinction between the observed and predicted data. Review how the error is calculated. auxiliary variable, x

total error single number summarizing the error sum of squares of individual errors

principle of least-squares that minimizes Remind students that the total error was defined in order to quantify what “about equal” means.

least-squares and probability suppose that each observation has a Normal p.d.f. 2

for uncorrelated data the joint p. d. f for uncorrelated data the joint p.d.f. is just the product of the individual p.d.f.’s least-squares formula for E suggests a link between probability and least-squares

now assume that Gm predicts the mean of d Gm substituted for d minimizing E(m) is equivalent to maximizing p(d)

the principle of least-squares determines the m that makes the observations “most probable” in the sense of maximizing p(dobs) Emphasize the strong link between two ways of thinking about goodness-of-fit, one involving error, the other involving probability.

the principle of least-squares determines the model parameters that makes the observations “most probable” (provided that the data are Normal) this is the principle of maximum likelihood

a formula for mest at the point of minimum error, E ∂E / ∂mi = 0 so solve this equation for mest

Result

where the result comes from so

unity when k=j zero when k≠j since m’s are independent use the chain rule so just delete sum over j and replace j with k unity when k=j zero when k≠j since m’s are independent

which gives

covariance of mest mest is a linear function of d of the form mest = M d so Cm = M Cd MT, with M=[GTG]-1GT assume Cd uncorrelated with uniform variance, σd2 then Show how many of the factors in the expression cancel one another.

two methods of estimating the variance of the data prior estimate: use knowledge of measurement technique the ruler has 1mm tic marks, so σd≈½mm posterior estimate: use prediction error

posterior estimates are overestimates when the model is poor reduce N by M since an M-parameter model can exactly fit N data

m=mest±2σmi (95% confidence) confidence intervals for the estimated model parameters (assuming uncorrelated data of equal variance) so σmi = √[Cm]ii and m=mest±2σmi (95% confidence)

MatLab script for least squares solution mest = (G’*G)\(G’*d); Cm = sd2 * inv(G’*G); sm = sqrt(diag(Cm)); Emphasize that this solution works regardless of the details of G. Explain use of backslash operator. Explain that the diag() function extracts the main diagonal from a matrix and puts it in a vector.

exemplary least squares problems part 2 exemplary least squares problems

Example 1: the mean of data the constant will turn out to be the mean

usual formula for the mean variance decreases with number of data usual formula for the mean

2σd ± m1est = d = √N formula for mean formula for covariance combining the two into confidence limits m1est = d = 2σd ± √N (95% confidence)

Example 2: fitting a straight line intercept slope

[GTG]-1= (uses the rule)

intercept and slope are uncorrelated when the mean of x is zero

keep in mind that none of this algrbraic manipulation is needed if we just compute using MatLab

Generic MatLab script for least-squares problems mest = (G’*G)\(G’*dobs); dpre = G*mest; e = dobs-dpre; E = e’*e; sigmad2 = E / (N-M); covm = sigmad2 * inv(G’*G); sigmam = sqrt(diag(covm)); mlow95 = mest – 2*sigmam; mhigh95 = mest + 2*sigmam;

Example 3: modeling long-term trend and annual cycle in Black Rock Forest temperature data d(t)obs d(t)pre error, e(t) time t, days Note that this plot has the observations, dobs, the “results”, dpre, and the error, e. The error large in the sense that it is an appreciable fraction of the amplitude of the data.

the model: long-term trend annual cycle Identify m2 as the long-term slope. Note that the cosine and sine are paired.

MatLab script to create the data kernel Ty=365.25; G=zeros(N,4); G(:,1)=1; G(:,2)=t; G(:,3)=cos(2*pi*t/Ty); G(:,4)=sin(2*pi*t/Ty); Ty is the number of days in a year, the period of the annual cycle.

posterior variance of data based on error of fit σd = 5.60 deg C prior variance of data based on accuracy of thermometer σd = 0.01 deg C posterior variance of data based on error of fit σd = 5.60 deg C You might discuss the rationale for using one of these estimates over the other. There are arguments both ways. The prior estimate is the better estimate of measurement noise. The posterior estimate more honestly reflects how well the model works (poorly). huge difference, since the model does not include diurnal cycle of weather patterns

long-term slope 95% confidence limits based on prior variance m2 = -0.03 ± 0.00002 deg C / yr 95% confidence limits based on posterior variance m2 = -0.03 ± 0.00460 deg C / yr in both cases, the cooling trend is significant, in the sense that the confidence intervals do not include zero or positive slopes. The key point if that the error bars do not overlap the m2=0, giving confidence to the assertion that the rate is negative.

However The fit to the data is poor, so the results should be used with caution. More effort needs to be put into developing a better model. Scientists should always be suspicious of models that only poorly fit the data.

covariance and the shape of the error surface part 3 covariance and the shape of the error surface

solutions within the region of low error are almost as good as mest 4 m2 mest m1 m2est large range of m1 E(m) The error surface was shown in the previous lecture. Mention that it was computed via a grid search. mi miest small range of m2 near the minimum the error is shaped like a parabola. The curvature of the parabola controls the with of the region of low error

near the minimum, the Taylor series for the error is: You should skip this slide and the nest two if the class has too low a mathematical level to understand Taylor series. curvature of the error surface

starting with the formula for error we compute its 2nd derivative

but so covariance of the model parameters curvature of the error surface

the covariance of the least squares solution is expressed in the shape of the error surface large variance small variance if you skipped the math, just asset the correspondence between variance and curvature of the error surface, E(m) E(m) mi mi miest miest