Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 171 Least Squares Regression Chapter 17 Linear Regression Fitting a straight line to a set of paired observations: (x 1, y 1 ), (x 2, y 2 ),…,(x n, y n ). y=a 0 +a 1 x+e a 1 - slope a 0 - intercept e- error, or residual, between the model and the observations

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 172 Criteria for a “Best” Fit/ Minimize the sum of the residual errors for all available data: n = total number of points However, this is an inadequate criterion, so is the sum of the absolute values

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 173 Figure 17.2

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 174 Best strategy is to minimize the sum of the squares of the residuals between the measured y and the y calculated with the linear model: Yields a unique line for a given set of data.

List-Squares Fit of a Straight Line/ Normal equations, can be solved simultaneously Mean values

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 176 Figure 17.3

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 177 Figure 17.4

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 178 Figure 17.5

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University 9 “Goodness” of our fit/ If Total sum of the squares around the mean for the dependent variable, y, is S t Sum of the squares of residuals around the regression line is S r S t -S r quantifies the improvement or error reduction due to describing data in terms of a straight line rather than as an average value. r 2 -coefficient of determination Sqrt(r 2 ) – correlation coefficient

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1710 For a perfect fit S r =0 and r=r 2 =1, signifying that the line explains 100 percent of the variability of the data. For r=r 2 =0, S r =S t, the fit represents no improvement.

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1711 Polynomial Regression Some engineering data is poorly represented by a straight line. For these cases a curve is better suited to fit the data. The least squares method can readily be extended to fit the data to higher order polynomials (Sec. 17.2).

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1712 General Linear Least Squares Minimized by taking its partial derivative w.r.t. each of the coefficients and setting the resulting equation equal to zero

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1713 Interpolation Chapter 18 Estimation of intermediate values between precise data points. The most common method is: Although there is one and only one nth-order polynomial that fits n+1 points, there are a variety of mathematical formats in which this polynomial can be expressed: –The Newton polynomial –The Lagrange polynomial

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1714 Figure 18.1

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1715 Newton’s Divided-Difference Interpolating Polynomials Linear Interpolation/ Is the simplest form of interpolation, connecting two data points with a straight line. f 1 (x) designates that this is a first-order interpolating polynomial. Linear-interpolation formula Slope and a finite divided difference approximation to 1 st derivative

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1716 Figure 18.2

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 17 Quadratic Interpolation/ If three data points are available, the estimate is improved by introducing some curvature into the line connecting the points. A simple procedure can be used to determine the values of the coefficients.

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Chapter 1818 General Form of Newton’s Interpolating Polynomials/ Bracketed function evaluations are finite divided differences

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1719 Errors of Newton’s Interpolating Polynomials/ Structure of interpolating polynomials is similar to the Taylor series expansion in the sense that finite divided differences are added sequentially to capture the higher order derivatives. For an n th -order interpolating polynomial, an analogous relationship for the error is: For non differentiable functions, if an additional point f(x n+1 ) is available, an alternative formula can be used that does not require prior knowledge of the function:  Is somewhere containing the unknown and he data

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1720 Lagrange Interpolating Polynomials The Lagrange interpolating polynomial is simply a reformulation of the Newton’s polynomial that avoids the computation of divided differences:

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1721 As with Newton’s method, the Lagrange version has an estimated error of:

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1722 Figure 18.10

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1723 Coefficients of an Interpolating Polynomial Although both the Newton and Lagrange polynomials are well suited for determining intermediate values between points, they do not provide a polynomial in conventional form: Since n+1 data points are required to determine n+1 coefficients, simultaneous linear systems of equations can be used to calculate “a”s.

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1724 Where “x”s are the knowns and “a”s are the unknowns.

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1725 Figure 18.13

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1726 Spline Interpolation There are cases where polynomials can lead to erroneous results because of round off error and overshoot. Alternative approach is to apply lower-order polynomials to subsets of data points. Such connecting polynomials are called spline functions.

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1727 Figure 18.14

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1728 Figure 18.15

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1729 Figure 18.16

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1730 Figure 18.17