Read Chapter 17 of the textbook SE301: Numerical Methods Topic 4: Least Squares Curve Fitting Lectures 18-19: KFUPM Read Chapter 17 of the textbook CISE301_Topic4 KFUPM
Lecture 18 Introduction to Least Squares CISE301_Topic4 KFUPM
Motivation Given a set of experimental data: x 1 2 3 y 5.1 5.9 6.3 The relationship between x and y may not be clear. We want to find an expression for f(x). 1 2 3 CISE301_Topic4 KFUPM
Motivation - Model Building In engineering, two types of applications are encountered: Trend analysis: Predicting values of dependent variable, may include extrapolation beyond data points or interpolation between data points. Hypothesis testing: Comparing existing mathematical model with measured data. What is the best mathematical model (function f , y) that represents the dataset yi? What is the best criterion to assess the fitting of the function y to the data? CISE301_Topic4 KFUPM
Motivation - Curve Fitting Given a set of tabulated data, find a curve or a function that best represents the data. Given: The tabulated data The form of the function The curve fitting criteria Find the unknown coefficients CISE301_Topic4 KFUPM
Least Squares Regression Linear Regression Fitting a straight line to a set of paired observations: (x1, y1), (x2, y2),…,(xn, yn). y=a0+a1x+e a1-slope. a0-intercept. e-error, or residual, between the model and the observations. CISE301_Topic4 KFUPM
Selection of the Functions CISE301_Topic4 KFUPM
Decide on the Criterion Chapter 17 Chapter 18 CISE301_Topic4 KFUPM
Least Squares xi x1 x2 …. xn yi y1 y2 yn Given: The form of the function is assumed to be known but the coefficients are unknown. The difference is assumed to be the result of experimental error. CISE301_Topic4 KFUPM
Determine the Unknowns CISE301_Topic4 KFUPM
Determine the Unknowns CISE301_Topic4 KFUPM
Example 1 x 1 2 3 y 5.1 5.9 6.3 CISE301_Topic4 KFUPM
Remember CISE301_Topic4 KFUPM
Example 1 CISE301_Topic4 KFUPM
Example 1 CISE301_Topic4 KFUPM
Example 1 i 1 2 3 sum xi 6 yi 5.1 5.9 6.3 17.3 xi2 4 9 14 xi yi 11.8 18.9 35.8 CISE301_Topic4 KFUPM
Example 1 CISE301_Topic4 KFUPM
Example 2 - Fitting with Nonlinear Functions - 0.24 0.65 0.95 1.24 1.73 2.01 2.23 2.52 y 0.23 -0.23 -1.1 -0.45 0.27 0.1 -0.29 CISE301_Topic4 KFUPM
Example 2 CISE301_Topic4 KFUPM
Example 2 CISE301_Topic4 KFUPM
How Do You Judge Performance? CISE301_Topic4 KFUPM
Multiple Regression Example: Given the following data: It is required to determine a function of two variables: f(x,t) = a + b x + c t to explain the data that is best in the least square sense. t 1 2 3 x 0.1 0.4 0.2 f(x,t) CISE301_Topic4 KFUPM
Solution of Multiple Regression 1 2 3 x 0.1 0.4 0.2 f(x,t) Construct , the sum of the square of the error and derive the necessary conditions by equating the partial derivatives with respect to the unknown parameters to zero, then solve the equations. CISE301_Topic4 KFUPM
Solution of Multiple Regression CISE301_Topic4 KFUPM
Lecture 19 Nonlinear Least Squares Problems + More Examples of Nonlinear Least Squares Solution of Inconsistent Equations Continuous Least Square Problems CISE301_Topic4 KFUPM
Nonlinear Problem x 1 2 3 y 2.4 5 9 Given: CISE301_Topic4 KFUPM
Alternative Solution (Linearization Method) x 1 2 3 y 2.4 5 9 Given: CISE301_Topic4 KFUPM
Example (Linearization Method) 1 2 3 y 0.23 .2 .14 Given: CISE301_Topic4 KFUPM
Inconsistent System of Equations CISE301_Topic4 KFUPM
Inconsistent System of Equations - Reasons - Inconsistent equations may occur because of: Errors in formulating the problem, Errors in collecting the data, or Computational errors. Solution exists if all lines intersect at one point. CISE301_Topic4 KFUPM
Inconsistent System of Equations - Formulation as a Least Squares Problem - CISE301_Topic4 KFUPM
Solution CISE301_Topic4 KFUPM
Solution CISE301_Topic4 KFUPM