EE3561_Unit 4(c)AL-DHAIFALLAH14351 EE 3561 : Computational Methods Unit 4 : Least Squares Curve Fitting Dr. Mujahed Al-Dhaifallah (Term 342) Reading Assignment :

Elementary Statistics  A statistical sample is a fraction or a portion of the whole (population) that is studied.  For example Consider Table 1 which contains 14 measurements of the concentration of sodium chlorate produced in a chemical reactor operated at a pH of 7.0. EE3561_Unit 4(c)AL-DHAIFALLAH14352

Elementary Statistics  One of the measures of the spread of the data is the range of the data. The range is defined as the difference between the maximum and minimum value of the data as R  However, range may not give a good idea of the spread of the data as some data points may be far away from most other data points (such data points are called outliers). EE3561_Unit 4(c)AL-DHAIFALLAH14353

Elementary Statistics  That is why the sum of the square of the differences is considered a better measure. The sum of the squares of the differences, also called summed squared error (SSE), S t, is given by  Since the magnitude of the summed squared error is dependent on the number of data points, an average value of the summed squared error is defined as the variance, σ 2 EE3561_Unit 4(c)AL-DHAIFALLAH14354

Elementary Statistics  To bring the variation back to the same level of units as the original data, a new term called standard deviation, σ, is defined as  Furthermore, the ratio of the standard deviation to the mean, known as the coefficient of variation is also used to normalize the spread of a sample. EE3561_Unit 4(c)AL-DHAIFALLAH14355

Example EE3561_Unit 46

Example EE3561_Unit 4(c)AL-DHAIFALLAH14357

EE3561_Unit 4(c)AL-DHAIFALLAH14358 Motivation Given a set of experimental data x123 y The relationship between x and y may not be clear we want to find an expression for f(x) 1 2 3

EE3561_Unit 4(c)AL-DHAIFALLAH14359 Curve Fitting Given a set of tabulated data, find a curve or a function that best represents the data. Given: 1. The tabulated data 2. The form of the function 3. The curve fitting criteria Find the unknown coefficients

EE3561_Unit 4(c)AL-DHAIFALLAH Selection of the functions

EE3561_Unit 4(c)AL-DHAIFALLAH Decide on the criterion Chapter 17 Chapter 18

EE3561_Unit 4(c)AL-DHAIFALLAH Least Squares xixi x1x1 x2x2 ….xnxn yiyi y1y1 y2y2 ynyn Given The form of the function is assumed to be known but the coefficients are unknown The difference is assumed to be the result of experimental error

EE3561_Unit 4(c)AL-DHAIFALLAH Determine the Unknowns

EE3561_Unit 4(c)AL-DHAIFALLAH Determine the Unknowns

EE3561_Unit 4(c)AL-DHAIFALLAH Example 1 x123 y

EE3561_Unit 4(c)AL-DHAIFALLAH Remember

EE3561_Unit 4(c)AL-DHAIFALLAH Example 1

EE3561_Unit 4(c)AL-DHAIFALLAH Example 1

EE3561_Unit 4(c)AL-DHAIFALLAH Example 1 i123sum xixi 1236 yiyi xi2xi x i y i

EE3561_Unit 4(c)AL-DHAIFALLAH Example 1

EE3561_Unit 4(c)AL-DHAIFALLAH Example 2 x y

EE3561_Unit 4(c)AL-DHAIFALLAH Example 2

EE3561_Unit 4(c)AL-DHAIFALLAH Example 2

EE3561_Unit 4(c)AL-DHAIFALLAH i sum xixi yiyi ln(x i ) cos(x i ) e xi (ln(x i )) ln(x)cos(x) ln(x i ) e xi ln(x i ) y i (cos(x i )) cos(x i )e xi cos(x i )y i (e xi ) e xi y i

Example 2 EE3561_Unit 4(c)AL-DHAIFALLAH Solve to get

EE3561_Unit 4(c)AL-DHAIFALLAH How do you judge performance?

EE3561_Unit 4(c)AL-DHAIFALLAH Multiple Regression Example: Given the following data It is required to determine a function of two variables f(x,t) = a + b x + c t to explain the data that is best in the least square sense. t0123 x f(x,t)3212

EE3561_Unit 4(c)AL-DHAIFALLAH Solution of Multiple Regression Construct, the sum of the square of the error and derive the necessary conditions by equating the partial derivatives with respect to the unknown parameters to zero then solve the equations. t0123 x f(x,t)3212

EE3561_Unit 4(c)AL-DHAIFALLAH Solution of Multiple Regression

EE3561_Unit 4(c)AL-DHAIFALLAH Nonlinear least squares problems + More  Examples of nonlinear least squares  Solution of inconsistent equations  Continuous least square problems

EE3561_Unit 4(c)AL-DHAIFALLAH Nonlinear Problem Given x123 y2.45.9

EE3561_Unit 4(c)AL-DHAIFALLAH Alternative Solution (Linearization Method) Given x123 y2.45.9

EE3561_Unit 4(c)AL-DHAIFALLAH Inconsistent System of Equations

EE3561_Unit 4(c)AL-DHAIFALLAH Inconsistent System of Equations Reasons Inconsistent equations may occur because of presence of noise in input or output data, or modeling error, etc. Solution if all lines intersect at one point

EE3561_Unit 4(c)AL-DHAIFALLAH Inconsistent System of Equations Formulation as a least squares problem

EE3561_Unit 4(c)AL-DHAIFALLAH Solution

EE3561_Unit 4(c)AL-DHAIFALLAH Solution

EE3561_Unit 4(c)AL-DHAIFALLAH Examples (Linearization Method) Given x123 y

Exercise EE3561_Unit 4(c)AL-DHAIFALLAH143539