1. Read Chapter 17 of the textbook
SE301: Numerical Methods Topic 4: Least Squares Curve Fitting Lectures 18-19:
KFUPM
Read Chapter 17 of the textbook
CISE301_Topic4
KFUPM

2. Lecture 18 Introduction to Least Squares
CISE301_Topic4
KFUPM

3. Motivation Given a set of experimental data: x 1 2 3 y 5.1 5.9 6.3
The relationship between x and y may not be clear.
We want to find an expression for f(x).
CISE301_Topic4
KFUPM

4. Motivation - Model Building
In engineering, two types of applications are encountered:
Trend analysis: Predicting values of dependent variable, may include extrapolation beyond data points or interpolation between data points.
Hypothesis testing: Comparing existing mathematical model with measured data.
What is the best mathematical model (function f , y) that represents the dataset yi?
What is the best criterion to assess the fitting of the function y to the data?
CISE301_Topic4
KFUPM

5. Motivation - Curve Fitting
Given a set of tabulated data, find a curve or a function that best represents the data.
Given:
The tabulated data
The form of the function
The curve fitting criteria
Find the unknown coefficients
CISE301_Topic4
KFUPM

6. Least Squares Regression
Linear Regression
‌Fitting a straight line to a set of paired observations:
(x1, y1), (x2, y2),…,(xn, yn).
y=a0+a1x+e
a1-slope.
a0-intercept.
e-error, or residual, between the model and the observations.
CISE301_Topic4
KFUPM

7. Selection of the Functions
CISE301_Topic4
KFUPM

8. Decide on the Criterion
Chapter 17
Chapter 18
CISE301_Topic4
KFUPM

9. Least Squares xi x1 x2 …. xn yi y1 y2 yn Given:
The form of the function is assumed to be known but the coefficients are unknown.
The difference is assumed to be the result of experimental error.
CISE301_Topic4
KFUPM

10. Determine the Unknowns
CISE301_Topic4
KFUPM

11. Determine the Unknowns
CISE301_Topic4
KFUPM

12. Example 1 x 1 2 3 y
5.1
5.9
6.3
CISE301_Topic4
KFUPM

13. Remember
CISE301_Topic4
KFUPM

14. Example 1
CISE301_Topic4
KFUPM

15. Example 1
CISE301_Topic4
KFUPM

16. Example 1 i 1 2 3 sum xi 6 yi 5.1 5.9 6.3 17.3 xi2 4 9 14 xi yi 11.8
18.9
35.8
CISE301_Topic4
KFUPM

17. Example 1
CISE301_Topic4
KFUPM

18. Example 2 - Fitting with Nonlinear Functions -
0.24
0.65
0.95
1.24
1.73
2.01
2.23
2.52 y
0.23
-0.23
-1.1
-0.45
0.27
0.1
-0.29
CISE301_Topic4
KFUPM

19. Example 2
CISE301_Topic4
KFUPM

20. Example 2
CISE301_Topic4
KFUPM

21. How Do You Judge Performance?
CISE301_Topic4
KFUPM

22. Multiple Regression Example: Given the following data:
It is required to determine a function of two variables:
f(x,t) = a + b x + c t
to explain the data that is best in the least square sense. t 1 2 3 x
0.1
0.4
0.2
f(x,t)
CISE301_Topic4
KFUPM

23. Solution of Multiple Regression1 2 3 x
0.1
0.4
0.2
f(x,t)
Construct , the sum of the square of the error and derive the necessary conditions by equating the partial derivatives with respect to the unknown parameters to zero, then solve the equations.
CISE301_Topic4
KFUPM

24. Solution of Multiple Regression
CISE301_Topic4
KFUPM

25. Lecture 19 Nonlinear Least Squares Problems + More
Examples of Nonlinear Least Squares
Solution of Inconsistent Equations
Continuous Least Square Problems
CISE301_Topic4
KFUPM

26. Nonlinear Problem x 1 2 3 y
2.4 5 9
Given:
CISE301_Topic4
KFUPM

27. Alternative Solution (Linearization Method)x 1 2 3 y
2.4 5 9
Given:
CISE301_Topic4
KFUPM

28. Example (Linearization Method)1 2 3 y
0.23 .2
.14
Given:
CISE301_Topic4
KFUPM

29. Inconsistent System of Equations
CISE301_Topic4
KFUPM

30. Inconsistent System of Equations - Reasons -
Inconsistent equations may occur because of:
Errors in formulating the problem,
Errors in collecting the data, or
Computational errors.
Solution exists if all lines intersect at one point.
CISE301_Topic4
KFUPM

31. Inconsistent System of Equations - Formulation as a Least Squares Problem -
CISE301_Topic4
KFUPM

32. Solution
CISE301_Topic4
KFUPM

33. Solution
CISE301_Topic4
KFUPM