1. ELEC 413 Linear Least Squares

2. Regression Analysis The study and measure of the statistical relationship that exists between two or more variables Two variables  simple regression Three or more variables  multiple regression An estimating or predicting equation is developed to describe the pattern or functional nature of the relationship A regression plane replaces a regression line in multiple regression

3. Regression Analysis An independent or explanatory or regressor or predictor variable is the one that presumably exerts an influence on or explains variations in the dependent variable. The dependent or response variable is the variable to be estimated; customarily plotted on the vertical, or y-axis  denoted by ‘y’

4. The Scatter Diagram Outputs and aptitude test results of 8 employees of a toy manufacturing company are as shown: Employee Aptitude Test Output (Dozens Results (X) of units) (Y) A 6 30 B 9 49 C 3 18 D 8 42 E 7 39 F 5 25 G 8 41 H 10 52

7. Simple Linear Regression Analysis Considers a single regressor or predictor x and a dependent or response variable Y. The relationship between x and y can be adequately described by a straight line = computed estimate of dependent variable  0 = y-intercept  1 = slope of the regression line x = a given value of the independent variable

11. The method of least squares is used to estimate the parameters,  0 and  1 by minimizing the sum of the squares of the vertical deviations

12. Employee XYXYX 2 Y 2 A6 30 180 36 900 B 9 49 441 81 2401 C 3 18 54 9 324 D 8 42 336 64 1764 E 7 39 273 49 1521 F 5 25 125 25 625 G 8 41 328 64 1681 H 10 52 520 100 2704 __ ___ ____ ___ _____ Total 56 296 2257 428 11920

14. Matrix Formulation of Simple Least Squares Rewriting the two normal equations, In matrix notation,

15. Matrix Formulation of Simple Least Squares Recall, where

16. Matrix Formulation of Simple Least Squares The normal equations resulting from is given by If the matrix X T X is nonsingular, the solution for the least-squares coefficients can be found by is called a pseudo-inverse.

17. Polynomial Model Fit the polynomial equation to the N pairs of data M is the degree of the polynomial The polynomial model is considered a special case of general multiple linear model

18. Example Fit a parabola curve to the following data x0123456789 y9.17.33.24.64.82.95.77.18.810.2

19. Making Predictions with Regression Eq. Estimate or predict values of the dependent variable given values of the independent variable

20. Logical Reasoning Fallacy Post Hoc Ergo Propter Hoc The Trend must go on

21. Does a true relationship exist? A true relationship doesn’t exist if  1 is zero

22. Example The electric power consumed each month (y) by a chemical plant is thought to be related to x 1 = the average ambient temperature, x 2 = the number of days in the month, x 3 = the average product purity, and x 4 = the tons of product produced. The data were recorded in the table (next page). Fit a multiple linear regression model to the data and predict power consumption in a month in which x 1 = 75 o F, x 2 = 24 days, x 3 = 90%, and x 4 = 98 tons.

23. Power X 1 X 2 X 3 X 4 24025 2491100 236312190 95 290452488110 274602587 88 301652591 94 316 722694 99 300802587 97 296842586 96 267752488110 276602591105 288502590100 261382389 98

24. Example Find c 0, c 1, and c 2 so that the function is a least-squares fit to a triangular wave with period of 5 ms. Use a sampling frequency of 3000 Hz. Show the fit by plotting one period of the triangular wave.