ELEC 413 Linear Least Squares

Regression Analysis The study and measure of the statistical relationship that exists between two or more variables Two variables  simple regression Three or more variables  multiple regression An estimating or predicting equation is developed to describe the pattern or functional nature of the relationship A regression plane replaces a regression line in multiple regression

Regression Analysis An independent or explanatory or regressor or predictor variable is the one that presumably exerts an influence on or explains variations in the dependent variable. The dependent or response variable is the variable to be estimated; customarily plotted on the vertical, or y-axis  denoted by ‘y’

The Scatter Diagram Outputs and aptitude test results of 8 employees of a toy manufacturing company are as shown: Employee Aptitude Test Output (Dozens Results (X) of units) (Y) A 6 30 B 9 49 C 3 18 D 8 42 E 7 39 F 5 25 G 8 41 H 10 52

Simple Linear Regression Analysis Considers a single regressor or predictor x and a dependent or response variable Y. The relationship between x and y can be adequately described by a straight line = computed estimate of dependent variable  0 = y-intercept  1 = slope of the regression line x = a given value of the independent variable

The method of least squares is used to estimate the parameters,  0 and  1 by minimizing the sum of the squares of the vertical deviations

Employee XYXYX 2 Y 2 A B C D E F G H __ ___ ____ ___ _____ Total

Matrix Formulation of Simple Least Squares Rewriting the two normal equations, In matrix notation,

Matrix Formulation of Simple Least Squares Recall, where

Matrix Formulation of Simple Least Squares The normal equations resulting from is given by If the matrix X T X is nonsingular, the solution for the least-squares coefficients can be found by is called a pseudo-inverse.

Polynomial Model Fit the polynomial equation to the N pairs of data M is the degree of the polynomial The polynomial model is considered a special case of general multiple linear model

Example Fit a parabola curve to the following data x y

Making Predictions with Regression Eq. Estimate or predict values of the dependent variable given values of the independent variable

Logical Reasoning Fallacy Post Hoc Ergo Propter Hoc The Trend must go on

Does a true relationship exist? A true relationship doesn’t exist if  1 is zero

Example The electric power consumed each month (y) by a chemical plant is thought to be related to x 1 = the average ambient temperature, x 2 = the number of days in the month, x 3 = the average product purity, and x 4 = the tons of product produced. The data were recorded in the table (next page). Fit a multiple linear regression model to the data and predict power consumption in a month in which x 1 = 75 o F, x 2 = 24 days, x 3 = 90%, and x 4 = 98 tons.

Power X 1 X 2 X 3 X

Example Find c 0, c 1, and c 2 so that the function is a least-squares fit to a triangular wave with period of 5 ms. Use a sampling frequency of 3000 Hz. Show the fit by plotting one period of the triangular wave.