1. LESSON 21: REGRESSION ANALYSIS
Outline
Linear Regression
The Method of Least Squares

2. Linear Regression
This lesson addresses the problem of finding a relationship between two population. The goal is to predict values for one population on the basis of observations taken from the other.
Engineers often encounter such problems whenever they need to fit a curve to experimental data. Recall that a scatter diagram is a graph that plots points representing relationship between two variables, an independent variable and a dependent variable A curve can be fitted to the scatter plot using a regression analysis.
A linear regression assumes a linear relationship and fits a straight line. The method of least square is a method that finds the particular line.

3. Linear Regression Y Dependent variable
Independent variable X Y
The scatter diagram on this slide shows a linear relationship between two variables. 9

4. Linear Regression Dependent variable Independent variable X Y
equation:
Y = a + bX
A straight line of the form Y = a+bX nicely fits the points. Linear regression provides values of parameters a and b 10

5. The Method of Least Squares
Dependent variable
Independent variable X Y
Actual
value
of Y
Estimate of
Y from
regression
equation
Value of X used
to estimate Y
Deviation,
or error {
Regression
equation:
Y = a + bX
The least square method finds a and b such that the sum of the squares of errors is minimum 13

6. The Method of Least Squares
Consider n observations
For the ith observation Xi, the predicted value
The least square method finds a and b to minimize the sum of the squares of deviations of the predicted values from the actual values

7. The Method of Least Squares
The following a and b minimize the sum of the squares of deviations

8. Linear Regression
The standard deviation of the individual Y observations:
The standard error of the Y estimate
Note: The divisor is n minus the number of regression coefficients.

9. Linear Regression Using Least Squares
Future sales are unknown, but future advertising expenses are given by marketing plan. A known value of advertising expense is used to forecast sales. For such a forecast, we need the relationship between advertising and sales.
Sales Advertising
Month (000 units) (000 $)
Since sales depends on advertising, sales is the dependent variable and shown on the Y-axis. Advertising is the independent variable and shown on the X-axis. 14

10. Linear Regression Using Least Squares
a = Y - bX
b =
XY - nXY
X 2 - nX 2
Sales, Y Advertising, X
Month (000 units) (000 $) XY X 2
Total
Y= X = 17

11. Linear Regression Using Least Squares
a = Y - bX
b =
XY - nXY
X 2 - nX 2
Sales, Y Advertising, X
Month (000 units) (000 $) XY X 2
Total
Y= 171 X = 1.64

12. Linear Regression Using Least Squares
b =
Sales, Y Advertising, X
Month (000 units) (000 $) XY X 2
Total
Y = 171 X = 1.64

13. Linear Regression Using Least Squares
300 —
250 —
200 —
150 —
100 — 50
Y = (X)
Interpretation: For each $1000 increase in advertising, sales increases by 109,229 units.
Sales (000s)
| | | | 24

14. Linear Regression Using Least Squares
The regression equation can be used to forecast sales of Month 6 from a known value of advertising expenditure in Month 6.
Forecast for Month 6:
Let advertising expenditure = $1750
Y = 29

15. Linear Regression Using Least Squares
Forecast for Month 6:
Let advertising expenditure = $1750
Y = (1.75) = thousand units 29

16. Example
Example 1: The following sample observations have been obtained by a chemical engineer investigating the relationship between weight of final product Y (in pounds) and volume of raw materials X in gallons. Find a and b:
X Y

17. Example
X Y XY X2

18. Example
Example 2: Consider Example 1. Compute the sample standard deviation for final product weight.

19. Example
Example 3: Consider Example 1. Compute the standard error of estimate for final product weight.

20. READING AND EXERCISES Lesson 21 Reading: Section 4-1 pp. 90-102
4-1, 4-2