1. ENME 392 Regression Theory
Regression Analysis
Regression analysis is an approach to develop a mathematical function from the experimental data in which uses a rigorous procedure that controls prediction errors.
Not just “curve fitting”
Guided by physics
ENME 392 Regression Theory

2. ENME 392 Regression Theory
Physical modeling – when you do not know the exact relationship, you can use Regression Analysis to identify the important factors AND using your understanding help identify the appropriate relationship.
Remember from the previous lecture the possible relationships:
Linear regression (Single independent variable)
Nonlinear regression
Multiple regression (Many independent variables)
Polynomial regression
ENME 392 Regression Theory

3. Regression Physical Example
In extrusion, fill length as a function of throughput is important. Suppose Dr. Bigio conducts some experiments and he wants to figure out what is the relation between the two variables.
Why? This model would help in defining a simple to use relationship between the variables. Could be used in a controls algorithm
ENME 392 Regression Theory

4. ENME 392 Regression Theory
Sample data
ENME 392 Regression Theory

5. ENME 392 Regression Theory
Data Plot
ENME 392 Regression Theory

6. ENME 392 Regression Theory
Method of Least Squares
Principle: Minimize [ Yi]2
Assume = a+bXi (linear relationship)
Minimize [Yi-(a+bXi)]2
ENME 392 Regression Theory

7. ENME 392 Regression Theory
Linear regression
ENME 392 Regression Theory

8. ENME 392 Regression Theory
Linear regression
Minimizing w.r.t. “a” (1)
Minimizing w.r.t. “b” (2)
ENME 392 Regression Theory

9. ENME 392 Regression Theory
Linear regression
Solving (1) and (2) we get:
ENME 392 Regression Theory

10. Linear regression line
This is a linear regression line, with Y linearly dependent on X
ENME 392 Regression Theory

11. ENME 392 Regression Theory

12. ENME 392 Regression Theory
Regression Line
The regression line from the output comes out as:
ENME 392 Regression Theory

13. ENME 392 Regression Theory
Interpretations
What do you observe from the results:
R-squared value –
Nature of the residuals
Is the result physically possible
NO! you cannot have a negative fill length.
Can you use the model?
YES!! It could be fine for a controls algorithm, as long as the range of the specific throughput is limited!!
ENME 392 Regression Theory

14. Standard Error -Single Variable
Estimates the error between the predicted values of Y and the actual values
From the output the value of = 7.29
ENME 392 Regression Theory

15. Correlation Coefficient
Correlation Coefficient ( r )
Estimates the strength of the linear relationship between X and Y.
Varies between -1 and 1
The closer “r” is in absolute value to 1, the greater the degree of correlation
ENME 392 Regression Theory

16. Correlation Coefficient
Mathematical expression where,
From the output r = 0.948
ENME 392 Regression Theory

17. ENME 392 Regression Theory
Nonlinear regression
Suppose one assumed a nonlinear relationship between Y and X. Would there be a better fit in that case?
ENME 392 Regression Theory

18. ENME 392 Regression Theory
Non linear regression
ENME 392 Regression Theory

19. ENME 392 Regression Theory
Nonlinear regression
Let us assume:
Taking a natural log on both sides:
This is now a linear relationship between lnY and X
ENME 392 Regression Theory

20. ENME 392 Regression Theory
Nonlinear regression
In this case,
ENME 392 Regression Theory

21. ENME 392 Regression Theory
Nonlinear regression
ENME 392 Regression Theory

22. ENME 392 Regression Theory

23. ENME 392 Regression Theory
Nonlinear regression
From the output data, the regression equation comes out to be:
This implies
ENME 392 Regression Theory

24. ENME 392 Regression Theory
Nonlinear regression
In this case, r= and Standard error = 0.05
This clearly shows that for the given set of values nonlinear regression achieved a better fit
ENME 392 Regression Theory

25. What happens when Y is dependent on more than one variable?
Multiple regression
What happens when Y is dependent on more than one variable?
ENME 392 Regression Theory

26. ENME 392 Regression Theory
Multiple regression
Suppose gasoline mileage (Y) is dependent on two variables: Fuel Octane rating (X1) Average Speed (X2)
How can one establish a regression equation relating Y with X1, X2 given the values of Y, X1 and X2 for some experiments?
ENME 392 Regression Theory

27. ENME 392 Regression Theory
Sample data
ENME 392 Regression Theory

28. ENME 392 Regression Theory
Multiple regression
Let = a + b1X1+b2X2
Using Method of Least Squares, one gets:
ENME 392 Regression Theory

29. ENME 392 Regression Theory
Multiple regression
Minimizing (1) w.r.t. a, b1, b (b) (c)
ENME 392 Regression Theory

30. ENME 392 Regression Theory
Multiple regression
ENME 392 Regression Theory

31. ENME 392 Regression Theory
Multiple regression
Solving the three equations, one can get the values of a, b1 and b2
Multiple Linear Regression Equation:
ENME 392 Regression Theory

32. ENME 392 Regression Theory
Multiple regression
In our example, a = b1 = b2=
This implies:
ENME 392 Regression Theory

33. ENME 392 Regression Theory
Multiple regression
Standard error: Similar to single variable linear regression, one can have standard error in multiple variables
ENME 392 Regression Theory

34. ENME 392 Regression Theory
Multiple regression
ENME 392 Regression Theory

35. ENME 392 Regression Theory
Multiple regression
In our example:
This summarizes the degree to which points are scattered around the regression plane.
ENME 392 Regression Theory

36. ENME 392 Regression Theory
Multiple regression
Advantages
rYX1=.74, rYX2=.081 and rX1X2=.53
There is poor correlation between Y and X2, and at the same time a good correlation between X1 and X2
This shows that the effect of X2 on Y is camouflaged by its interaction with X1
ENME 392 Regression Theory

37. ENME 392 Regression Theory

38. ENME 392 Regression Theory
Multiple regression
Advantages (contd.)
Thus, it would be a mistake not to include the effect of X2 on Y just on the fact that its correlation coefficient is low.
ENME 392 Regression Theory

39. Polynomial regression
ENME 392 Regression Theory

40. Polynomial regression
Suppose one had to develop a regression equation for the stress strain curve. It is difficult to apply any nonlinear transformation.
What can one do in this case?
ENME 392 Regression Theory

41. Polynomial regression
Assume
In other words, incorporate higher powers of X to obtain a better fit
ENME 392 Regression Theory

42. Polynomial regression
Data for the stress strain curve
ENME 392 Regression Theory

43. Polynomial regression
ENME 392 Regression Theory

44. ENME 392 Regression Theory

45. Polynomial regression
Regression Equation
Correlation Coefficient ( r ) = .922
ENME 392 Regression Theory

46. ENME 392 Regression Theory
Wrap Up
Regression
Linear (single variable)
Non linear
Multiple
Polynomial
Correlation coefficient
Standard error
ENME 392 Regression Theory