2. Multiple Regression (continued)& Polynomial Regression

3. Theory Consider simplest form of multiple regression where y is the dependant variable and x 1 and x 2 independent variables y = b 0 + b 1 x 1 + b 2 x 2 + e Where e is a random error term

4. Theory b 1 = [  (x 2 2 )(  x 1 y)-(  x 1 x 2 )(  x 2 y)] [(  x 1 2 )(  x 2 2 )-(  x 1 x 2 ) 2 ] [(  x 1 2 )(  x 2 2 )-(  x 1 x 2 ) 2 ] b 2 = [  (x 1 2 )(  x 2 y)-(  x 1 x 2 )(  x 1 y)] [(  x 2 2 )(  x 1 2 )-(  x 1 x 2 ) 2 ] [(  x 2 2 )(  x 1 2 )-(  x 1 x 2 ) 2 ] b 0 = mean(y)-b 1 mean(x 1 )-b 2 mean(x 2 )

5. Analysis of Variance Table

6. Introduction to Matrixes 6b 1 + 3b 2 = 24 4b 1 + 4 b 2 = 20 } Simultaneous equations 6 3 b 1 24 4 4 b 2 20 x= } [[[]]] Matrix Form

7. Matrix Formation y = b 0 + b 1 x 1 + b 2 x 2 + ….. + b n x n e Y = X x b + e

8. Matrix Formation Y = X x b

9. Matrix Formation F = ee’ = YY’ - 2YX’b + bb’ XX’ dF/db = 2XX’ b - 2YX’ = 0 XX’b = YX’

10. Two Variable Example x 11 x 12 x 21 x 22 x 31 x 32 x 41 x 42 x 51 x 52 x 11 x 21 x 31 x 41 x 51 x 12 x 22 x 32 x 42 x 52 =  x 1 2  x 1 x 2  x 2 x 1  x 2 2 x

11. Matrix Formation XX’ =

12. Two Variable Example = x1y x1yx2yx2y x1y x1yx2yx2y y1y1y2y2y1y1y2y2 x x 11 x 12 x 21 x 22

13. Matrix Formation = YX’

14. Two Variable Example  x 1 2  x 1 x 2 b 1  x 1 y  x 2 x 1  x 2 2 b 2  x 2 y x= [[[]]] XX’ x b = YX’ (XX’) -1 XX’ x b = (XX’) -1 YX ’ b = (XX’) -1 YX ’ b = (XX’) -1 YX ’

15. Matrix Formation Find the inverse of XX’ Donated by (XX’) -1 b = (XX’) -1 YX’

16. Matrix Inverse with Two Variables A x A -1 = [U] A x A -1 = [U]

17. Matrix Inverse with Two Variables a b c d d -b -c a 1ad-bc [[]] x [] 1 0 1 0 0 1 0 1 = A x A -1 = [U] A x A -1 = [U]

18. Matrix Inverse with Two Variables  x 1 2  x 1 x 2 b 1  x 1 y  x 2 x 1  x 2 2 b 2  x 2 y x= [[[]]]

19. Matrix Inverse with Two Variables  x 1 2  x 1 x 2 b 1  x 1 y  x 2 x 1  x 2 2 b 2  x 2 y x= [[[]]] XX’ x b = X’Y

20. Matrix Inverse with Two Variables  x 1 2  x 1 x 2 b 1  x 1 y  x 2 x 1  x 2 2 b 2  x 2 y x= [[[]]] XX’ x b = X’Y  x 1 2  x 1 x 2  x 2 2 -  x 1 x 2  x 2 x 1  x 2 2 -  x 2 x 1  x 1 2 x [[]] 1 ad-bc = [U] XX’ x (XX’) -1 = Unit

21. Matrix Inverse with Two Variables  x 2 2 -  x 1 x 2  x 1 y b 1 -  x 2 x 1  x 1 2  x 2 y b 2 x [[]] 1 ad-bc= (XX’) -1 x Y = b [] 1 ad-bc =  x 2 2  x 1 2 - [  x 2 x 1 ] 2

22. Compare Matrix with None b 1 = [  (x 2 2 )(  x 1 y)-(  x 1 x 2 )(  x 2 y)] [(  x 1 2 )(  x 2 2 )-(  x 1 x 2 ) 2 ] [(  x 1 2 )(  x 2 2 )-(  x 1 x 2 ) 2 ] b 2 = [  (x 1 2 )(  x 2 y)-(  x 1 x 2 )(  x 1 y)] [(  x 2 2 )(  x 1 2 )-(  x 1 x 2 ) 2 ] [(  x 2 2 )(  x 1 2 )-(  x 1 x 2 ) 2 ]

23. Forward Step- Wise Regression

24. Two Variable Multiple Regression

25. Analysis of Variance Table y = 6336 - 23.75 x 1 + 150.27 x 2

26. Two Variable Multiple Regression  There is significant regression effects by regressing both independent variables onto the dependant variable  The is significant linear relationship between height (x 1 ) and yield but no relationship between yield and tiller  There is significant linear relationship between tiller (x 2 ) and yield and no relationship between yield and height

27. Two Variable Multiple Regression  Forward Step-wise Regression  Backward Step-wise Regression We may have made the relationship too complex by including both variables.

28. Two Variable Multiple Regression

29. Analysis of Variance Table y = 10,131 - 37.11 Height (x 1 )

30. Analysis of Variance Table y = 6336 - 23.75 Height + 150.27 Tiller

31. Analysis of Variance Table y = 6336 - 23.75 Height + 150.27 Tiller

32. Analysis of Variance Table y = 6336 - 23.75 Height + 150.27 Tiller

33. Analysis of Variance Table y = 10,131 - 37.11 Height (x 1 )

34. Forward Step-Wise Regression Example 2 20 Spring Canola Cultivars Average over 10 environments Seed yield; plant establishment; days to first flowering, days to end of flowering; plant height; and oil content

35. CharacterEst.F.St.F.Fi.Ht.%OilYield Establish1.00 F.Start-0.301.00 F.Finish-0.150.931.00 Height-0.450.720.701.00 %Oil0.04-0.51-0.52-0.271.00 Yield0.31-0.82-0.80-0.53-0.211.00 Example #2

36. CharacterEst.F.St.F.Fi.Ht.%OilYield Establish1.00-0.30-0.15-0.450.040.31 F.Start-0.301.000.930.72-0.51-0.82 F.Finish-0.150.931.000.700.520.80 Height-0.450.720.701.00-0.27-0.53 %Oil0.04-0.51-0.52-0.271.00-0.21 Yield0.31-0.82-0.80-0.53-0.211.00 Example #2

37. Analysis of Variance Table y = 3,194 - 32.9 x F.Start

38. CharacterEst.F.St.F.Fi.Ht.%OilYield Establish1.00-0.30-0.15-0.450.040.31 F.Start-0.301.000.930.72-0.51-0.82 F.Finish-0.150.931.000.700.520.80 Height-0.450.720.701.00-0.27-0.53 %Oil0.04-0.51-0.52-0.271.00-0.21 Yield0.31-0.82-0.80-0.53-0.211.00 Example #2 A[i,j] = A[i,j]–{A i,x x A x,j }/A x,x

39. CharacterEst.F.St.F.Fi.Ht.%OilYield Establish1.00-0.30-0.15-0.450.040.31 F.Start-0.301.000.930.72-0.51-0.82 F.Finish-0.150.931.000.700.520.80 Height-0.450.720.701.00-0.27-0.53 %Oil0.04-0.51-0.52-0.271.00-0.21 Yield0.31-0.82-0.80-0.53-0.211.00 Example #2 A[i,j] = A[i,j]–{A i,x x A x,j }/A x,x

40. CharacterEst.F.St.F.Fi.Ht.%OilYield Establish1.00-0.30-0.15-0.450.040.31 F.Start-0.301.000.930.72-0.51-0.82 F.Finish-0.150.931.000.700.520.80 Height-0.450.720.701.00-0.27-0.53 %Oil0.04-0.51-0.52-0.271.00-0.21 Yield0.31-0.820.04-0.53-0.211.00 Example #2 A[i,j] = A[i,j]–{A i,x x A x,j }/A x,x

41. CharacterEst.F.St.F.Fi.Ht.%OilYield Establish1.00-0.30-0.15-0.450.040.31 F.Start-0.301.000.930.72-0.51-0.82 F.Finish-0.150.931.000.700.520.80 Height-0.450.720.701.00-0.27-0.53 %Oil0.04-0.51-0.52-0.271.00-0.21 Yield0.31-0.820.04-0.53-0.211.00 Example #2 A[i,j] = A[i,j]–{A i,x x A x,j }/A x,x

42. CharacterEst.F.St.F.Fi.Ht.%OilYield Establish1.00-0.30-0.15-0.450.040.31 F.Start-0.301.000.930.72-0.51-0.82 F.Finish-0.150.931.000.700.520.80 Height-0.450.720.701.00-0.27-0.53 %Oil0.04-0.51-0.52-0.271.00-0.21 Yield0.31-0.82-0.80-0.53-0.631.00 Example #2 A[i,j] = A[i,j]–{A i,x x A x,j }/A x,x

43. CharacterEst.F.St.F.Fi.Ht.%OilYield Establish0.9100.38-0.230.110.06 F.Start000000 F.Finish0.3800.360.03-0.050.04 Height-0.2300.030.480.100.06 %Oil0.110-0.050.100.740.21 Yield0.0600.040.06-0.630.33 Example #2 A[i,j] = A[i,j]–{A i,x x A x,j }/A x,x

44. Analysis of Variance Table y = 5335 - 38.4 F.Start - 57.7 %Oil

45. CharacterEst.F.St.F.Fi.Ht.%OilYield Establish0.9000.39-0.2300.04 F.Start000000 F.Finish0.3900.36-0.2300.04 Height-0.230 0.4800.11 %Oil000000 Yield0.0400.050.1100.29 Example #2 A[i,j] = A[i,j]–{A i,x x A x,j }/A x,x

46. Analysis of Variance Table y = 6779 - 30.4 FS - 63.0 %Oil + 8.7 Height

47. Analysis of Variance Table y = 5335 - 38.4 F.Start - 57.7 %Oil

48. Forward Step-Wise Regression  Enter the variable “most associated with the dependant variable.  Check to see if relationship is significant  Adjust the relationship between the dependant variable and the other remaining variables, accounting for the relationship between the dependant variable and the entered variable(s)

49. Forward Step- Wise Regression

50. Enter most correlated variable Forward Step- Wise Regression

51. Enter most correlated variable Check that entry is significant Forward Step- Wise Regression

52. Enter most correlated variable Check that entry is significant Adjust correlation with other variables Forward Step- Wise Regression

53. Enter most correlated variable Check that entry is significant Adjust correlation with other variables Forward Step- Wise Regression

54. Polynomial Regression

58. Analysis of Variance Table y = -36.25 + 15.730 N - 0.218 N 2

59. Polynomial Regression dY/dN = Slope y = -36.25 + 15.730 N - 0.218 N 2

60. Polynomial Regression y = -36.25 + 15.730 N - 0.218 N 2 dy/dN = +15.730 - 0.436 N 0.436 N = 15.730 n = 36.08

61. Multivariate Transformation