1. Statistics for Business and Economics Chapter 10 Simple Linear Regression

2. Learning Objectives 1.Describe the Linear Regression Model 2.State the Regression Modeling Steps 3.Explain Least Squares 4.Compute Regression Coefficients 5.Explain Correlation 6.Predict Response Variable

3. Models

4. Representation of some phenomenon Mathematical model is a mathematical expression of some phenomenon Often describe relationships between variables Types –Deterministic models –Probabilistic models

5. Deterministic Models Hypothesize exact relationships Suitable when prediction error is negligible Example: force is exactly mass times acceleration –F = m·a

6. Probabilistic Models Hypothesize two components –Deterministic –Random error Example: sales volume (y) is 10 times advertising spending (x) + random error –y = 10x +  –Random error may be due to factors other than advertising

7. Types of Probabilistic Models Probabilistic Models Regression Models Correlation Models

8. Regression Models

9. Types of Probabilistic Models Probabilistic Models Regression Models Correlation Models

10. Regression Models Answers ‘What is the relationship between the variables?’ Equation used –One numerical dependent (response) variable  What is to be predicted –One or more numerical or categorical independent (explanatory) variables Used mainly for prediction and estimation

11. Regression Modeling Steps 1.Hypothesize deterministic component 2.Estimate unknown model parameters 3.Specify probability distribution of random error term Estimate standard deviation of error 4.Evaluate model 5.Use model for prediction and estimation

12. Model Specification

13. Regression Modeling Steps 1.Hypothesize deterministic component 2.Estimate unknown model parameters 3.Specify probability distribution of random error term Estimate standard deviation of error 4.Evaluate model 5.Use model for prediction and estimation

14. Specifying the Model 1.Define variables Conceptual (e.g., Advertising, price) Empirical (e.g., List price, regular price) Measurement (e.g., $, Units) 2.Hypothesize nature of relationship Expected effects (i.e., Coefficients’ signs) Functional form (linear or non-linear) Interactions

15. Model Specification Is Based on Theory Theory of field (e.g., Sociology) Mathematical theory Previous research ‘Common sense’

16. Advertising Sales Advertising Sales Advertising Sales Advertising Sales Thinking Challenge: Which Is More Logical?

17. Types of Relationships Y X Y X Y Y X X Strong relationshipsWeak relationships (continued)

18. Types of Relationships Y X Y X No relationship (continued)

19. Simple 1 Explanatory Variable Types of Regression Models Regression Models 2+ Explanatory Variables Multiple Linear Non- Linear Non- Linear

20. Linear Regression Model

21. Types of Regression Models Simple 1 Explanatory Variable Regression Models 2+ Explanatory Variables Multiple Linear Non- Linear Non- Linear

22. yx  01 Linear Regression Model Relationship between variables is a linear function Dependent (Response) Variable Independent (Explanatory) Variable Population Slope Population y-intercept Random Error

23. y E(y) = β 0 + β 1 x (line of means) β 0 = y-intercept x Change in y Change in x β 1 = Slope Line of Means

24. Population & Sample Regression Models Population $ $ $ $ $ Unknown Relationship Random Sample $ $

25. y x Population Linear Regression Model Observed value  i = Random error

26. y x Sample Linear Regression Model Unsampled observation  i = Random error Observed value ^

27. Estimating Parameters: Least Squares Method

28. Regression Modeling Steps 1.Hypothesize deterministic component 2.Estimate unknown model parameters 3.Specify probability distribution of random error term Estimate standard deviation of error 4.Evaluate model 5.Use model for prediction and estimation

29. Scattergram 1.Plot of all (x i, y i ) pairs 2.Suggests how well model will fit 0 20 40 60 0204060 x y

30. 0 20 40 60 0204060 x y Thinking Challenge How would you draw a line through the points? How do you determine which line ‘fits best’?

31. Least Squares ‘Best fit’ means difference between actual y values and predicted y values are a minimum – But positive differences off-set negative Least Squares minimizes the Sum of the Squared Differences (SSE)

32. Least Squares Graphically  2 y x  1  3  4 ^ ^ ^ ^

33. Coefficient Equations Slope y-intercept Prediction Equation

34. Computation Table xixi 2 x1x1 2 x2x2 2 :: xnxn 2 ΣxiΣxi 2 yiyi y1y1 y2y2 : ynyn ΣyiΣyi xixi x1x1 x2x2 : xnxn ΣxiΣxi 2 2 2 2 2 yiyi y1y1 y2y2 : ynyn ΣyiΣyi xiyixiyi ΣxiyiΣxiyi x1y1x1y1 x2y2x2y2 xnynxnyn

35. Interpretation of Coefficients ^ ^^ 1.Slope (  1 ) Estimated y changes by  1 for each 1unit increase in x —If  1 = 2, then Sales (y) is expected to increase by 2 for each 1 unit increase in Advertising (x) 2.Y-Intercept (  0 ) Average value of y when x = 0 —If  0 = 4, then Average Sales (y) is expected to be 4 when Advertising (x) is 0^^

36. Least Squares Example You’re a marketing analyst for Hasbro Toys. You gather the following data: Ad $Sales (Units) 11 21 32 42 54 Find the least squares line relating sales and advertising.

37. 0 1 2 3 4 012345 Scattergram Sales vs. Advertising Sales Advertising

38. Parameter Estimation Solution Table 22 11111 21412 32946 421648 54251620 1510552637 xixi yiyi xixi yiyi xiyixiyi

39. Parameter Estimation Solution

40. Parameter Estimates Parameter Standard T for H0: Variable DF Estimate Error Param=0 Prob>|T| INTERCEP 1 -0.1000 0.6350 -0.157 0.8849 ADVERT 1 0.7000 0.1914 3.656 0.0354 Parameter Estimation Computer Output 00 ^ 11 ^

41. Coefficient Interpretation Solution 1.Slope (  1 ) Sales Volume (y) is expected to increase by.7 units for each $1 increase in Advertising (x) ^ 2.Y-Intercept (  0 ) Average value of Sales Volume (y) is -.10 units when Advertising (x) is 0 — Difficult to explain to marketing manager — Expect some sales without advertising^

42. 0 1 2 3 4 012345 Regression Line Fitted to the Data Sales Advertising

43. Least Squares Thinking Challenge You’re an economist for the county cooperative. You gather the following data: Fertilizer (lb.)Yield (lb.) 43.0 65.5 106.5 129.0 Find the least squares line relating crop yield and fertilizer. © 1984-1994 T/Maker Co.

44. 0 2 4 6 8 10 05 15 Scattergram Crop Yield vs. Fertilizer* Yield (lb.) Fertilizer (lb.)

45. Parameter Estimation Solution Table* 43.0 16 9.00 12 65.5 3630.25 33 106.510042.25 65 129.014481.00108 3224.0296162.50218 xixi 2 yiyi xixi yiyi xiyixiyi 2

46. Parameter Estimation Solution*

47. Coefficient Interpretation Solution* 2.Y-Intercept (  0 ) Average Crop Yield (y) is expected to be 0.8 lb. when no Fertilizer (x) is used ^ ^ 1.Slope (  1 ) Crop Yield (y) is expected to increase by.65 lb. for each 1 lb. increase in Fertilizer (x)

48. Regression Line Fitted to the Data* 0 2 4 6 8 10 05 15 Yield (lb.) Fertilizer (lb.)

49. Probability Distribution of Random Error

50. Regression Modeling Steps 1.Hypothesize deterministic component 2.Estimate unknown model parameters 3.Specify probability distribution of random error term Estimate standard deviation of error 4.Evaluate model 5.Use model for prediction and estimation

51. Linear Regression Assumptions 1.Mean of probability distribution of error, ε, is 0 2.Probability distribution of error has constant variance 3.Probability distribution of error, ε, is normal 4.Errors are independent

52. Error Probability Distribution x1x1 x2x2 x3x3 y E(y) = β 0 + β 1 x x

53. Random Error Variation Measured by standard error of regression model – Sample standard deviation of  : s ^ Affects several factors –Parameter significance –Prediction accuracy ^ Variation of actual y from predicted y, y

54. y x xixi Variation Measures yiyi Unexplained sum of squares Total sum of squares Explained sum of squares y

55. Estimation of σ 2

56. Calculating SSE, s 2, s Example You’re a marketing analyst for Hasbro Toys. You gather the following data: Ad $Sales (Units) 11 21 32 42 54 Find SSE, s 2, and s.

57. Calculating SSE Solution 11 21 32 42 54 xixi yiyi.6 1.3 2 2.7 3.4.4 -.3 0 -.7.6.16.09 0.49.36 SSE=1.1

58. Calculating s 2 and s Solution

59. Residual Analysis The residual for observation i, e i, is the difference between its observed and predicted value Check the assumptions of regression by examining the residuals –Examine for linearity assumption –Evaluate independence assumption –Evaluate normal distribution assumption –Examine for constant variance for all levels of X (homoscedasticity)

60. Residual Analysis for Linearity Not Linear Linear x residuals x Y x Y x

61. Residual Analysis for Independence Not Independent Independent X X residuals X

62. Checking for Normality Examine the Stem-and-Leaf Display of the Residuals Examine the Boxplot of the Residuals Examine the Histogram of the Residuals Construct a Normal Probability Plot of the Residuals

63. Residual Analysis for Normality Percent Residual When using a normal probability plot, normal errors will approximately display in a straight line -3 -2 -1 0 1 2 3 0 100

64. Residual Analysis for Equal Variance Non-constant variance Constant variance xx Y x x Y residuals

65. Simple Linear Regression Example: Excel Residual Output RESIDUAL OUTPUT Predicted House PriceResiduals 1251.92316-6.923162 2273.8767138.12329 3284.85348-5.853484 4304.062843.937162 5218.99284-19.99284 6268.38832-49.38832 7356.2025148.79749 8367.17929-43.17929 9254.667464.33264 10284.85348-29.85348 Does not appear to violate any regression assumptions

66. Evaluating the Model Testing for Significance

67. Regression Modeling Steps 1.Hypothesize deterministic component 2.Estimate unknown model parameters 3.Specify probability distribution of random error term Estimate standard deviation of error 4.Evaluate model 5.Use model for prediction and estimation

68. Test of Slope Coefficient Shows if there is a linear relationship between x and y Involves population slope  1 Hypotheses –H 0 :  1 = 0 (No Linear Relationship) –H a :  1  0 (Linear Relationship) Theoretical basis is sampling distribution of slope

69. y Population Line x Sample 1 Line Sample 2 Line Sampling Distribution of Sample Slopes  1 Sampling Distribution 1111 1111 S ^ ^ All Possible Sample Slopes Sample 1:2.5 Sample 2:1.6 Sample 3:1.8 Sample 4:2.1 : : Very large number of sample slopes

70. Slope Coefficient Test Statistic

71. Test of Slope Coefficient Example You’re a marketing analyst for Hasbro Toys. You find β 0 = –.1, β 1 =.7 and s =.6055. Ad $Sales (Units) 11 21 32 42 54 Is the relationship significant at the.05 level of significance? ^ ^

72. Solution Table xixi yiyi xixi 2 yiyi 2 xiyixiyi 11111 21412 32946 421648 54251620 1510552637

73. Test Statistic Solution

74. Test of Slope Coefficient Solution H 0 :  1 = 0 H a :  1  0  .05 df  5 - 2 = 3 Critical Value(s): t 03.182-3.182.025 Reject H 0.025

75. Test of Slope Coefficient Solution Test Statistic: Decision: Reject at  =.05 Conclusion: There is evidence of a relationship

76. Test of Slope Coefficient Computer Output Parameter Estimates Parameter Standard T for H0: Variable DF Estimate Error Param=0 Prob>|T| INTERCEP 1 -0.1000 0.6350 -0.157 0.8849 ADVERT 1 0.7000 0.1914 3.656 0.0354 t =  1 / S  P-Value SS 11 1 1 ^ ^ ^ ^

77. Correlation Models

78. Types of Probabilistic Models Probabilistic Models Regression Models Correlation Models

79. Answers ‘How strong is the linear relationship between two variables?’ Coefficient of correlation –Sample correlation coefficient denoted r –Values range from –1 to +1 –Measures degree of association –Does not indicate cause–effect relationship

80. Coefficient of Correlation where

81. Coefficient of Correlation Values –1.0+1.00–.5+.5 No Linear Correlation Perfect Negative Correlation Perfect Positive Correlation Increasing degree of positive correlation Increasing degree of negative correlation

82. Coefficient of Correlation Example You’re a marketing analyst for Hasbro Toys. Ad $Sales (Units) 11 21 32 42 54 Calculate the coefficient of correlation.

83. Solution Table xixi yiyi xixi 2 yiyi 2 xiyixiyi 11111 21412 32946 421648 54251620 1510552637

84. Coefficient of Correlation Solution

85. Coefficient of Correlation Thinking Challenge You’re an economist for the county cooperative. You gather the following data: Fertilizer (lb.)Yield (lb.) 43.0 65.5 106.5 129.0 Find the coefficient of correlation. © 1984-1994 T/Maker Co.

86. Solution Table* 43.0 16 9.00 12 65.5 3630.25 33 106.510042.25 65 129.014481.00108 3224.0296162.50218 xixi 2 yiyi xixi yiyi xiyixiyi 2

87. Coefficient of Correlation Solution*

88. Proportion of variation ‘explained’ by relationship between x and y Coefficient of Determination 0  r 2  1 r 2 = (coefficient of correlation) 2

89. r 2 = 1 Examples of Approximate r 2 Values Y X Y X r 2 = 1 Perfect linear relationship between X and Y: 100% of the variation in Y is explained by variation in X

90. Examples of Approximate r 2 Values Y X Y X 0 < r 2 < 1 Weaker linear relationships between X and Y: Some but not all of the variation in Y is explained by variation in X

91. Examples of Approximate r 2 Values r 2 = 0 No linear relationship between X and Y: The value of Y does not depend on X. (None of the variation in Y is explained by variation in X) Y X r 2 = 0

92. Coefficient of Determination Example You’re a marketing analyst for Hasbro Toys. You know r =.904. Ad $Sales (Units) 11 21 32 42 54 Calculate and interpret the coefficient of determination.

93. Coefficient of Determination Solution r 2 = (coefficient of correlation) 2 r 2 = (.904) 2 r 2 =.817 Interpretation: About 81.7% of the sample variation in Sales (y) can be explained by using Ad $ (x) to predict Sales (y) in the linear model.

94. r 2 Computer Output Root MSE 0.60553 R-square 0.8167 Dep Mean 2.00000 Adj R-sq 0.7556 C.V. 30.27650 r 2 adjusted for number of explanatory variables & sample size r2r2

95. Using the Model for Prediction & Estimation

96. Regression Modeling Steps 1.Hypothesize deterministic component 2.Estimate unknown model parameters 3.Specify probability distribution of random error term Estimate standard deviation of error 4.Evaluate model 5.Use model for prediction and estimation

97. Prediction With Regression Models Types of predictions –Point estimates –Interval estimates What is predicted –Population mean response E(y) for given x  Point on population regression line –Individual response (y i ) for given x

98. What Is Predicted Mean y, E(y) y ^ y Individual Prediction, y E(y) =      x ^ x xPxP ^ ^ y i =      x

99. Confidence Interval Estimate for Mean Value of y at x = x p df = n – 2

100. Factors Affecting Interval Width 1.Level of confidence (1 –  ) Width increases as confidence increases 2.Data dispersion (s) Width increases as variation increases 3.Sample size Width decreases as sample size increases 4.Distance of x p from mean  x Width increases as distance increases

101. Why Distance from Mean? Sample 2 Line y x x1x1 x2x2 y Sample 1 Line Greater dispersion than x 1 x

102. Confidence Interval Estimate Example You’re a marketing analyst for Hasbro Toys. You find β 0 = -.1, β 1 =.7 and s =.6055. Ad $Sales (Units) 11 21 32 42 54 Find a 95% confidence interval for the mean sales when advertising is $4. ^ ^

103. Solution Table x i y i x i 2 y i 2 x i y i 11111 21412 32946 421648 54251620 1510552637

104. Confidence Interval Estimate Solution x to be predicted

105. Prediction Interval of Individual Value of y at x = x p Note! df = n – 2

106. Why the Extra ‘S’? Expected (Mean) y y y we're trying to predict Prediction, y ^ x xpxp  E(y) =      x ^ ^ ^ y i =      x i

107. Prediction Interval Example You’re a marketing analyst for Hasbro Toys. You find β 0 = -.1, β 1 =.7 and s =.6055. Ad $Sales (Units) 11 21 32 42 54 Predict the sales when advertising is $4. Use a 95% prediction interval. ^ ^

108. Solution Table x i y i x i 2 y i 2 x i y i 11111 21412 32946 421648 54251620 1510552637

109. Prediction Interval Solution x to be predicted

110. Dep Var Pred Std Err Low95% Upp95% Low95% Upp95% Obs SALES Value Predict Mean Mean Predict Predict 1 1.000 0.600 0.469 -0.892 2.092 -1.837 3.037 2 1.000 1.300 0.332 0.244 2.355 -0.897 3.497 3 2.000 2.000 0.271 1.138 2.861 -0.111 4.111 4 2.000 2.700 0.332 1.644 3.755 0.502 4.897 5 4.000 3.400 0.469 1.907 4.892 0.962 5.837 Interval Estimate Computer Output Predicted y when x = 4 Confidence Interval SYSY ^ Prediction Interval

111. Confidence Intervals v. Prediction Intervals x y x ^ ^ ^ y i =      x i

112. Conclusion 1.Described the Linear Regression Model 2.Stated the Regression Modeling Steps 3.Explained Least Squares 4.Computed Regression Coefficients 5.Explained Correlation 6.Predicted Response Variable