4. 6-1 Introduction To Empirical Models

5. 6-1 Introduction To Empirical Models

6. 6-1 Introduction To Empirical Models

7. 6-1 Introduction To Empirical Models
Based on the scatter diagram, it is probably reasonable to assume that the mean of the random variable Y is related to x by the following straight-line relationship:
where the slope and intercept of the line are called regression coefficients.
The simple linear regression model is given by
where  is the random error term.

8. 6-1 Introduction To Empirical Models
We think of the regression model as an empirical model.
Suppose that the mean and variance of  are 0 and 2, respectively, then
The variance of Y given x is

9. 6-1 Introduction To Empirical Models
The true regression model is a line of mean values:
where 1 can be interpreted as the change in the mean of Y for a unit change in x.
Also, the variability of Y at a particular value of x is determined by the error variance, 2.
This implies there is a distribution of Y-values at each x and that the variance of this distribution is the same at each x.

10. 6-1 Introduction To Empirical Models

11. 6-1 Introduction To Empirical Models
A Multiple Regression Model:

12. 6-1 Introduction To Empirical Models

13. 6-1 Introduction To Empirical Models

14. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation
The case of simple linear regression considers a single regressor or predictor x and a dependent or response variable Y.
The expected value of Y at each level of x is a random variable:
We assume that each observation, Y, can be described by the model

15. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation
Suppose that we have n pairs of observations (x1, y1), (x2, y2), …, (xn, yn).
The method of least squares is used to estimate the parameters, 0 and 1 by minimizing the sum of the squares of the vertical deviations in Figure 6-6.

16. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation
Using Equation 6-8, the n observations in the sample can be expressed as
The sum of the squares of the deviations of the observations from the true regression line is

17. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation

18. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation

19. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation

20. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation

21. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation
Notation

22. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation

23. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation

24. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation

25. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation

26. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation

27. 6-2 Simple Linear Regression
6-2.1 Least Squares Estimation

28. 6-2 Simple Linear Regression
Regression Assumptions and Model Properties

29. 6-2 Simple Linear Regression
Regression Assumptions and Model Properties

30. 6-2 Simple Linear Regression
Regression and Analysis of Variance

31. 6-2 Simple Linear Regression
Regression and Analysis of Variance

32. 6-2 Simple Linear Regression
6-2.2 Testing Hypothesis in Simple Linear Regression
Use of t-Tests
Suppose we wish to test
An appropriate test statistic would be

33. 6-2 Simple Linear Regression
6-2.2 Testing Hypothesis in Simple Linear Regression
Use of t-Tests
We would reject the null hypothesis if

34. 6-2 Simple Linear Regression
6-2.2 Testing Hypothesis in Simple Linear Regression
Use of t-Tests
Suppose we wish to test
An appropriate test statistic would be

35. 6-2 Simple Linear Regression
6-2.2 Testing Hypothesis in Simple Linear Regression
Use of t-Tests
We would reject the null hypothesis if

36. 6-2 Simple Linear Regression
6-2.2 Testing Hypothesis in Simple Linear Regression
Use of t-Tests
An important special case of the hypotheses of Equation 6-23 is
These hypotheses relate to the significance of regression.
Failure to reject H0 is equivalent to concluding that there is no linear relationship between x and Y.

37. 6-2 Simple Linear Regression
6-2.2 Testing Hypothesis in Simple Linear Regression
Use of t-Tests

38. 6-2 Simple Linear Regression
6-2.2 Testing Hypothesis in Simple Linear Regression
Use of t-Tests

39. 6-2 Simple Linear Regression
The Analysis of Variance Approach

40. 6-2 Simple Linear Regression
6-2.2 Testing Hypothesis in Simple Linear Regression
The Analysis of Variance Approach

41. 6-2 Simple Linear Regression
6-2.3 Confidence Intervals in Simple Linear Regression

42. 6-2 Simple Linear Regression
6-2.3 Confidence Intervals in Simple Linear Regression

43. 6-2 Simple Linear Regression

44. 6-2 Simple Linear Regression
6-2.4 Prediction of New Observations

45. 6-2 Simple Linear Regression
6-2.4 Prediction of New Observations

46. 6-2 Simple Linear Regression
6-2.5 Checking Model Adequacy
Fitting a regression model requires several assumptions.
Errors are uncorrelated random variables with mean zero;
Errors have constant variance; and,
Errors be normally distributed.
The analyst should always consider the validity of these assumptions to be doubtful and conduct analyses to examine the adequacy of the model

47. 6-2 Simple Linear Regression
6-2.5 Checking Model Adequacy
The residuals from a regression model are ei = yi - ŷi , where yi is an actual observation and ŷi is the corresponding fitted value from the regression model.
Analysis of the residuals is frequently helpful in checking the assumption that the errors are approximately normally distributed with constant variance, and in determining whether additional terms in the model would be useful.

48. 6-2 Simple Linear Regression
6-2.5 Checking Model Adequacy

49. 6-2 Simple Linear Regression
6-2.5 Checking Model Adequacy

50. 6-2 Simple Linear Regression
6-2.5 Checking Model Adequacy

51. 6-2 Simple Linear Regression
6-2.5 Checking Model Adequacy

52. 6-2 Simple Linear Regression
6-2.5 Checking Model Adequacy

53. 6-2 Simple Linear Regression
6-2.6 Correlation and Regression
The sample correlation coefficient between X and Y is

54. 6-2 Simple Linear Regression
6-2.6 Correlation and Regression
The sample correlation coefficient is also closely related to the slope in a linear regression model

55. 6-2 Simple Linear Regression
6-2.6 Correlation and Regression
It is often useful to test the hypotheses
The appropriate test statistic for these hypotheses is
Reject H0 if |t0| > t/2,n-2.

56. 6-3 Multiple Regression
6-3.1 Estimation of Parameters in Multiple Regression

57. 6-3 Multiple Regression
6-3.1 Estimation of Parameters in Multiple Regression
The least squares function is given by
The least squares estimates must satisfy

58. 6-3 Multiple Regression
6-3.1 Estimation of Parameters in Multiple Regression
The least squares normal equations are
The solution to the normal equations are the least squares estimators of the regression coefficients.

59. 6-3 Multiple Regression

60. 6-3 Multiple Regression

61. 6-3 Multiple Regression

62. 6-3 Multiple Regression

63. 6-3 Multiple Regression

64. 6-3 Multiple Regression
6-3.1 Estimation of Parameters in Multiple Regression

65. 6-3 Multiple Regression 6-3.2 Inferences in Multiple Regression
Test for Significance of Regression

66. 6-3 Multiple Regression 6-3.2 Inferences in Multiple Regression
Inference on Individual Regression Coefficients
This is called a partial or marginal test

67. 6-3 Multiple Regression 6-3.2 Inferences in Multiple Regression
Confidence Intervals on the Mean Response and Prediction Intervals

68. 6-3 Multiple Regression 6-3.2 Inferences in Multiple Regression
Confidence Intervals on the Mean Response and Prediction Intervals

69. 6-3 Multiple Regression 6-3.2 Inferences in Multiple Regression
Confidence Intervals on the Mean Response and Prediction Intervals

70. 6-3 Multiple Regression 6-3.2 Inferences in Multiple Regression
A Test for the Significance of a Group of Regressors

71. 6-3 Multiple Regression 6-3.3 Checking Model Adequacy
Residual Analysis

72. 6-3 Multiple Regression 6-3.3 Checking Model Adequacy
Residual Analysis

73. 6-3 Multiple Regression 6-3.3 Checking Model Adequacy
Residual Analysis

74. 6-3 Multiple Regression 6-3.3 Checking Model Adequacy
Residual Analysis

75. 6-3 Multiple Regression 6-3.3 Checking Model Adequacy
Residual Analysis

76. 6-3 Multiple Regression 6-3.3 Checking Model Adequacy
Influential Observations

77. 6-3 Multiple Regression 6-3.3 Checking Model Adequacy
Influential Observations

78. 6-3 Multiple Regression
6-3.3 Checking Model Adequacy

79. 6-3 Multiple Regression 6-3.3 Checking Model Adequacy
Multicollinearity

80. 6-4 Other Aspects of Regression
6-4.1 Polynomial Models

81. 6-4 Other Aspects of Regression
6-4.1 Polynomial Models

82. 6-4 Other Aspects of Regression
6-4.1 Polynomial Models

83. 6-4 Other Aspects of Regression
6-4.1 Polynomial Models

84. 6-4 Other Aspects of Regression
6-4.1 Polynomial Models

85. 6-4 Other Aspects of Regression
6-4.2 Categorical Regressors
Many problems may involve qualitative or categorical variables.
The usual method for the different levels of a qualitative variable is to use indicator variables.
For example, to introduce the effect of two different operators into a regression model, we could define an indicator variable as follows:

86. 6-4 Other Aspects of Regression
6-4.2 Categorical Regressors

87. 6-4 Other Aspects of Regression
6-4.2 Categorical Regressors

88. 6-4 Other Aspects of Regression
6-4.3 Variable Selection Procedures
Best Subsets Regressions

89. 6-4 Other Aspects of Regression
6-4.3 Variable Selection Procedures
Backward Elimination

90. 6-4 Other Aspects of Regression
6-4.3 Variable Selection Procedures
Forward Selection

91. 6-4 Other Aspects of Regression
6-4.3 Variable Selection Procedures
Stepwise Regression