1. Regression Analysis Part C Confidence Intervals and Hypothesis Testing
Read Chapters 3, 4 and 5
of Forecasting and Time Series, An Applied Approach.
L01C MGS Regression Inference

2. Regression Analysis Modules
Part A – Basic Model & Parameter Estimation
Part B – Calculation Procedures
Part C – Inference: Confidence Intervals & Hypothesis Testing
Part D – Goodness of Fit
Part E – Model Building
Part F – Transformed Variables
Part G – Standardized Variables
Part H – Dummy Variables
Part I – Eliminating Intercept
Part J - Outliers
Part K – Regression Example #1
Part L – Regression Example #2
Part N – Non-linear Regression
Part P – Non-linear Example R
L01C MGS Regression Inference
L01C MGS Regression Inference

3. Overview of Part L01C Confidence Intervals and Hypothesis Testing
For Yi prediction and Yi mean
Formulas for univariate and multivariate cases.
Example calculation: 1) Manual in Excel and 2) SPSS.
For Regression Coefficients, bi
Example calculation: 1) Data Analysis in Excel and 2) SPSS.
Hypothesis Testing
For Entire Regression Model, F-test
L01C MGS Regression Inference
L01C MGS Regression Inference

4. Underlying Statistical Theory
Underlying Statistical Theory Confidence Intervals and Hypothesis Testing
L01C MGS Regression Inference
L01C MGS Regression Inference

5. The Standard Error of a Regression Equation
The Standard Error of a Regression Equation single independent variable
where
Yi is the actually observed values of the dependent variable.
Yihat is the predicted value from the fitted regression equation.
p = 1 is the number of independent variables. k = p+1 = 2 for the number of parameters, b0, b1.
n is the sample size used when calculating s.
L01C MGS Regression Inference
L01C MGS Regression Inference

6. Confidence Interval for Individual Prediction
Confidence Interval for Individual Prediction single independent variable
where
f denotes the future (forecasted) or predicted value.
p = 1 is the number of independent variables. k = p+1 = 2 for the number of parameters, b0, b1.
n is the sample size used when calculating s.
1-a is the confidence level, typically So a/2 = .025.
L01C MGS Regression Inference
L01C MGS Regression Inference

7. Confidence Interval for Mean Prediction. single independent variable
Confidence Interval for Mean Prediction single independent variable (1 of 2)
where
f denotes the future (forecasted) or predicted value.
p = 1 is the number of independent variables. k = p+1 = 2 for the number of parameters, b0, b1.
n is the sample size used when calculating s.
m is the sample size that is going to be used to calculate the mean value.
1-a is the confidence level, typically So a/2 = .025.
L01C MGS Regression Inference
L01C MGS Regression Inference

8. Confidence Interval for Mean Prediction. single independent variable
Confidence Interval for Mean Prediction single independent variable (2 of 2)
When m=1, the CI for the mean becomes the CI for an individual Y.
When m = infinity, the CI for the mean become the CI for a general mean.
L01C MGS Regression Inference
L01C MGS Regression Inference

9. L01C MGS 8110 - Regression Inference

10. CI Manual Calculations single independent variable
L01C MGS Regression Inference
L01C MGS Regression Inference

11. CI Manual Calculations single independent variable
L01C MGS Regression Inference
L01C MGS Regression Inference

12. SPSS Data Analysis Calculations single independent variable
L01C MGS Regression Inference

13. SPSS Data Analysis Calculations. single independent variable
SPSS Data Analysis Calculations single independent variable (continued)
L01C MGS Regression Inference

14. The Standard Error of a Regression Equation multivariate case
where
Y is the actually observed values of the dependent variable, an [n x 1] matrix vector.
X is the actually observed values of the independent variable, an [n x 1] matrix vector.
b is the calculated regression parameters, a [k x 1] matrix. b=(X’X)-1(X’Y)
p is the number of independent variables. k=p+1 is the number of parameters, b0, b1, … bp.
n is the sample size used when calculating s.
L01C MGS Regression Inference
L01C MGS Regression Inference

15. Confidence Interval for Individual Predictions multivariate case
where
Xf is a matrix vector of specified values for the independent variables. X’f = [1 Xf,1, Xf,2, … Xf,p]
p is the number of independent variables. k = p+1 is the number of parameters, b0, b1, … bp.
n is the sample size used when calculating s.
1-a is the confidence level, typically So a/2 = .025.
L01C MGS Regression Inference
L01C MGS Regression Inference

16. Confidence Interval for Mean Predictions multivariate case
where
Xf is a matrix vector of specified values for the independent variables. X’f = [1 Xf,1, Xf,2, … Xf,p]
p is the number of independent variables. k = p+1 is the number of parameters, b0, b1, … bp.
n is the sample size used when calculating s.
1-a is the confidence level, typically So a/2 = .025.
L01C MGS Regression Inference
L01C MGS Regression Inference

17. CI Manual Calculations multivariate case
L01C MGS Regression Inference
L01C MGS Regression Inference

18. SPSS Data Analysis Calculations multivariate case
L01C MGS Regression Inference

19. SPSS Data Analysis Calculations multivariate case (continued)
L01C MGS Regression Inference

20. The Standard Error of a Regression Equation
where
Yi is the actually observed values of the dependent variable.
Yihat is the predicted value from the fitted regression equation.
p = 1 is the number of independent variables. k = p+1 = 2 for the number of parameters, b0, b1.
n is the sample size used when calculating s.
Review from previous slide.
L01C MGS Regression Inference
L01C MGS Regression Inference

21. Skip’s Quick and Dirty method to Estimate the Confidence Interval for a Regression Line.
Procedure:
Select a range of X values from Minimum X to Maximum X.
Calculate the corresponding predicted values for Y, Yhat.
Add and subtract 2 times the Standard Error for Regression to the predicted values.
Optional – plot the two CL line on the scatter plot.
L01C MGS Regression Inference
L01C MGS Regression Inference

22. Confidence Interval for. Regression Coefficients
Confidence Interval for Regression Coefficients single independent variable
where
p = 1 is the number of independent variables. k = p+1 = 2 for the number of parameters, b0, b1.
n is the sample size used when calculating s.
1-a is the confidence level, typically So a/2 = .025.
L01C MGS Regression Inference
L01C MGS Regression Inference

23. Confidence Interval for Regression Coefficients multivariate case
where
p is the number of independent variables. k = p+1 is the number of parameters, b0, b1, … bp.
n is the sample size used when calculating s.
1-a is the confidence level, typically So a/2 = .025.
L01C MGS Regression Inference
L01C MGS Regression Inference

24. Excel, Data Analysis Calculations Multivariate Case
L01C MGS Regression Inference

25. Excel, Data Analysis Calculations Multivariate Case (continued)
L01C MGS Regression Inference

26. SPSS Data Analysis Calculations Multivariate Case
L01C MGS Regression Inference

27. SPSS Data Analysis Calculations Multivariate Case (continued)
L01C MGS Regression Inference

28. Hypothesis Test of Regression Coefficient
where
p is the number of independent variables. k = p+1 is the number of parameters, b0, b1, … bp.
n is the sample size used when calculating s.
1-a is the confidence level, typically So a/2 = .025.
L01C MGS Regression Inference
L01C MGS Regression Inference

29. Excel, Data Analysis Calculation Multivariate Case
L01C MGS Regression Inference

30. SPSS Data Analysis Calculations Multivariate Case
L01C MGS Regression Inference

31. Summary: Never test the intercept (constant)
Summary: Never test the intercept (constant). Discussed in more detail in L01I If sig is less than .05, keep the variable (slope not equal to zero). If sig is greater than .05, consider eliminating the variable from the model (slope could be zero).
L01C MGS Regression Inference

32. Summary: Never test the intercept (constant). If sig is less than
Summary: Never test the intercept (constant). If sig is less than .05, keep the variable (slope not equal to zero). If sig is greater than .05, consider eliminating the variable from the model (slope could be zero).
If you can’t remember theses rules a year from now, look at the confidence interval. Does the confidence interval contain 0 (zero)
L01C MGS Regression Inference

33. F-test for Overall Model
L01C MGS Regression Inference
L01C MGS Regression Inference

34. Excel, Data Analysis Calculation Multivariate Case
L01C MGS Regression Inference

35. SPSS Data Analysis Calculations Multivariate Case
L01C MGS Regression Inference

36. Review of ANOVA Analysis
Green = Residual from mean.
Blue, dashed = portion of residual explained by regression equation.
Red = portion of residual still unexplained after fitting regression equation.
L01C MGS Regression Inference
L01C MGS Regression Inference

37. Fundamental Concept of ANOVA Analysis
Residual Analysis
Total = Unexplained Explained
It can be shown (algebraically complex)
Total SS = Unexplained SS + Explained SS
L01C MGS Regression Inference
L01C MGS Regression Inference

38. Review of ANOVA Table (1 of 3) Terminology and Table Calculations
L01C MGS Regression Inference
L01C MGS Regression Inference

39. Review of ANOVA Table (2 of 3) Algebraic explanation of terms
L01C MGS Regression Inference
L01C MGS Regression Inference

40. Review of ANOVA Table (3 of 3) Calculation formulas
L01C MGS Regression Inference
L01C MGS Regression Inference

41. Review of ANOVA Table. (1 of 3). Matrix explanation of terms
Review of ANOVA Table (1 of 3) Matrix explanation of terms Regression prediction compared to prediction mean of y
L01C MGS Regression Inference
L01C MGS Regression Inference

42. Review of ANOVA Table (2 of 3) Alternative matrix explanation of terms Regression prediction compared to prediction 0 (zero)
L01C MGS Regression Inference
L01C MGS Regression Inference

43. Review of ANOVA Table (3 of 3) Alternative matrix explanation of terms Regression prediction compared to prediction mean of Y & 0 (zero)
L01C MGS Regression Inference
L01C MGS Regression Inference

44. Statistical Assumptions
0. The expected value of the residuals is zero, E(ei)=0. The algebraic equation is the correct functional form and accurately predicts E(Yi,j) for all j.
Inference Assumptions
The residual variance is constant. That is, sj,j2 = s2 for all Xj,j and all i and j. The variance of the observations (Yi,j) does not change as more observations are obtained and/or as different values of Xj are observed.
The observations are statistically independent. That is, Yi,j is statistically independent of all other Y’,j values for all i (& j fixed). Knowing the current value of Y does not provide insights into the value of the next Y.
The residual errors are normally distributed. The ei,j terms are N(0,s2).
L01C MGS Regression Inference
L01C MGS Regression Inference