1. The Multiple Regression Model
Chapter 7
The Multiple Regression Model
Undergraduated Econometrics
Page 1
Chapter 7: The Multiple Regression Model

2. 7.1 Model Specification and the Data
Chapter Contents
7.1 Model Specification and the Data
7.2 Estimating the Parameters of the Multiple Regression Model
7.3 Sampling properties of the Least Squares Estimator
7.4 Interval Estimation
7.5 Hypothesis Testing for a Single Coefficient
7.6 Measuring Goodness of Fit
Undergraduated Econometrics
Page 2
Chapter 7: The Multiple Regression Model

3. Model Specification and the Data
7.1
Model Specification and the Data
Undergraduated Econometrics
Page 3
Chapter 7: The Multiple Regression Model

4. 7.1
Model Specification and the Data
Let’s set up an economic model in which sales revenue depends on one or more explanatory variables
We initially hypothesize that sales revenue is linearly related to price and advertising expenditure
The economic model is:
Undergraduated Econometrics
Page 4
Chapter 7: The Multiple Regression Model

5. In most economic models there are two or more explanatory variables
7.1
Model Specification and the Data
In most economic models there are two or more explanatory variables
When we turn an economic model with more than one explanatory variable into its corresponding econometric model, we refer to it as a multiple regression model
Most of the results we developed for the simple regression model can be extended naturally to this general case
Undergraduated Econometrics
Page 5
Chapter 7: The Multiple Regression Model

6. 7.1
Model Specification and the Data
β2 = change in tr($1000) when p is increased by one unit ($1), and a is held constant
Undergraduated Econometrics
Page 6
Chapter 7: The Multiple Regression Model

7. 7.1
Model Specification and the Data
Similarly, β3 = change in tr ($1000) when the a is increased by one unit ($1000), and p is held constant
Undergraduated Econometrics
Page 7
Chapter 7: The Multiple Regression Model

8. The econometric model is:
7.1
Model Specification and the Data
The econometric model is:
To allow for a difference between observable sales revenue and the expected value of sales revenue, we add a random error term, e = tr - E(tr)
Undergraduated Econometrics
Page 8
Chapter 7: The Multiple Regression Model

9. 7.1
Model Specification and the Data
Table 7.1 Weekly Observations on Revenue, Price, and Advertising Expenditure for the Hamburger Chain
Week
Revenue (tr)
$1 000 units
Price (p) $
Advertising (a) 1
123.1
1.92
12.4 2
124.3
2.15
9.9 3
89.3
1.67
2.4 4
141.3
1.68
13.8 5
112.8
1.75
3.5 48
120.1
1.94
9.3 49
107.4
2.37
8.3 50
128.6
2.10
15.4 51
124.6
2.29
9.2 52
127.2
2.36
10.2
Undergraduated Econometrics
Page 9
Chapter 7: The Multiple Regression Model

10. FIGURE 7.1 The multiple regression plane
Model Specification and the Data
FIGURE 7.1 The multiple regression plane
Undergraduated Econometrics
Page 10
Chapter 7: The Multiple Regression Model

11. 7.1
Model Specification and the Data
7.1.2a
The General Model
In a general multiple regression model, a dependent variable y is related to a number of explanatory variables x2, x3, …, xK through a linear equation that can be written as:
Undergraduated Econometrics
Page 11
Chapter 7: The Multiple Regression Model

12. The parameter β1 is the intercept term.
7.1
Model Specification and the Data
7.1.2a
The General Model
A single parameter, call it βk, measures the effect of a change in the variable xk upon the expected value of y, all other variables held constant
The parameter β1 is the intercept term.
We can think of it as being attached to a variable x1 that is always equal to 1
That is, x1 = 1
Undergraduated Econometrics
Page 12
Chapter 7: The Multiple Regression Model

13. 7.1
Model Specification and the Data
7.1.2a
The General Model
The equation for sales revenue can be viewed as a special case of Eq where K = 3, y = tr, x1 =1, x2 = p and x3 = a
Undergraduated Econometrics
Page 13
Chapter 7: The Multiple Regression Model

14. 7.1
Model Specification and the Data
7.1.2b
The Assumptions of the Model
MR1.
MR2.
MR3.
MR4.
MR5. The values of each xtk are not random and are not exact linear functions of the other explanatory variables
MR6.
Undergraduated Econometrics
Page 4
Page 14
Chapter 7: The Multiple Regression Model

15. Estimating the Parameters of the Multiple Regression Model
7.2
Estimating the Parameters of the Multiple Regression Model
Undergraduated Econometrics
Page 15
Chapter 7: The Multiple Regression Model

16. 7.2
Estimating the Parameters of the Multiple Regression Model
7.1.2a
The General Model
We will discuss estimation in the context of the model in Eq , which we repeat here for convenience.
This model is simpler than the full model, yet all the results we present carry over to the general case with only minor modifications
Undergraduated Econometrics
Page 4
Page 16
Chapter 7: The Multiple Regression Model

17. 7.2
Estimating the Parameters of the Multiple Regression Model
7.1.2a
The General Model
Mathematically we minimize the sum of squares function S(β1, β2, β3), which is a function of the unknown parameters, given the data:
Undergraduated Econometrics
Page 17
Chapter 7: The Multiple Regression Model

18. 7.2
Estimating the Parameters of the Multiple Regression Model
7.1.2a
The General Model
The formulas in Eq are referred to as estimation rules or procedures, which are called the least squares estimators of the unknown parameters
In general, since their values are not known until the data are observed and the estimates calculated, the least squares estimators are random variables
Undergraduated Econometrics
Page 4
Page 18
Chapter 7: The Multiple Regression Model

19. Table 7.2 Least Squares Estimates for Sales Equation for Big Andy’s
Estimating the Parameters of the Multiple Regression Model
Table 7.2 Least Squares Estimates for Sales Equation for Big Andy’s
Burger Barn
7.2.2
Least Squares Estimates Using Hamburger Chain Data
Variable
Coefficient
Std.Error
t-Statistic
Prob C
0.0000
PRICE
0.0427
ADVERT
R-squared
Adjusted R-squared
S.E.of regression
Sum squared resid.
Undergraduated Econometrics
Page 19
Chapter 7: The Multiple Regression Model

20. Interpretations of the results:
7.2
Estimating the Parameters of the Multiple Regression Model
7.2.2
Least Squares Estimates Using Hamburger Chain Data
Interpretations of the results:
The negative coefficient of pt suggests that demand is price elastic; we estimate that, with advertising held constant, an increase in price of $1 will lead to a fall in weekly revenue of $6,642
The coefficient on advertising is positive; we estimate that with price held constant, an increase in advertising expenditure of $1,000 will lead to an increase in sales revenue of $2,984
Undergraduated Econometrics
Page 20
Chapter 7: The Multiple Regression Model

21. Interpretations of the results (Continued):
7.2
Estimating the Parameters of the Multiple Regression Model
7.2.2
Least Squares Estimates Using Hamburger Chain Data
Interpretations of the results (Continued):
The estimated intercept implies that if both price and advertising expenditure were zero the sales revenue would be $104,790
Clearly, this outcome is not possible; a zero price implies zero sales revenue
In this model, as in many others, it is important to recognize that the model is an approximation to reality in the region for which we have data
Including an intercept improves this approximation even when it is not directly interpretable
Undergraduated Econometrics
Page 4
Page 21
Chapter 7: The Multiple Regression Model

22. 7.2
Estimating the Parameters of the Multiple Regression Model
7.2.2
Least Squares Estimates Using Hamburger Chain Data
Using the model to predict sales if price is $2 and advertising expenditure is $10,000:
The predicted value of sales revenue for p and a is approximately $121,340.
Undergraduated Econometrics
Page 22
Chapter 7: The Multiple Regression Model

23. A word of caution is in order about interpreting regression results:
7.2
Estimating the Parameters of the Multiple Regression Model
7.2.2
Least Squares Estimates Using Hamburger Chain Data
A word of caution is in order about interpreting regression results:
The negative sign attached to price implies that reducing the price will increase sales revenue.
If taken literally, why should we not keep reducing the price to zero?
Obviously that would not keep increasing total revenue
This makes the following important point:
Estimated regression models describe the relationship between the economic variables for values similar to those found in the sample data
Extrapolating the results to extreme values is generally not a good idea
Predicting the value of the dependent variable for values of the explanatory variables far from the sample values invites disaster
Undergraduated Econometrics
Page 4
Page 23
Chapter 7: The Multiple Regression Model

24. We need to estimate the error variance, σ2 Recall that:
7.2
Estimating the Parameters of the Multiple Regression Model
7.2.3
Estimation of the Error Variance σ2
We need to estimate the error variance, σ2
Recall that:
But, the squared errors are unobservable, so we develop an estimator for σ2 based on the squares of the least squares residuals:
Undergraduated Econometrics
Page 24
Chapter 7: The Multiple Regression Model

25. An estimator for σ2 that uses the information from
7.2
Estimating the Parameters of the Multiple Regression Model
7.2.3
Estimation of the Error Variance σ2
An estimator for σ2 that uses the information from
and has good statistical properties is:
where K is the number of β parameters being estimated in the multiple regression model.
Undergraduated Econometrics
Page 25
Chapter 7: The Multiple Regression Model

26. For the hamburger chain example:
7.2
Estimating the Parameters of the Multiple Regression Model
7.2.3
Estimation of the Error Variance σ2
For the hamburger chain example:
Undergraduated Econometrics
Page 26
Chapter 7: The Multiple Regression Model

27. Sampling Properties of the Least Squares Estimators
7.3
Sampling Properties of the Least Squares Estimators
Undergraduated Econometrics
Page 27
Chapter 7: The Multiple Regression Model

28. THE GAUSS-MARKOV THEOREM
7.3
Sampling Properties of the Least Squares Estimators
THE GAUSS-MARKOV THEOREM
For the multiple regression model, if assumptions MR1–MR5 hold, then the least squares estimators are the best linear unbiased estimators (BLUE) of the parameters.
Undergraduated Econometrics
Page 28
Chapter 7: The Multiple Regression Model

29. What constitutes ‘‘large’’ is tricky
7.3
Sampling Properties of the Least Squares Estimators
If the errors are not normally distributed, then the least squares estimators are approximately normally distributed in large samples
What constitutes ‘‘large’’ is tricky
It depends on a number of factors specific to each application
Frequently, N – K = 50 will be large enough
Undergraduated Econometrics
Page 29
Chapter 7: The Multiple Regression Model

30. 7.3
Sampling Properties of the Least Squares Estimators
7.3.1
The Variances and Covariances of the Least Squares Estimators
We can show that:
where
Undergraduated Econometrics
Page 30
Chapter 7: The Multiple Regression Model

31. 7.3
Sampling Properties of the Least Squares Estimators
7.3.1
The Variances and Covariances of the Least Squares Estimators
We can see that:
Larger error variances 2 lead to larger variances of the least squares estimators
Larger sample sizes N imply smaller variances of the least squares estimators
More variation in an explanatory variable around its mean, leads to a smaller variance of the least squares estimator
A larger correlation between x2 and x3 leads to a larger variance of b2
Undergraduated Econometrics
Page 31
Chapter 7: The Multiple Regression Model

32. We can arrange the variances and covariances in a matrix format:
7.3
Sampling Properties of the Least Squares Estimators
7.3.1
The Variances and Covariances of the Least Squares Estimators
We can arrange the variances and covariances in a matrix format:
Undergraduated Econometrics
Page 32
Chapter 7: The Multiple Regression Model

33. Consider the general for of a multiple regression model:
7.3
Sampling Properties of the Least Squares Estimators
7.3.2
The Properties of the Least Squares Estimators Assuming Normally Distributed Errors
Consider the general for of a multiple regression model:
If we add assumption MR6, that the random errors ei have normal probability distributions, then the dependent variable yi is normally distributed:
Undergraduated Econometrics
Page 33
Chapter 7: The Multiple Regression Model

34. 7.3
Sampling Properties of the Least Squares Estimators
7.3.2
The Properties of the Least Squares Estimators Assuming Normally Distributed Errors
Since the least squares estimators are linear functions of dependent variables, it follows that the least squares estimators are also normally distributed:
Undergraduated Econometrics
Page 34
Chapter 7: The Multiple Regression Model

35. We can now form the standard normal variable Z:
7.3
Sampling Properties of the Least Squares Estimators
7.3.2
The Properties of the Least Squares Estimators Assuming Normally Distributed Errors
We can now form the standard normal variable Z:
Undergraduated Econometrics
Page 35
Chapter 7: The Multiple Regression Model

36. Replacing the variance of bk with its estimate:
7.3
Sampling Properties of the Least Squares Estimators
7.3.2
The Properties of the Least Squares Estimators Assuming Normally Distributed Errors
Replacing the variance of bk with its estimate:
Notice that the number of degrees of freedom for t-statistics is N - K
Undergraduated Econometrics
Page 36
Chapter 7: The Multiple Regression Model

37. Consequently, we will usually express the t random variable as
7.3
Sampling Properties of the Least Squares Estimators
7.3.2
The Properties of the Least Squares Estimators Assuming Normally Distributed Errors
The square root of the variance estimator vâr(bk) is called the standard error of bk , which is written as
Consequently, we will usually express the t random variable as
Undergraduated Econometrics
Page 37
Chapter 7: The Multiple Regression Model

38. 7.4 Interval Estimation Undergraduated Econometrics Page 38
Chapter 7: The Multiple Regression Model

39. The interval endpoints
7.3
Interval Estimation
Interval estimates of unknown parameters are based on the probability statement that:
Rearranging, we get:
The interval endpoints
Undergraduated Econometrics
Page 39
Page 4
Chapter 7: The Multiple Regression Model

40. 7.3
Interval Estimation
Using our data, we have b2 = and se(b2) = Thus, our 95% interval estimate for β2 is given by (-13.06, -0.23)
This interval estimate suggests that decreasing price by $1 will lead to an increase in revenue somewhere between $230 and $13,060.
Undergraduated Econometrics
Page 40
Chapter 7: The Multiple Regression Model

41. Hypothesis Testing for a Single Coefficient
7.5
Hypothesis Testing for a Single Coefficient
Undergraduated Econometrics
Page 41
Chapter 7: The Multiple Regression Model

42. 7.5
Hypothesis Testing for a Single Coefficient
7.5.1
Testing the Significance of a Single Coefficient
We need to ask whether the data provide any evidence to suggest that y is related to each of the explanatory variables
If a given explanatory variable, say xk, has no bearing on y, then βk = 0
Testing this null hypothesis is sometimes called a test of significance for the explanatory variable xk
Undergraduated Econometrics
Page 42
Chapter 7: The Multiple Regression Model

43. Alternative hypothesis:
7.5
Hypothesis Testing for a Single Coefficient
7.5.1
Testing the Significance of a Single Coefficient
Null hypothesis:
Alternative hypothesis:
Test statistic:
Undergraduated Econometrics
Page 43
Chapter 7: The Multiple Regression Model

44. 7.5
Hypothesis Testing for a Single Coefficient
7.5.1
Testing the Significance of a Single Coefficient
For Bay Area Rapid Food example, following our standard testing format, whether revenue is related to price: 1. 2.
3. The test statistic, if the null hypothesis is true, is:
4. Using a 5% significance level (α=.05), and 49 degrees of freedom, the critical values that lead to a probability of in each tail of the distribution are:
Undergraduated Econometrics
Page 44
Chapter 7: The Multiple Regression Model

45. For Bay Area Rapid Food example(Continued) :
7.5
Hypothesis Testing for a Single Coefficient
7.5.1
Testing the Significance of a Single Coefficient
For Bay Area Rapid Food example(Continued) :
5. The computed value of the t-statistic is:
Since <-2.01, we reject H0: β2 = 0 and conclude that there is evidence from the data to suggest revenue depends on price.
Using the p-value to perform the test, we reject H0 because < 0.05
Undergraduated Econometrics
Page 45
Chapter 7: The Multiple Regression Model

46. For the demand elasticity, we wish to know if:
7.5
Hypothesis Testing for a Single Coefficient
We now are in a position to state the following questions as testable hypotheses and ask whether the hypotheses are compatible with the data
For the demand elasticity, we wish to know if:
β2 ≥ 0: a decrease in price leads to a decrease in sales revenue (demand is price-inelastic or has an elasticity of unity), or
β2 < 0: a decrease in price leads to a decrease in sales revenue (demand is price-inelastic)
7.5.2a
Testing for Elastic Demand
Undergraduated Econometrics
Page 46
Chapter 7: The Multiple Regression Model

47. 7.5
Hypothesis Testing for a Single Coefficient
7.5.2a
Testing for Elastic Demand
As before: 1. 2.
3. The test statistic, if the null hypothesis is true, is:
4. At a 5% significance level, we reject H0 if t ≤ tc
5. The test statistic is:
Since < -1.68, we reject H0: β2 ≥ 0 and conclude that H0: β2 < 0 (demand is elastic)
Undergraduated Econometrics
Page 47
Chapter 7: The Multiple Regression Model

48. 7.5
Hypothesis Testing for a Single Coefficient
7.5.2a
Testing Advertising Effectiveness
As before: 1. 2.
3. The test statistic, if the null hypothesis is true, is:
4. At a 5% significance level, we reject H0 if t ≥tc
5. The test statistic is:
Since 11.89> 1.68, we reject H0
Undergraduated Econometrics
Page 48
Chapter 7: The Multiple Regression Model

49. Measuring Goodness-of-fit
5.8
Measuring Goodness-of-fit
Undergraduated Econometrics
Page 49
Chapter 7: The Multiple Regression Model

50. 7.6
Measuring Goodness-of-fit
In the multiple regression model the R2 is relevant and the same formulas are valid, but now we talk of the proportion of variation in the dependent variable explained by all the explanatory variables included in the linear model
Undergraduated Econometrics
Page 50
Chapter 7: The Multiple Regression Model

51. The coefficient of determination is:
7.6
Measuring Goodness-of-fit
The coefficient of determination is: Eq
Undergraduated Econometrics
Page 51
Chapter 7: The Multiple Regression Model

52. For the Bay Area Rapid Food example:
7.6
Measuring Goodness-of-fit
For the Bay Area Rapid Food example:
Undergraduated Econometrics
Page 52
Chapter 7: The Multiple Regression Model

53. 7.6
Measuring Goodness-of-fit
Interpretation
86.7% of the variation in sales revenue is explained by the variation in price and by the variation in the level of advertising expenditure
In our sample, 13.3% of the variation in revenue is left unexplained and is due to variation in the error term or to variation in other variables that implicitly form part of the error term
Undergraduated Econometrics
Page 53
Chapter 7: The Multiple Regression Model

54. 7.6
Measuring Goodness-of-fit
If the model does not contain an intercept parameter, then the measure R2 given in Eq is no longer appropriate
The reason it is no longer appropriate is that without an intercept term in the model
so SST ≠ SSR + SSE
Undergraduated Econometrics
Page 54
Chapter 7: The Multiple Regression Model

55. Even if your computer displays one
7.6
Measuring Goodness-of-fit
Under these circumstances it does not make sense to talk of the proportion of total variation that is explained by the regression
When your model does not contain a constant, it is better not to report R2
Even if your computer displays one
Undergraduated Econometrics
Page 55
Chapter 7: The Multiple Regression Model