1 Regression Econ 240A

2 Outline w A cognitive device to help understand the formulas for estimating the slope and the intercept, as well as the analysis of variance w Table of Analysis of Variance (ANOVA) for regression w F distribution for testing the significance of the regression, i.e. does the independent variable, x, significantly explain the dependent variable, y?

3 Outline (Cont.) w The Coefficient of Determination, R 2, and the Coefficient of Correlation, r.  Estimate of the error variance,  2. w Hypothesis tests on the slope, b.

4 Part I: A Cognitive Device

5 A Cognitive Device: The Conceptual Model w (1) y i = a + b*x i + e i w Take expectations, E: w (2) E y i = a + b*E x i +E e i, where (3) E e i =0 w Subtract (2) from (1) to obtain model in deviations: w (4) [y i - E y i ] = b*[x i - E x i ] + e i w Multiply (3) by [x i - E x i ] and take expectations:

6 A Cognitive Device: (Cont.) w (5) E{[y i - E y i ] [x i - E x i ]} = b*E[x i - E x i ] 2 + E{e i [x i - E x i ] }, where E{e i [x i - E x i ] }=0 w By definition, (6) cov yx = b* var x, i.e. w (7) b= cov yx/ var x w The corresponding empirical estimate:

7 A Cognitive Device (Cont.) w The empirical counter part to (2) w Square both sides of (4), and take expectations, w (10) E [y i - E y i ] 2 = b 2 *E[x i - E x i ] 2 + 2E{e i *[x i - E x i ]}+ E[e i ] 2 w Where (11) E{e i *[x i - E x i ] = 0, i.e. the explanatory variable x and the error e are assumed to be independent, cov ex = 0

8 A Cognitive Device (Cont.) w From (10) by definition w (11) var y = b 2 * var x + var e, this is the partition of the total variance in y into the variance explained by x, b 2 * var x, and the unexplained or error variance, var e. w the empirical counterpart to (11) is the total sum of squares equals the explained sum of squares plus the unexplained sum of squares:

9 Part II: ANOVA in Regression

10 ANOVA w Testing the significance of the regression, i.e. does x significantly explain y? w F 1, n -2 = EMS/UMS w Distributed with the F distribution with 1 degree of freedom in the numerator and n-2 degrees of freedom in the denominator

11 Table of Analysis of Variance (ANOVA) F 1,n -2 = Explained Mean Square / Error Mean Square

12 Example from Lab Four w Linear Trend Model for UC Budget

Time index, t = 0 for , t=1 for etc.

15 Example from Lab Four w Exponential trend model for UC Budget w UCBud(t) =exp[a+b*t+e(t)] w taking the logarithms of both sides w ln UCBud(t) = a + b*t +e(t)

ln = 5.83 Time index, t = 0 for , t=1 for etc.

18 Part III: The F Distribution

19 The F Distribution  The density function of the F distribution: 1 and 2 are the numerator and denominator degrees of freedom. ! ! !

20  This density function generates a rich family of distributions, depending on the values of 1 and 2 The F Distribution 1 = 5, 2 = 10 1 = 50, 2 = 10 1 = 5, 2 = 10 1 = 5, 2 = 1

21 Determining Values of F w The values of the F variable can be found in the F table, Table 6(a) in Appendix B for a type I error of 5%, or Excel.  The entries in the table are the values of the F variable of the right hand tail probability (A), for which P(F 1, 2 >F A ) = A.

22 Part IV: The Pearson Coefficient of Correlation, r w The Pearson coefficient of correlation, r, is (13) r = cov yx/[var x] 1/2 [var y] 1/2 w Estimated counterpart w Comparing (13) to (7) note that (15) r*{[var y] 1/2 /[var x] 1/2 } = b

23 Part IV (Cont.) The coefficient of Determination, R 2 w For a bivariate regression of y on a single explanatory variable, x, R 2 = r 2, i.e. the coefficient of determination equals the square of the Pearson coefficient of correlation w Using (14) to square the estimate of r

24 Part IV (Cont.) w Using (8), (16) can be expressed as w And so w In general, including multivariate regression, the estimate of the coefficient of determination,, can be calculated from (21) =1 -USS/TSS.

25 Part IV (Cont.) w For the bivariate regression, the F-test can be calculated from F 1, n-2 = [(n-2)/1][ESS/TSS]/[USS/TSS] F 1, n-2 = [(n-2)/1][ESS/USS]=[(n-2)] w For a multivariate regression with k explanatory variables, the F-test can be calculated as F k, n-2 = [(n-2)/k][ESS/USS] F k, n-2 = [(n-2)/k]

F 1, 32 = (n-2)*[R 2 /(1 - R 2 ) = 32*(0.903/0.093) = 312

27 Part V:Estimate of the Error Variance  Var e i =   w Estimate is unexplained mean square, UMS w Standard error of the regression is

29 Part VI: Hypothesis Tests on the Slope w Hypotheses, H 0 : b=0; H A : b>0 w Test statistic: w Set probability for the type I error, say 5% w Note: for bivariate regression, the square of the t-statistic for the null that the slope is zero is the F-statistic

t = { ]/4.80 = 17.7 t 2 = F, i.e *17.67 = 312

31 Part VII: Student’s t-Distribution

32 The Student t Distribution w The Student t density function  is the parameter of the student t distribution E(t) = 0 V(t) =  (  – 2) (for n > 2)

33 The Student t Distribution = 3 = 10

34 Determining Student t Values w The student t distribution is used extensively in statistical inference. w Thus, it is important to determine values of t A associated with a given number of degrees of freedom. w We can do this using t tables, Table 4 Appendix B Excel

35 tAtA t.100 t.05 t.025 t.01 t.005 A=.05 A -t A The t distribution is symmetrical around 0 =1.812 =  The table provides the t values (t A ) for which P(t > t A ) = A Using the t Table tttt