Part 12: Asymptotics for the Regression Model 12-1/39 Econometrics I Professor William Greene Stern School of Business Department of Economics

Part 12: Asymptotics for the Regression Model 12-2/39 Econometrics I Part 12 – Asymptotics for the Regression Model

Part 12: Asymptotics for the Regression Model 12-3/39 Setting The least squares estimator is (XX) -1 Xy = (XX) -1  i x i y i =  + (XX) -1  i x i ε i So, it is a constant vector plus a sum of random variables. Our ‘finite sample’ results established the behavior of the sum according to the rules of statistics. The question for the present is how does this sum of random variables behave in large samples?

Part 12: Asymptotics for the Regression Model 12-4/39 Well Behaved Regressors A crucial assumption: Convergence of the moment matrix XX/n to a positive definite matrix of finite elements, Q What kind of data will satisfy this assumption? What won’t? Does stochastic vs. nonstochastic matter? Various conditions for “well behaved X”

Part 12: Asymptotics for the Regression Model 12-5/39 Probability Limit

Part 12: Asymptotics for the Regression Model 12-6/39 Mean Square Convergence E[b|X]=β for any X. Var[b|X]  0 for any specific well behaved X b converges in mean square to β

Part 12: Asymptotics for the Regression Model 12-7/39 Crucial Assumption of the Model

Part 12: Asymptotics for the Regression Model 12-8/39 Consistency of s 2

Part 12: Asymptotics for the Regression Model 12-9/39 Asymptotic Distribution

Part 12: Asymptotics for the Regression Model 12-10/39 Asymptotics

Part 12: Asymptotics for the Regression Model 12-11/39 Asymptotic Distributions  Finding the asymptotic distribution  b  β in probability. How to describe the distribution? Has no ‘limiting’ distribution  Variance  0; it is O(1/n)  Stabilize the variance? Var[  n b] ~ σ 2 Q -1 is O(1)  But, E[  n b]=  n β which diverges  n (b - β)  a random variable with finite mean and variance. (stabilizing transformation) b apx. β +1/  n times that random variable

Part 12: Asymptotics for the Regression Model 12-12/39 Limiting Distribution  n (b - β)=  n (X’X) -1 X’ ε =  n (X’X/n) -1 (X’ ε/ n) Limiting behavior is the same as that of  n Q -1 (X’ ε/ n) Q is a fixed matrix. Behavior depends on the random vector  n (X’ ε/ n)

Part 12: Asymptotics for the Regression Model 12-13/39 Limiting Normality

Part 12: Asymptotics for the Regression Model 12-14/39 Asymptotic Distribution

Part 12: Asymptotics for the Regression Model 12-15/39 Asymptotic Properties  Probability Limit and Consistency  Asymptotic Variance  Asymptotic Distribution

Part 12: Asymptotics for the Regression Model 12-16/39 Root n Consistency  How ‘fast’ does b  β?  Asy.Var[b] =σ 2 /n Q -1 is O(1/n) Convergence is at the rate of 1/  n  n b has variance of O(1)  Is there any other kind of convergence? x 1,…,x n = a sample from exponential population; min has variance O(1/n 2 ). This is ‘n – convergent’ Certain nonparametric estimators have variances that are O(1/n 2/3 ). Less than root n convergent. Kernel density estimators converge slower than  n

Part 12: Asymptotics for the Regression Model 12-17/39 Asymptotic Results  Distribution of b does not depend on normality of ε  Estimator of the asymptotic variance (σ 2 /n)Q -1 is (s 2 /n) (X’X/n) -1. (Degrees of freedom corrections are irrelevant but conventional.)  Slutsky theorem and the delta method apply to functions of b.

Part 12: Asymptotics for the Regression Model 12-18/39 Test Statistics We have established the asymptotic distribution of b. We now turn to the construction of test statistics. In particular, we based tests on the Wald statistic F[J,n-K] = (1/J)(Rb - q)’[R s 2 (XX) -1 R] -1 (Rb - q) This is the usual test statistic for testing linear hypotheses in the linear regression model, distributed exactly as F if the disturbances are normally distributed. We now obtain some general results that will let us construct test statistics in more general situations.

Part 12: Asymptotics for the Regression Model 12-19/39 Full Rank Quadratic Form A crucial distributional result (exact): If the random vector x has a K-variate normal distribution with mean vector  and covariance matrix , then the random variable W = (x -  )  -1 (x -  ) has a chi-squared distribution with K degrees of freedom. (See Section in the text.)

Part 12: Asymptotics for the Regression Model 12-20/39 Building the Wald Statistic-1 Suppose that the same normal distribution assumptions hold, but instead of the parameter matrix  we do the computation using a matrix S n which has the property plim S n = . The exact chi-squared result no longer holds, but the limiting distribution is the same as if the true  were used.

Part 12: Asymptotics for the Regression Model 12-21/39 Building the Wald Statistic-2 Suppose the statistic is computed not with an x that has an exact normal distribution, but with an x n which has a limiting normal distribution, but whose finite sample distribution might be something else. Our earlier results for functions of random variables give us the result (x n -  ) S n -1 (x n -  )   2 [K] (!!!)VVIR! Note that in fact, nothing in this relies on the normal distribution. What we used is consistency of a certain estimator (S n ) and the central limit theorem for x n.

Part 12: Asymptotics for the Regression Model 12-22/39 General Result for Wald Distance The Wald distance measure: If plim x n = , x n is asymptotically normally distributed with a mean of  and variance , and if S n is a consistent estimator of , then the Wald statistic, which is a generalized distance measure between x n converges to a chi- squared variate. (x n -  ) S n -1 (x n -  )   2 [K]

Part 12: Asymptotics for the Regression Model 12-23/39 The F Statistic An application: (Familiar) Suppose b n is the least squares estimator of  based on a sample of n observations. No assumption of normality of the disturbances or about nonstochastic regressors is made. The standard F statistic for testing the hypothesis H0: R  - q = 0 is F[J, n-K] = [(e*’e* - e’e)/J] / [e’e / (n-K)] where this is built of two sums of squared residuals. The statistic does not have an F distribution. How can we test the hypothesis?

Part 12: Asymptotics for the Regression Model 12-24/39 JF is a Wald Statistic F[J,n-K] = (1/J)  (Rb n - q)[R s 2 (XX) -1 R’] -1 (Rb n - q). Write m = (Rb n - q). Under the hypothesis, plim m=0.  n m  N[0, R(  2 /n)Q -1 R’] Estimate the variance with R(s 2 /n)(X’X/n) -1 R’] Then, (  n m )’ [Est.Var(  n m)] -1 (  n m ) fits exactly into the apparatus developed earlier. If plim b n = , plim s 2 =  2, and the other asymptotic results we developed for least squares hold, then JF[J,n-K]   2 [J].

Part 12: Asymptotics for the Regression Model 12-25/39 Application: Wald Tests read;nobs=27;nvar=10;names= Year, G, Pg, Y, Pnc, Puc, Ppt, Pd, Pn, Ps $

Part 12: Asymptotics for the Regression Model 12-26/39 Data Setup Create; G=log(G); Pg=log(PG); y=log(y); pnc=log(pnc); puc=log(puc); ppt=log(ppt); pd=log(pd); pn=log(pn); ps=log(ps); t=year-1960$ Namelist;X=one,y,pg,pnc,puc,ppt,pd,pn,ps,t$ Regress;lhs=g;rhs=X$

Part 12: Asymptotics for the Regression Model 12-27/39 Regression Model Based on the gasoline data: The regression equation is g =  1 +  2 y +  3 pg +  4 pnc +  5 puc +  6 ppt +  7 pd +  8 pn +  9 ps +  10 t +  All variables are logs of the raw variables, so that coefficients are elasticities. The new variable, t, is a time trend, 0,1,…,26, so that  10 is the autonomous yearly proportional growth in G.

Part 12: Asymptotics for the Regression Model 12-28/39 Least Squares Results | Ordinary least squares regression | | LHS=G Mean = | | Standard deviation = | | Model size Parameters = 10 | | Degrees of freedom = 17 | | Residuals Sum of squares = | | Standard error of e = | | Fit R-squared = | | Adjusted R-squared = | | Model test F[ 9, 17] (prob) = (.0000) | | Chi-sq [ 9] (prob) = (.0000) | |Variable | Coefficient | Standard Error |t-ratio |P[|T|>t] | Mean of X| Constant Y PG PNC PUC PPT PD PN PS T

Part 12: Asymptotics for the Regression Model 12-29/39 Covariance Matrix

Part 12: Asymptotics for the Regression Model 12-30/39 Linear Hypothesis H 0 : Aggregate price variables are not significant determinants of gasoline consumption H 0 : β 7 = β 8 = β 9 = 0 H 1 : At least one is nonzero

Part 12: Asymptotics for the Regression Model 12-31/39 Wald Test Matrix ; R = [0,0,0,0,0,0,1,0,0,0/ 0,0,0,0,0,0,0,1,0,0/ 0,0,0,0,0,0,0,0,1,0] ; q = [0 / 0 / 0 ] $ Matrix ; m = R*b - q ; Vm = R*Varb*R' ; List ; Wald = m' m $ Matrix WALD has 1 rows and 1 columns |

Part 12: Asymptotics for the Regression Model 12-32/39 Restricted Regression Compare Sums of Squares Regress; lhs=g;rhs=X; cls:pd=0,pn=0,ps=0$ | Linearly restricted regression | | Ordinary least squares regression | | LHS=G Mean = | | Standard deviation = | | Residuals Sum of squares = | | Standard error of e = E-01 | | Fit R-squared = | without restrictions | Adjusted R-squared = | | Model test F[ 6, 20] (prob) = (.0000) | | Restrictns. F[ 3, 17] (prob) = (.0000) | Note: J(=3)*F = Chi-Squared = from before | Not using OLS or no constant. Rsqd & F may be < 0. | | Note, with restrictions imposed, Rsqd may be < 0. | |Variable | Coefficient | Standard Error |t-ratio |P[|T|>t] | Mean of X| Constant Y PG PNC PUC PPT PD (Fixed Parameter) PN D (Fixed Parameter) PS D (Fixed Parameter) T

Part 12: Asymptotics for the Regression Model 12-33/39 Nonlinear Restrictions I am interested in testing the hypothesis that certain ratios of elasticities are equal. In particular,  1 =  4 /  5 -  7 /  8 = 0  2 =  4 /  5 -  9 /  8 = 0

Part 12: Asymptotics for the Regression Model 12-34/39 Setting Up the Wald Statistic To do the Wald test, I first need to estimate the asymptotic covariance matrix for the sample estimates of  1 and  2. After estimating the regression by least squares, the estimates are f 1 = b 4 /b 5 - b 7 /b 8 f 2 = b 4 /b 5 - b 9 /b 8. Then, using the delta method, I will estimate the asymptotic variances of f 1 and f 2 and the asymptotic covariance of f 1 and f 2. For this, write f 1 = f 1 (b), that is a function of the entire 10  1 coefficient vector. Then, I compute the 1  10 derivative vectors, d 1 =  f 1 (b)/  b and d 2 =  f 2 (b)/  b These vectors are d 1 = 0, 0, 0, 1/b 5, -b 4 /b 5 2, 0, -1/b 8, b 7 /b 8 2, 0, 0 d 2 = 0, 0, 0, 1/b 5, -b 4 /b 5 2, 0, 0, b 9 /b 8 2, -1/b 8, 0

Part 12: Asymptotics for the Regression Model 12-35/39 Wald Statistics Then, D = the 2  10 matrix with first row d 1 and second row d 2. The estimator of the asymptotic covariance matrix of [f 1,f 2 ] (a 2  1 column vector) is V = D  s 2 (XX) -1  D. Finally, the Wald test of the hypothesis that  = 0 is carried out by using the chi- squared statistic W = (f-0)V -1 (f-0). This is a chi-squared statistic with 2 degrees of freedom. The critical value from the chi-squared table is 5.99, so if my sample chi- squared statistic is greater than 5.99, I reject the hypothesis.

Part 12: Asymptotics for the Regression Model 12-36/39 Wald Test In the example below, to make this a little simpler, I computed the 10 variable regression, then extracted the 5  1 subvector of the coefficient vector c = (b 4,b 5,b 7,b 8,b 9 ) and its associated part of the 10  10 covariance matrix. Then, I manipulated this smaller set of values.

Part 12: Asymptotics for the Regression Model 12-37/39 Application of the Wald Statistic ? Extract subvector and submatrix for the test matrix;list ; c =b(4:9)]$ matrix;list ; vc=varb(4:9,4:9) ? Compute derivatives calc ;list ; g11=1/c(2); g12=-c(1)*g11*g11; g13=-1/c(4) ; g14=c(3)*g13*g13 ; g15=0 ; g21= g11 ; g22=g12 ; g23=0 ; g24=c(5)/c(4)^2 ; g25=-1/c(4)$ ? Move derivatives to matrix matrix;list; dfdc=[g11,g12,g13,g14,g15 / g21,g22,g23,g24,g25]$ ? Compute functions, then move to matrix and compute Wald statistic calc;list ; f1=c(1)/c(2) - c(3)/c(4) ; f2=c(1)/c(2) - c(5)/c(4) $ matrix ; list; f = [f1/f2]$ matrix ; list; vf=dfdc * vc * dfdc' $ matrix ; list ; wald = f' * * f$

Part 12: Asymptotics for the Regression Model 12-38/39 Computations Matrix C is 5 rows by 1 columns Matrix VC is 5 rows by 5 columns E E E E E E E E E E E E E E E E E E E E E E G11 = G12 = G13= G14 = G15 = G21 = G22 = G23 = 0 G24 = G25 = DFDC=[G11,G12,G13,G14,G15/G21,G22,G23,G24,G25] Matrix DFDC is 2 rows by 5 columns F1= E-01 F2= F=[F1/F2] VF=DFDC*VC*DFDC' Matrix VF is 2 rows by 2 columns WALD Matrix Result is 1 rows by 1 columns

Part 12: Asymptotics for the Regression Model 12-39/39 Noninvariance of the Wald Test