Large Sample Theory EC 532 Burak Saltoğlu

Additional readings W Greene (Econometric Analysis), 2012. pg: 1106-1128 Hayashi (Econometrics), 2000

Outline Convergence in Probability Laws of Large Numbers Convergence of Functions Convergence in Distribution: Limiting Distributions Central Limit Theorems Asymtotic Distributions

Why large sample We have studied the finite (exact) distribution of OLS estimator and its associated tests. If the regressors are endogeneous (i.e. X and u are correlated) we won’t be able to handle the estimation. Rather than making assumptions on a sample of a given size, large sample theory makes assumptions on the stochastic processes that generates the sample.

Asymptotics What happens to a rv or a distribution as n tends to infinity. What is the approximate distribution under these limiting conditions. What is the rate of convergence to a limit. 2 Critical Laws of statistics are studied under large sample theory. Law of Large Numbers Central Limit Theorem

Convergence in Probability Definition : Let xn be a sequence random variable where n is sample size, the random variable xn converges in probability to a constant c if the values that the x may take that are not close to c become increasingly unlikely as n increases. If xn converges to c, then we say, All the mass of the probability distribution concentrates around c.

mean square Convergence Definition mean square convergence:

Convergence in Probability Definition : An estimator of a parameter is a consistent estimator iff

Almost sure convergence İntiutively: once the sequence gets closer to c it stays that way .

Almost sure convergence:alternative definition The random variable İs said to converge “almost surely” to c if and only if .

Law of large numbers: Strong versus Weak Law of Large Numbers Based on convergence in probability Strong form of large numbers Based on Almost sure convergence

Laws of Large Numbers Weak Law of Large Numbers: (Khinchine) Remarks: 1) No finite variance assumption . 2) Requires i.i.d sampling

Strong law of large numbers: Kolmogorov Remarks: İt is stronger because of in AS convergence iid’ness is not required . But with iid’ness we will obtain a convergence to a constant mean.

Convergence of Functions Theorem (Slutsky): For a continious function, g(xn) Using Slutsky theorem, we can write some rules of plim.

Convergence of Functions Rules for Probability Limits 1) For plimx=c and plimy=d i) plim (x+y) = c+d ii) plim xy=cd iii) plim (x/y)=c/d 2) For matrices X and Y with plimX=A and plimY=B i) plim X-1=A-1 ii) plim XY=AB

Convergence in Distribution: Limiting Distributions Definition 6: xn with cdf Fn(x) converges in distribution to a random variable with cdf, F(x) if then F(x) is the limiting distribution of xn and can be shown as

Example: Limiting Probability Distributions Student’s t Distribution: Given that Properties: (i) It is symmetric, positive everywhere and leptokurtic(has fatter tails than normal dist.) (ii) Only parameter is n, degrees of freedom. (iii)

Convergence in Distribution: Limiting Distributions Rules for Limiting Distributions: 1) If and plim yn=c, then 2) As a corrolary to Slutsky theorem, if and g(x) is a cont. function

Convergence in Distribution: Limiting Distributions 3) If yn has a limiting distribution and plim(xn-yn)=0 then xn has the same limiting distribution with yn

Central Limit Theorems Lindberg-Levy Central Limit Theorem: CLT states that any sampling distribution from any distribution would converge to normal distribution the mean of a sufficiently large number of independent random variables, each with finite mean and variance, will be approximately normally distributed

Example: distirbution of sample mean So X’s are said to be identically and independently distributed (iid) random variables. derive the distribution of

Central limit idea 27.10.2009

A simple monte carlo for CLT See Matlab: clt.m Let X1,X2…Xn are ~N(0,1), As E(Xbar)=mu ninfinity Stdev(Xbar)0

X=(x1) x1~N(0,1): E(Xbar)=-0.0182, sigma=0.98

CLT when n=5, aver(Xbar)=0.01, stdev=0.47

N=1000,s.aver=0.002, st.dev=0.031 27.10.2009

Stdev of sample average disappears 27.10.2009

End of the Lecture