1 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION In practice we deal with finite samples, not infinite ones. So why should we be interested in whether an estimator is consistent? Why are we interested in consistency?

2 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION We have already considered this issue in the previous sequence. One reason is that often, in practice, it is impossible to find an estimator that is unbiased for small samples. If you can find one that is at least consistent, that may be better than having no estimate at all. If no unbiased estimator exists, a consistent estimator may be preferred to an inconsistent one. Why are we interested in consistency?

3 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION If no unbiased estimator exists, a consistent estimator may be preferred to an inconsistent one. It is often not possible to determine the expectation of an estimator, but still possible to evaluate probability limits. A second reason is that often we are unable to say anything at all about the expectation of an estimator. The expected value rules are weak analytical instruments that can be applied in relatively simple contexts. Why are we interested in consistency?

4 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION We will discuss the first point further here. If we can show that an estimator is consistent, then we may be optimistic about its finite sample properties, whereas if the estimator is inconsistent, we know that for finite samples it will definitely be biased. Why are we interested in consistency? If no unbiased estimator exists, a consistent estimator may be preferred to an inconsistent one. If the estimator is consistent, it may have desirable finite sample properties. If the estimator is inconsistent, we know that for finite samples it will definitely be biased.

5 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION However, there are reasons for being cautious about preferring consistent estimators to inconsistent ones. First, a consistent estimator may be biased for finite samples. Why are we interested in consistency? If no unbiased estimator exists, a consistent estimator may be preferred to an inconsistent one. If the estimator is consistent, it may have desirable finite sample properties. If the estimator is inconsistent, we know that for finite samples it will definitely be biased. Reasons for caution A consistent estimator may be biased for finite samples.

6 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION Second, we are usually also interested in variances. If a consistent estimator has a larger variance than an inconsistent one, the latter might be preferable if judged by the mean square error or similar criterion that allows a trade-off between bias and variance. Why are we interested in consistency? If no unbiased estimator exists, a consistent estimator may be preferred to an inconsistent one. If the estimator is consistent, it may have desirable finite sample properties. If the estimator is inconsistent, we know that for finite samples it will definitely be biased. Reasons for caution A consistent estimator may be biased for finite samples. A consistent estimator may have a larger variance than an inconsistent one and be inferior using the mean square error or similar criterion.

7 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION (We saw in a previous sequence that a consistent estimator might be preferable to an unbiased estimator, using this criterion. The same is true of an inconsistent estimator.) Why are we interested in consistency? If no unbiased estimator exists, a consistent estimator may be preferred to an inconsistent one. If the estimator is consistent, it may have desirable finite sample properties. If the estimator is inconsistent, we know that for finite samples it will definitely be biased. Reasons for caution A consistent estimator may be biased for finite samples. A consistent estimator may have a larger variance than an inconsistent one and be inferior using the mean square error or similar criterion. An inconsistent estimator could even be preferable to an unbiased one.

8 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION How can you resolve these issues? Mathematically they are intractable, otherwise we would not have resorted to large sample analysis in the first place. Simulation These issues cannot be resolved using mathematical analysis

9 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION Simulation The answer is to conduct a simulation experiment, directly investigating the distributions of estimators under controlled conditions. These issues cannot be resolved using mathematical analysis Alternative technique: simulation

10 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION We will do this for the example presented at the end of the previous sequence. We will set  equal to 5, so the value of Y in any observation is 5 times the value of Z. Simulation Model: Estimator of  :

11 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION We will generate Z as a random variable with a normal distribution with mean 1 and variance Simulation Model: Estimator of  :

12 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION We will generate the measurement error w as a normally distributed random variable with zero mean and unit variance. Simulation Model: Estimator of  :

13 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION We then use Y/X as an estimator of . Simulation Model: Estimator of  :

14 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION We have already seen that the estimator is consistent. The question now is how well it performs in finite samples. Simulation Model: Estimator of  :

15 We will start by taking samples of size 25. The figure shows the distribution of a, the estimator of , for 10 million samples. ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION n = 25 n mean s.d million samples

16 Although the mode of the distribution Is lower than the true value, the estimator is upwards biased, the mean estimate in the 10 million samples being ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION n = 25 n mean s.d million samples

17 If we increase the sample size to 100 and then 400, the distribution of the estimates obtained with 10 million samples becomes less skewed and the bias diminishes. ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION n = 400 n = 100 n = 25 n mean s.d million samples

18 If we increase the sample size to 1,600, the distribution of the estimates obtained with 10 million samples is hardly skewed at all and the bias has almost vanished. The mean is ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION n mean s.d n = 1,600 n = 400 n = 100 n = million samples

19 We demonstrated analytically that the estimator is consistent, but this is a theoretical property relating to samples of infinite size. ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION n mean s.d n = 1,600 n = 400 n = 100 n = million samples

20 The simulation shows us that for sample size 1,600, the estimator is almost unbiased. However for smaller sample sizes the estimator is biased, especially when the sample size is as small as 25. ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION n mean s.d n = 1,600 n = 400 n = 100 n = million samples

Copyright Christopher Dougherty These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section R.14 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre Individuals studying econometrics on their own who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics or the University of London International Programmes distance learning course EC2020 Elements of Econometrics