Chapter 19: Unbiased estimators CIS 3033

19.1 Estimators A dataset is often modeled as a realization of a random sample from a probability distribution determined by one or more model parameters.   Let t = h(x1, . . . , xn) be an estimate of a parameter based on the dataset x1, . . . , xn only. Then t is a realization of the random variable  T = h(X1, . . .,Xn), which is called an estimator. Some examples are in Table 17.2.

19.1 Estimators Sometimes there can be more than one estimators, based on different analysis of the process. Example: Given a dataset x1, x2, ..., xn on the number of packages in a minute received at a network server, what is the probability for the server  to receive 0 package within a minute? Approach 1: as the relative frequency of the event Approach 2: as p(0) of a Poisson process

19.2 The behavior of an estimator Which of S and T is a better estimator for the probability of zero arrivals in a Pois(μ) distribution? When n = 30, neither S nor T can give the exact value of p0. To compare them, assume a parameter value (μ = ln 10, so that p0 = e−μ = 0.1), draw 30 numbers, and get S and T from them. Repeat the process 500 times, and get histograms of S and T.

19.2 The behavior of an estimator

19.3 Unbiasedness Let T = h(X1, . . . , Xn) be an estimator. The probability distribution of T is called the sampling distribution of T.   An estimator T is called an unbiased estimator for the parameter θ, if E[T] = θ irrespective of the value of θ. Otherwise T is biased, with E[T] − θ as its bias. For the previous example, S is unbiased, but T has a positive bias, which converges to 0 when n goes to infinity.

19.4 Unbiased estimators

19.4 Unbiased estimators When calculating sample variance, if the divisor n − 1 is replaced by n, the result will have a negative bias when used as estimator of the "true" variance of the distribution. If T is an unbiased estimator for a parameter θ, then g(T) does not have to be an unbiased estimator for g(θ). For example, sample standard derivation has a negative bias when used as an estimator of standard derivation.    An exception is if g(T) = aT + b.