Chapter 7 Point Estimation of Parameters

Learning Objectives Explain the general concepts of estimating Explain important properties of point estimators Know how to construct point estimators using the method of maximum likelihood Understand the central limit theorem Explain the important role of the normal distribution

Statistical Inference Used to make decisions or to draw conclusions about a population Utilize the information contained in a sample from the population Divided into two major areas –parameter estimation –hypothesis testing Use sample data to compute a number Called a point estimate

Statistic and Sampling Distribution Obtain a point estimate of a population parameter Observations are random variables Any function of the observation, or any statistic, is also a random variable Sample mean and sample variance are statistics Has a probability distribution Call the probability distribution of a statistic a sampling distribution

Definition of the Point Estimate Suppose we need to estimate the mean of a single population by a sample mean –Population mean, , is the unknown parameter –Estimator of the unknown parameter is the sample mean – is a statistic and can take on any value Convenient to have a general symbol Symbols are used in parameter estimation –Unknown population parameter id denoted by –Point estimate of this parameter by –Point estimator is a statistic and is denoted by

General Concepts of Point Estimation Unbiased Estimator –Estimator should be “close” to the true value of the unknown parameter –Estimator is unbiased when its expected value is equal to the parameter of interest –Bias is zero Variance of a Point Estimator –Considering all unbiased estimators, the one with the smallest variance is called the minimum variance unbiased estimator (MVUE) –MVUE is most likely estimator that gives a close value to the true value of the parameter of interest

Standard Error Measure of precision can be indicated by the standard error Sampling from a normal distribution with mean  and variance  2 Distribution of is normal with mean  and variance  2 /n Standard error of Not know , we will substitute the s into the above equation

Mean Square Error (MSE) It is necessary to use a biased estimator Mean square error of the estimator can be used Mean square error of an estimator is difference between the estimator and the unknown parameter An estimator is an unbiased estimator –If the MSE of the estimator is equal to the variance of the estimator –Bias is equal to zero Eq.7-3

Relative Efficiency Suppose we have two estimators of a parameter with their corresponding mean square errors Defined as If this relative efficiency is less than 1 Conclude that the first estimator give us a more efficient estimator of the unknown parameter than the second estimator Smaller mean square error

Example Suppose we have a random sample of size 2n from a population denoted by X, and E(X)=  and V(X)=  2 Let be two estimators of  Which is the better estimator of ? Explain your choice.

Solution Expected values are and are unbiased estimators of  Variances are MSE Conclude that is the “better” estimator with the smaller variance

Methods of Point Estimation Definition of unbiasness and other properties do not provide any guidance about how good estimators can be obtained Discuss the method of maximum likelihood Estimator will be the value of the parameter that maximizes the probability of occurrence of the sample values

Definition Let X be a random variable with probability distribution f(x;  ) –  is a single unknown parameter –Let x 1, x 2, …, x n be the observed values in a random sample of size n –Then the likelihood function of the sample L(  )= f(x 1 :  ). f(x 2 :  ). … f(x n :  )

Sampling Distribution of Mean Sample mean is a statistic –Random variable that depends on the results obtained in each particular sample Employs a probability distribution Probability distribution of is a sampling distribution –Called sampling distribution of the mean

Sampling Distributions of Means Determine the sampling distribution of the sample mean Random sample of size n is taken from a normal population with mean  and variance  2 Each observation is a normally and independently distributed random variable with mean  and variance  2

Sampling Distributions of Means- Cont. By the reproductive property of the normal distribution X-bar has a normal distribution with mean Variance

Central Limit Theorem Sampling from an unknown probability distribution Sampling distribution of the sample mean will be approximately normal with mean  and variance  2 /n Limiting form of the distribution of Most useful theorems in statistics, called the central limit theorem If n  30, the normal approximation will be satisfactory regardless of the shape of the population

Two Independent Populations Consider a case in which we have two independent populations –First population with mean  1 and variance  2 1 and the second population with mean  2 and variance  2 2 –Both populations are normally distributed –Linear combinations of independent normal random variables follow a normal distribution Sampling distribution of is normal with mean and variance