6 Point Estimation.

Slides:



Advertisements
Similar presentations
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 6 Point Estimation.
Advertisements

Point Estimation Notes of STAT 6205 by Dr. Fan.
CHAPTER 8 More About Estimation. 8.1 Bayesian Estimation In this chapter we introduce the concepts related to estimation and begin this by considering.
1 12. Principles of Parameter Estimation The purpose of this lecture is to illustrate the usefulness of the various concepts introduced and studied in.
Chapter 7. Statistical Estimation and Sampling Distributions
Chapter 7 Title and Outline 1 7 Sampling Distributions and Point Estimation of Parameters 7-1 Point Estimation 7-2 Sampling Distributions and the Central.
Statistical Estimation and Sampling Distributions
Estimation  Samples are collected to estimate characteristics of the population of particular interest. Parameter – numerical characteristic of the population.
Chap 8: Estimation of parameters & Fitting of Probability Distributions Section 6.1: INTRODUCTION Unknown parameter(s) values must be estimated before.
Copyright © Cengage Learning. All rights reserved.
Maximum likelihood (ML) and likelihood ratio (LR) test
Maximum likelihood Conditional distribution and likelihood Maximum likelihood estimations Information in the data and likelihood Observed and Fisher’s.
Statistical Inference Chapter 12/13. COMP 5340/6340 Statistical Inference2 Statistical Inference Given a sample of observations from a population, the.
Section 6.1 Let X 1, X 2, …, X n be a random sample from a distribution described by p.m.f./p.d.f. f(x ;  ) where the value of  is unknown; then  is.
Maximum likelihood (ML)
Continuous Random Variables and Probability Distributions
1 STATISTICAL INFERENCE PART I EXPONENTIAL FAMILY & POINT ESTIMATION.
Copyright © Cengage Learning. All rights reserved. 6 Point Estimation.
Maximum likelihood (ML)
Estimation Basic Concepts & Estimation of Proportions
Chapter 6. Point Estimation Weiqi Luo ( 骆伟祺 ) School of Software Sun Yat-Sen University : Office : # A313
STATISTICAL INFERENCE PART I POINT ESTIMATION
Prof. Dr. S. K. Bhattacharjee Department of Statistics University of Rajshahi.
Random Sampling, Point Estimation and Maximum Likelihood.
Chapter 7 Point Estimation
Chapter 5.6 From DeGroot & Schervish. Uniform Distribution.
1 Lecture 16: Point Estimation Concepts and Methods Devore, Ch
PROBABILITY AND STATISTICS FOR ENGINEERING Hossein Sameti Department of Computer Engineering Sharif University of Technology Principles of Parameter Estimation.
Chapter 7 Point Estimation of Parameters. Learning Objectives Explain the general concepts of estimating Explain important properties of point estimators.
CLASS: B.Sc.II PAPER-I ELEMENTRY INFERENCE. TESTING OF HYPOTHESIS.
Confidence Interval & Unbiased Estimator Review and Foreword.
Point Estimation of Parameters and Sampling Distributions Outlines:  Sampling Distributions and the central limit theorem  Point estimation  Methods.
Continuous Random Variables and Probability Distributions
Statistics Sampling Distributions and Point Estimation of Parameters Contents, figures, and exercises come from the textbook: Applied Statistics and Probability.
Copyright © Cengage Learning. All rights reserved. 5 Joint Probability Distributions and Random Samples.
Week 21 Order Statistics The order statistics of a set of random variables X 1, X 2,…, X n are the same random variables arranged in increasing order.
Chapter 8 Estimation ©. Estimator and Estimate estimator estimate An estimator of a population parameter is a random variable that depends on the sample.
Conditional Expectation
Week 21 Statistical Model A statistical model for some data is a set of distributions, one of which corresponds to the true unknown distribution that produced.
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 7 Inferences Concerning Means.
LEARNING OBJECTIVES After careful study of this chapter you should be able to do the following: 1.Explain the general concepts of estimating the parameters.
STA302/1001 week 11 Regression Models - Introduction In regression models, two types of variables that are studied:  A dependent variable, Y, also called.
Copyright © Cengage Learning. All rights reserved. 4 Continuous Random Variables and Probability Distributions.
Virtual University of Pakistan
Copyright © Cengage Learning. All rights reserved.
Statistical Estimation
Points and Interval Estimates
STATISTICS POINT ESTIMATION
STATISTICAL INFERENCE
12. Principles of Parameter Estimation
6 Point Estimation.
Some General Concepts of Point Estimation
Point and interval estimations of parameters of the normally up-diffused sign. Concept of statistical evaluation.
3. The X and Y samples are independent of one another.
Joint Probability Distributions and Random Samples
7-1 Introduction The field of statistical inference consists of those methods used to make decisions or to draw conclusions about a population. These.
STATISTICAL INFERENCE PART I POINT ESTIMATION
Parameter, Statistic and Random Samples
The Bernoulli distribution
Random Sampling Population Random sample: Statistics Point estimate
CONCEPTS OF ESTIMATION
POINT ESTIMATOR OF PARAMETERS
Computing and Statistical Data Analysis / Stat 7
Learning From Observed Data
STATISTICAL INFERENCE PART I POINT ESTIMATION
12. Principles of Parameter Estimation
Chapter 8 Estimation.
Statistical Model A statistical model for some data is a set of distributions, one of which corresponds to the true unknown distribution that produced.
Statistical Model A statistical model for some data is a set of distributions, one of which corresponds to the true unknown distribution that produced.
Applied Statistics and Probability for Engineers
Presentation transcript:

6 Point Estimation

Introduction Given a parameter of interest, such as a population mean μ or population proportion p, the objective point estimating is to use a sample to compute a number that represents in some sense a good guess for the true value of the parameter. The resulting number is called a point estimate. In Section 6.2, we describe and illustrate two important methods for obtaining point estimates: the method of moments and the method of maximum likelihood.

Some General Concepts of Point Estimation 6.1 Definition A point estimate of a parameter θ is a single number that can be regarded as a sensible value for θ . A point estimate is obtained by selecting a suitable statistic and computing its value from the given sample data. The selected statistic is called the point estimator of θ.

Example 6.1 An automobile manufacturer has developed a new type of bumper, which is supposed to absorb impacts with less damage than previous bumpers. The manufacturer has used this bumper in a sequence of 25 controlled crashes against a wall, each at 10 mph, using one of its compact car models. Let X= the number of crashes that result in no visible damage to the automobile. The parameter to be estimate is p= the proportion of all such crashes that result in no damage [alternatively, p=P( no damage in a single crash)].

If is observed to be x=15, the most reasonable estimator and estimate are If for each parameter of interest there were only one reasonable point estimator, there would not be much to point estimation. In most problems, though, there will be more than one reasonable estimator.

Unbiased Estimators Definition A point estimator is said to be an unbiased estimator of if for every possible value of . If is not unbiased, the difference is called the bias of

Thus , is unbiased if its probability distribution is always “centered” at the true value of the parameter. Figure 6.1 pictures the distribution of several biased and unbiased estimators. Note that “centered” here means that the expected value, not the median, of the distribution of is equal to θ pdf of pdf of pdf of pdf of θ θ Bias of θ1 Bias of θ1 Figure 6.1 the pdf’s of a biased estimator and an unbiased estimated for a parameter

PROPOSITION PROPOSITION When X is a binomial rv with parameters n and p, the sample proportion =X/n is an unbiased estimator of p. Please read the Example 6.4 yourself. PROPOSITION Let X1, X 2,…,Xn be a random sample from a distribution with mean μ and variance σ2. Then the estimator Is an an unbiased estimator of σ2 The proof is omitted, please read it yourself

Estimators with Minimum Variance Suppose and are two estimator of θthat are both unbiased.Then,although the distribution of each estimator is centered at the true value of θ,the spread of the distributions about the true value may be different. Principle of Minimum Variance Unbiased Estimation Among all estimators of θthat are unbiased, choose the one that has minimum variance.The resulting is called the minimum variance unbiased estimator (MVUE) of θ.

Example 6.5 We argued in Figure 6.4 that when X1 ,X2,…,Xn is a random sample from a uniform distribution on [0, θ],the estimator is unbiased for θ(we previously denoted this estimator by ). This is not the only unbiased estimator of θ. The expected value of a uniformly distributed rv is just the midpoint of the interval of positive density, so E(Xi)=θ/2.This implies that ,from which .That is ,the estimator is unbiased for θ.

THEOREM Let X1, X 2,…,Xn be a random sample from a distribution with parameters μand σ.Then the estimator is the MVUE μ. Whenever we are convinced that the population being sampled is normal,the result says that should be used to estimate μ.In Example 6.2,then,our estimate would be . In some situations,it is possible to obtain an estimator with small bias that would be preferred to the best unbiased estimator.This is illustrated in Figure 6.3.However,MVUEs are often easier to obtain than the type of biased estimator whose distribution is pictured.

pdf of ,a biased estimator pdf of , the MVUE θ Figure 6.3 A biased estimator that is preferable to the MVUE

6.2 Method of Point Estimation The definition of unbiasedness does not in general indicate how unbiased estimators can be derived. We now discuss two “constructive” methods for obtaining point estimators: the method of moments and the method of maximum likelihood.

By constructive we mean that the general definition of each type of estimator suggests explicitly how to obtain the estimator in any specific problem. Although maximum likelihood estimators are generally preferable to moment estimators because of certain efficiency properties, they often require significantly more computation than do moment estimators. It is sometimes the case that these methods yield unbiased estimators.

The Method of Moments DEFINITION Let X1, X 2,…,Xn be a random sample from a pmf or pdf f(x).For k=1,2,3,…., the kth population moment,or kth moment of the distribution f(x),is .The kth sample moment is DEFINITION Let X1, X 2,…,Xn be a random sample from a distribution with pmf or pdf f(x;θ1,…..,θm),where θ1,…..,θm are parameters whose values are unknown.Then the moment estimators are obtained by equating the first m sample moments to the corresponding first m population moments and solving for θ1,…..,θm

For population X k-origin moment k-central moment For sample X1,X2,…,Xn k-origin moment k-central moment

Suppose, that we wish to estimate parameters θ1,θ2,…,θm. Then

So the method of moments estimates are Where are estimation of parameters θ1,θ2,…,θm.

Which is ,therefore, the method of moments estimate of λ: . Example Poisson Distribution The first origin moment for the Poisson distribution is the parameter λ=E(X). Then first sample origin moment is Which is ,therefore, the method of moments estimate of λ: .

The first and second origin moment for the normal distribution are Example B Normal Distribution The first and second origin moment for the normal distribution are Therefore,

The corresponding estimates of μandσ2 from the sample origin moments are Where S is the sample standard deviation.

Example 6.11 Let X1, X 2,…,Xn represent a random sample of service times of n customers at a certain facility, where the underlying distribution is assumed exponential with parameter λ. Since there is only one parameter to be estimated, the estimator is obtained by equating E(X) to .Since E(X)=1/λ for an exponential distribution,this gives

The estimation of parameters of some common distributions

Maximum Likelihood Estimation The method of maximum likelihood was first introduced by R.A.Fisher, a geneticist and statistician, in the 1920s. Most statisticians recommend this method, at least when the sample is large, since the resulting estimators have certain desirable efficiency properties.

Example 6.14 A sample of ten new bike helmets manufactured by a certain company is obtained.Upon testing, it is found that the first, third, and tenth helmets are flawed, whereas the others are not. Let p=P(flawed helmet) and define X1,….,X10 by Xi=1 if the ith helmet is flawed and zero otherwise.Then the observed xi’s are 1,0,1,0,0,0,0,0,0,1, so the joint pmf of the sample is

We now ask: For what value of p is the observed sample most likely to have occurred? That is, we wish to find the value of p that maximizes the pmf (6.4), or, equivalently, Which is a differentiable function of p, equating the derivative of (6.5) to zero gives the maximizing value

where x is the observed number of successes(flawed helmets) where x is the observed number of successes(flawed helmets).The estimate of p is now . It is called the maximum likelihood estimate because for fixed x1,……,x10, it is the parameter value that maximizes the likelihood of the observed sample. Note that if we had been told only that among the ten helmets there were three that were flawed,Equation would be replaced by the binomial pmf ,which is also maximized for

DEFINITION Let X1, X 2,…,Xn have joint pmf or pdf where the parameters θ1,…..,θm have unknown values.When x1,……,xn are the observed sample values and is regarded as a function of θ1,…..,θm,it is called the likelihood function.The maximum likelihood estimates are those values of the θi’s that maximize the likelihood function,so that When the Xi’s are substituted in place of the xi’s, the maximum likelihood estimators result. for all θ1,…..,θm

Suppose, that we wish to estimate parameters θ1,θ2,…,θm. X1,X2,…,Xn are independent Let X1,X2,…,Xn are sample Suppose, that we wish to estimate parameters θ1,θ2,…,θm. Set ----------(1) likelihood function

----------(2) System of equations of likelihood Why? The solutions of the system (2) are the maximum likelihood estimator ----------(3) log-likelihood

Example A Poisson Distribution Likelihood function is The log-likelihood is

same Setting the first derivative of the log-likelihood equal to zero, we find so same

Example B Normal Distribution Likelihood function is

The log-likelihood is

Not the unbiased estimator so Not the unbiased estimator

Exercise Let the probability of a random variable X is Find the moment estimator and likelihood estimator of θ.

Example 6.15 Suppose X1, X2,…,Xn is a random sample from an exponential distribution with parameter λ.Because of independence,the likelihood function is a product of the individual pdf’s: The ln (likelihood) is

Equating (d/dλ) to zero results in Thus, the mle is It is identical to the method of moments estimator. Example 6.16 Example 6.17

Estimating Function of Parameters PROPOSITION The Invariance Principle Let θ1,….,θm be the mle’s of the parameters θ1,…,θm .Then the mle of any function h(θ1,….,θm ) of these parameters is the function of the mle’s.

In the normal case,the mle’s of μ and σ2 are Example 6.19 In the normal case,the mle’s of μ and σ2 are and .To obtain the mle of the function ,substitute the mle’s into the function: The mle of σis not the sample standard deviation S, though they are close unless n is quit small. Example 6.20 (omit)

A Desirable Property of the Maximum Likelihood Estimate Although the principle of maximum likelihood estimation has considerable intuitive appeal, the following proposition provides additional rationale for the use of mle’s.

PROPOSITION Under very general conditions on the joint distribution of the sample, when the sample size n is large, the maximum likelihood estimator of any parameter θis approximately unbiased and has variance that is nearly as small as can be achieved by any estimator. Stated another way, the mle is approximately the MVUE of θ.

Because of this result and the fact that calculus-based techniques can usually be used to derive the mle’s (though often numerical methods, such as Newton’s method, are necessary), maximum likelihood estimation is the most widely used estimation technique among statisticians. Many of the estimators used in the remainder of the book are mle’s. Obtaining an mle, however, does require that the underlying distribution be specified.

Some Complications Example 6.21 Suppose my waiting time for a bus is uniformly distributed on [0,θ] and the results x1,…,xn of a random sample from this distribution have been observed. Since f(x;θ) for 0≤x≤ θ and 0 otherwise,

As long as max(xi) ≤θ,the likelihood is ,which is positive,but as soon as θ <max(xi) ,the likelihood drops to 0.This is illustrated in Figure 6.4.Calculus will not work because the maximum of the likelihood occurs at a point of discontinuity,but the figure shows that .Thus,if my waiting times are 2.3,3.7,1.5,.4,and 3.2,then the mle is

Likelihood max(xi) θ Figure 6.4 The likelihood function for Example 6.21 Example 6.22 is omitted,please read it yourself.