1. STATISTICS POINT ESTIMATION
Professor Ke-Sheng Cheng
Department of Bioenvironmental Systems Engineering
National Taiwan University

2. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

3. Is it originated from a normal (or gamma) distribution?
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

4. Statistical inference(統計推論)
Given a random sample from the distribution of a population, we often are interested in making inferences about the population.
Two important statistical inferences are
Estimation (推估)
Test of hypotheses (假設檢定).
Normal distribution?
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

5. Parameter estimation
Assume that some characteristics of the elements in a population can be represented by a RV X with pdf fX(．;θ), where the “form” of the density is assumed known except that it contains some unknown parameters θ.
We want to estimate an unknown parameter θ or some function of the unknown parameter, τ(θ), using observed values of a random sample, x1,x2,…,xn.
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

6. Point estimation
Let the value of some statistic, say , represent the unknown parameter θ or τ(θ); such a statistic is called a point estimator.
For example, and
respectively are point estimator of  and .
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

7. Two statistics and , where , so that
Interval estimation
Two statistics and , where , so that
( , ) constitutes an interval for which the probability can be determined that it contains the unknown parameter θ or
τ(θ).
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

8. Point Estimation of Distribution Parameters
What parameters are to be estimated?
Parameters are variables that characterize distributions. For example mean and standard deviation are parameters.
How can we estimate the parameters?
methods of finding estimators
desired properties of point estimators
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

9. To estimate parameters we need to have a random sample
To estimate parameters we need to have a random sample. For example, a random sample (x1,x2,…,xn) of f(．;θ) is collected in order to estimate the parameter θ. Therefore,
There can be many (or infinite) ways of estimation and we need to establish some kind of criteria in order to have an adequate estimator.
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

10. Estimators and Estimates
Estimator: Any “statistic” whose values are used to estimate τ(θ), where τ(．) is some function of the parameter θ, is defined to be an estimator of Note that an estimator is a random variable.
Estimate: The estimated value given by an estimator.
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11. Methods of finding estimator
Assume that x1,x2,…,xn is a random sample from a density f(．;θ), where the form of the density is known but the parameter θ is unknown. Further assume that θ is a vector of real numbers, say θ=(θ1,θ2,…,θk).
We often let , called the parameter space, denote the set of possible values that the parameter θ can assume. Our objective is to find statistics to be used as estimators of certain functions, say , of θ= (θ1,…,θk).
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

12. Method of moments
Let f(．;θ1,θ2,…,θk) be the density of random variable X which has k parameters
In general will be a known function of the k parameters , i.e.
Let x1, x2,…, xn be a random sample from the density f(．; ). From the k equations, we have
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

13. We can solve a solution of expressed in terms of x1, x2,…, xn
We can solve a solution of expressed in terms of x1, x2,…, xn. The solution, i.e , is called an estimator of
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

14. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

15. Given a random sample of a normal distribution with mean μ and variance σ2. Estimate the parameters μ and σ by the method of moments.
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

16. Note that the method of moment estimator of σ2 is NOT the sample variance.
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

17. Let x1, x2,…, xn be a random sample from a Poisson distribution with parameter λ.
Estimate using MOM.
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

18. Let be a random sample from a uniform distribution on
Let be a random sample from a uniform distribution on What are the method of moments estimator of and ?
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

19. Maximum Likelihood Method Definition of the likelihood function
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

20. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

21. Rationale of using likelihood function for parameter estimation
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

22. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

23. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

24. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

25. Definition of the maximum likelihood estimator
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

26. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

27. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

28. Example
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

29. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

30. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

31. Example
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

32. Example
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

33. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

34. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

35. Example
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

36. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

37. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

38. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

39. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

40. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

41. Parameter estimators of various distributions
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

42. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

43. Properties of point estimators
Among different estimators, we want to know whether one estimator is “better” than others or what properties an estimator may or may not possess.
Unbiasedness
Efficiency
Mean squared error
Consistency
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

44. Unbiased estimator An estimator ( ) of is said to be unbiased if
An unbiased estimator is said to be more efficient than any other unbiased estimator of , if
for all .
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

45. Mean squared error of an estimator
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

46. Naturally, we would prefer to find an estimator that has the smallest mean-squared error, however, such estimators rarely exist. In general, the mean-squared error of an estimator depends on
What should we look for if a uniformly minimum MSE estimator rarely exists?
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

47. Uniformly minimum MSE estimator
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

48. Let’s define = bias of T . [Note: is an estimator of .] Then,
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

49. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

50. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

51. Bias
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

52. We look for an estimator that has a uniformly minimum MSE within the class of unbiased estimators. Such an estimator is called a uniformly minimum-variance unbiased estimator (UMVUE).
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

53. Example
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

54. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

55. The MSE of is
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

56. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

57. Standard Error
The “standard error” of a statistic is the standard deviation of its sampling distribution. If the standard error involves unknown parameters whose values can be estimated, substitution of those estimates into the standard error results in an “estimated standard error”.
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

58. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

59. Mean-squared-error consistency
We discuss the mean-squared error of an estimator derived from a random sample of “fixed” size n. Properties of point estimators that are defined for a “fixed” sample size are referred to as “small-sample” properties, whereas properties that are defined for increasing sample size are referred to as “large-sample” properties.
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

60. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

61. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

62. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

63. Properties of the maximum likelihood estimators
Asymptotic properties of the MLEs
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

64. Invariance property of the MLEs
The invariance property does not hold for unbiasedness.
5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

65. 5/31/2018
Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.