Title Bout: MLE vs. MoM Opponents: R.A. Fisher and Karl Pearson By Lada Kyj and Galen Papkov Rice University - STAT 600 September 27, 2004

Outline Short Biographies Journal rage Criterion Method of Moments Maximum Likelihood MLE vs. MoM Issues

Who’s who?

R.A. Fisher (Born February 17, 1890 in London, England) Superstitious fact Studied mathematics and astronomy at Cambridge (also interested in biology) Rejected from enlisting in the military for WWI due to poor eyesight Introduced concepts such as randomization, likelihood, and ANOVA

Karl Pearson (born March 27, 1857 in London, England) Attended Cambridge Had various interests – Mathematics, physics, metaphysics, law, etc… Contributed to regression analysis and developed the correlation coefficient and the chi-square test. Is characterized as trying to use large samples to deduce correlations in the data whereas Fisher used small samples to determine causes.

Journal Rage Began in 1917 when Pearson claimed that Fisher had failed to distinguish likelihood from inverse probability in a paper he wrote in 1915 Feud continued for many years Fire of feud fed by injustice – “It would be fair to say that both showed hatred towards the other.”

Criteria of Consistency A statistic is consistent if, when it is calculated from the population, it is equal to the population parameter. PROBLEM! – Many statistics for the same parameter can be consistent.

Criteria of Efficiency A statistic is efficient if, when derived from a large sample, it tends to a normal distribution with minimum variance. Relates to estimation accuracy PROBLEM! – This criterion is still incomplete since different methods of calculation may tend to agreement for large samples, but not for finite samples.

Criteria of Sufficiency A statistic is sufficient when no other statistic which can be calculated from the same sample provides any additional information per the value of the parameter to be estimated. Relates to “information”

Method of Moments Developed by Pearson in 1894 Method:  k = E[X k ] –  1 = E[X] = Xbar (sample mean) Satisfies consistency

Example The Cauchy distribution is a great example that portrays the limitations of the MoM. – A few outliers appear to dominate the value of the mean. – Cannot use MoM, go to MLE

Maximum Likelihood Estimation Developed by Fisher – Officially called maximum likelihood estimation in 1921 “The likelihood of a parameter is proportional to the probability of the data.” Method: – Obtain L(x;  ) – Take first derivative, set = 0, solve for 

Conditions for Determining Maximum First order conditions: Second order conditions: (Hessian is negative definite)

Maximum Likelihood Estimation (cont.) Criterion: – Consistency – Efficiency – Sufficiency – poor “proof”! Asymptotically normal and invariant Led to the development of the factorization theorem. Is used as an efficiency yardstick for other estimators, such as the method of moments.

MLE vs. MoM MoM is easy to use for normal curves. – Mean is the best statistic for locating this curve MLE: – Evaluating and maximizing likelihood function is often challenging – Difficult to write down complete statistical model of the joint distribution of the data – More robust – Greater efficiency

Issues Neither are great for exploratory data analysis since the underlying distribution must be known. MLE – is believed to be sufficient (an acceptable proof has not been derived)

References Aldrich, John (1997). R.A. Fisher and the Making of Maximum Likelihood Statistical Science, Vol. 12, No. 3, p Fisher, R. A. (1921a). On the mathematical foundations of theoretical statistics. P