1. EC 331 The Theory of and applications of Maximum Likelihood Method
Burak Saltoğlu

2. outline Maximum Liklelihood Principle
Estimating population parameters via ML method
Properties of ML
OLS vs ML

3. This represents the joint density of y’s given parameter
1 Maximum Likelihood
ML method is based on the principle that the parameter
estimates can be obtained by maximising the likelihood of the selected sample
to reflect the population.
We choose the parameters in a way that we maximize the joint likelihood of representing the population. Suppose we are given iid observed sample of y
and also a parameter vector (of k dimesion) can be represented as
This represents the joint density of y’s given parameter

4. Likelihood Function
Joint likelihood function then can be written as the joint probability of observing y’s drawn from f(.)
Likelihood function is
Maximizing above function w.r.t will yield a special value that maximizes
the probability of obtaining sample values that have actually observed.
In most applications it is convinient to work with loglikelihood function, which is

5. Likelihood Function Note that
Also note that above equation is known as score .

6. Example-1 Poisson distribbution due to Siméon Denis Poisson expresses the probability of a given number of events occurring in a fixed interval of time these events occur with a known average rate and independently of the time since the last event use: defaults of countries, customers,

8. Example-1

9. Example-1

10. Numerical example

11. Numerical example

13. Likelihood profile (lambda in the horizontal axis)

14. Likelihood and log-likelihood for Poisson (rescaled Graph)

15. Example-2 It describes the time between events in a Poisson process

16. Example-2

17. Example-2

18. Example-3

19. Example-4

20. Convergence in Probability
Definition : Let xn be a sequence random variable where n is sample size,
the random variable xn converges in probability to a constant c if
the values that the x may take that are not close to c become
increasingly unlikely as n increases.
If xn converges to c, then we say,
All the mass of the probability distribution concentrates around c.

21. Properties of MLE Consistency: Asymtotic Normality:
where information matrix is
that is, the hessian of log-likelihood function.

22. 3.3 Properties of MLE Asymtotic Efficiency:
Assumimg that we are dealing with only one parameter θ;
which states that if there is another consistent and asymtotically normal estimator of to θ then ,
Invariance:

23. 4 Estimation of the Linear Regression Model

26. Matrix notation

27. 3.4 Estimation of the Linear Regression Model
Parameter vector is

28. 3.4 Estimation of the Linear Regression Model
To calculate variance matrix of parameters, we need hessian of likelihood
parameters. İf we take ot second derivatives
Taking expectations,

29. 3.4 Estimation of the Linear Regression Model

30. 3.4 Estimation of the Linear Regression Model
So, the information matrix is
The inverse of the information matrix will give us the variance-covariance matrix of the MLE estimators,

31. Testing in Maximum Likelihood Framework

35. Example from Poisson example

36. Example
This ratio is always between 0 and 1 and the less likely the assumption is, the smaller this ratio 

37. Likelihood Ratio Test If we want to test Restricted
likelihood ratio defined as
can be used with decision rule
Restricted
Unrestricted
q:#restrictions

38. Likelihood Ratio Test
Don’t reject the null

39. More on LR test in the context of Linear Regression

40. Likelihood Ratio Test