1. Statistics 200b. Spring 2008 David Brillinger
Statistical Models (2003) by A. C. Davison
Chapter 1: Introduction
Chapter 2: Variation
Chapter 3: Uncertainty
Chapter 4: Likelihood
A tool for making statistical inferences (uncertain conclusions based on data)
Midterm March 12, in class. Through Chapter 4.

3. Chapter 4: Likelihood.
"value y of r.v. Y with p.d.f f(y;)  in  f known"
"goal to make statements about distribution of Y based on observed data y" that are plausible, given y observed"

4. Likelihood, L.
L() = f(y;)  in 
regarded as function of  for fixed y
"L will be relatively larger for  near that which generated the data"
When Y's i.i.d., y = (y1,...,yn) sample
L() = f(y;) =   f(yj;)

5. Normal.
IN(,2)  = (,2)
data: (y1,...,yn)
L() =  f(yj;)
=  [exp{-(yj- )2/2 2 } / {2} ]
l() = log L()
= - { log 2 }/2 - { (yj - )2}/2 2

6. An aside
One sees that all one needs is
l() is a maximum at

7. Poisson.
L() = y exp{-} / y! >0, y = 0,1,2,3,...
Goes to 0 as   0 or 
Maximum at  = y
sample y = (y1,...,yn)
 f(yj;) = ^(y1+...+yn) exp{-n} / C
Maximum at  =

8. Exponential distribution
f(y;) = -1 exp{-y/} y >  > 0
L() =  f(yj;) = -n exp{-(y1+...+yn)/}
Maximum at  =

9. L() maximized at  approx 168 L(168) = 2.49E-27 L(150) = 2.32E-27
Spring failure at 950 N/mm2
cycles
1 | 244
1 | 66799
2 | 03
decimal point is 2 digit(s) to the right of the |
L() maximized at  approx 168
L(168) = 2.49E-27
L(150) = 2.32E-27
150 is .93 times less likely than 168
Might declare s with L() > .5 L(168) "plausible"

10. Spring failure times at stress 950 N/mm2
units of 1000 cycles
stleaf
L() maximized at theta approx 168
L(168) approx 2.49E-27
L(150) approx 2.32E-27
"150 is .93 times less likely that 168"
Values with L() > .5 L(168) " plausible": (120,260)

11. Weibull.
f(y;,) = y-1 exp{-(y/)}/ y>0 ,  > 0
L(,) =  f(yj;,)
Contour plot of log L(,)
Maximized at approx (181,6)
Exponential for  = dotted line

12. Richard Feynman

13. Challenger data.
Space shuttle exploded after launch 28 January 1986
Presidential Commission - cause: O-rings not pliable in cold weather or holed in pressure test
Thermal distress
Data
Plot proportions r/m vs temperature x1 and pressure x2 m=6
Statistical model, R binomial Bin(m,π), R=1,...,6
π = eu /(1+eu)
u = β0 + β1 x1 + β2 x2

15. rj O-rings out of m with thermal distress at launch temperature xj
Pr{distress} = π = eu /(1+eu)
u = β0 + β1 x
R ~ IBinomial (m,π), m = 6
L(β0 ,β1) =  Pr{Rj = rj; β0 ,β1}
Contour plot
maximum at (β0 ,β1) approx (5,-0.1)
ridge

17. Summaries.
"key idea is that in many cases log likelihood is approx quadratic in the parameter" {More later}
Example
exponential samples of sizes 5, 10, 20, 40, 80
each sample has = exp{-1}
the maximum of the log likelihood is at
l() = -n (log  /n)
l'( ) = l"( ) = -n/ 2
maximum likelihood estimate,

20. Likelihood, L.
L() = f(y;)  in 
regarded as function of  for fixed y
"L will be relatively larger for  near that which generated the data"
When Y's i.i.d., y = (y1,...,yn) sample
L() = f(y;) =   f(yj;)
l() = log L() =  log f(yj;)

21. Maximum likelihood estimate
likelihood equation.
when things exist.
score statistic/vector.
observed information.

22. Local maximum?
Computation and representation
iterate to convergence

23. log RL() = l() - l( )
 ( - )2l"( )/2
"In many cases the likelihood can be summarized in terms of the mle and the observed information -l"( ). "
In a sense one can act as if the distribution is approximately normal
There are large sample optimality results, but there are problems too, e.g. the uniform on (0,)

24. Observed information
Fisher information
I() = EJ()

25. Normal distribution

26. Properties of U(), u()
E(U()) = nE(u()), E(u()) = 0
Remember

27. Var U() = n Var u()
Var u() = i()

28. 0: true parameter value
U(0), J(0) sums of i.i.d.'s
LLN and CLT
EU(0) = 0, Var(0) = I(0) = ni(0)
EJ(0) = I(0) = ni(0)
I(0)-1 J(0)  1 in probability
I(0)-1/2 U(0)  Z in distribution, as n  
Z ~ Np (0,Ip )

29. In normal case

31. Approximate 100(1-2)% CI for real 0
CR for vector 0