1. Maximum likelihood estimation
Michail Tsagris & Ioannis Tsamardinos

2. Histograms revisited
Looks symmetric and unimodal.

3. Normal distribution
𝑓 𝑥 = 1 2𝜋 𝜎 2 𝑒 − 𝑥−𝜇 𝜎 2

4. MLE of normal distribution
Suppose we have collected some data, RNA expression measurements for example. We do a histogram and we see a nice bell shaped distribution.
How do we find the parameters μ and 𝜎 2 ;
Given, what we see, we will calculate the most probable values assuming that the sample comes from a normally distributed population.
We need to estimate the values of these parameters which are most likely to have produced those data. The values which maximise the likelihoods of observing such data.

5. MLE of normal distribution
So we will calculate the values that maximise the likelihood of having observed these data.
Denote the n values we have observed by 𝑥 1 , 𝑥 2 , …, 𝑥 𝑛 . Each of these 𝑥 𝑖𝑠 come from a normal population with some mean μ and some variance σ2. The two values are the same and in addition, the values are independent from each other (independent and identically distributed, iid).

6. MLE of normal distribution
So, each 𝑥 𝑖 can be plugged into the formula (probability density function) of the normal distribution:
f (𝑥 𝑖 )= 1 2𝜋 𝜎 2 𝑒 − 𝑥 𝑖 −𝜇 𝜎 2 .
Let us take the product of all the 𝑓 𝑖𝑠 :
𝐿(𝜇, 𝜎 2 )= 𝑖=1 𝑛 𝜋 𝜎 2 𝑒 − 𝑥 𝑖 −𝜇 𝜎
The goal is to maximise the above expression.
Better to maximise its logarithm. The two points (𝜇, 𝜎 2 ) which maximise the L will maximise the log(L) as well.

7. MLE of normal distribution
ℓ(𝜇, 𝜎 2 )=− 𝑛 2 log 2𝜋 𝜎 2 − 𝑖=1 𝑛 𝑥 𝑖 −𝜇 𝜎 2
𝜕ℓ 𝜕𝜇 = 𝑖=1 𝑛 𝑥 𝑖 −𝜇 2𝜎 2 =0 ⇒ 𝜇 = 𝑖=1 𝑛 𝑥 𝑖 𝑛 = 𝑥
𝜕ℓ 𝜕 𝜎 2 =− 𝑛 2 𝜎 𝑖=1 𝑛 𝑥 𝑖 −𝜇 𝜎 4 =0⇒
𝜎 2 = 𝑖=1 𝑛 𝑥 𝑖 −𝜇 2 𝑛 .

8. MLE of mean, median and proportion
So, the MLE of the mean is simply the sample mean of the data.
The sample median serves as the MLE of the median.
What about proportions? Suppose that a dose of 1μL kills 5 of the 30 mice in a wet lab experiment. The estimated rate of killing of the dose of this specific drug is 5/30 = or 16.67%.

9. MLE of mean and median
The sample mean can also be seen as the quantity θ that minimises the sum of squares of differences
𝑖=1 𝑛 𝑥 𝑖 −𝜃 2 .
The sample median on the other hand is the quantity θ that minimises the sum of absolute differences
𝑖=1 𝑛 𝑥 𝑖 −𝜃 .

10. Confidence intervals
Suppose one calculates the proportion of mice killed at a given drug dose. Is it enough to present just a number? What else can he say?
The uncertainty must be quantified via a range of most likely values, an interval of most likely values, an interval of high confidence.
The most common sentence is
95% confidence interval. The 5% is the standard, the most popular, but not the unique number.

11. Confidence intervals
We performed a study, found some summary statistics and constructed a 95% confidence interval for the mean or the proportion.
This means that with 95% probability the true means lies within the computed range.

12. Confidence intervals
We performed a study, found some summary statistics and constructed a 95% confidence interval for the mean or the proportion.
This means that with 95% probability the true means lies within the computed range.
Wrong!!!

13. Confidence intervals
We performed a study, found some summary statistics and constructed a 95% confidence interval for the mean or the proportion.
But we only did the analysis once. In other words, our confidence interval has a 95% probability (we expect it is one of the 95% intervals) to have included the true mean or proportion or any parameter in general.

14. Confidence intervals
We performed a study, found some summary statistics and constructed a 95% confidence interval for the mean or the proportion.
If we were to repeat this analysis many times, we would expect that 95% of the times we would have included the true mean.
Correct!!!

15. Confidence intervals

16. Confidence intervals

17. Confidence intervals

18. (Student’s) t distribution (William Gosset)
We have spoken of the normal distribution.
Let us now see the t distribution.
𝑓 𝑥;𝜇, 𝜎 2 = 1 2𝜋 𝜎 2 𝑒 − 𝑥−𝜇 𝜎 Normal density
𝑓 𝑥;𝑣 = Γ( 𝑣+1 2 ) 𝑣𝜋 Γ( 𝑣 2 ) 𝑥 2 𝑣 − 𝑣 t density

19. (Student’s) t distribution (William Gosset)
As v (degrees of freedom)
increases the distribution
approaches the normal
distribution.
lim 𝑣→∞ 𝑡 𝑣 −>𝑁(0, 1)

20. Population and sample revisit
Greek letters indicate population parameters
English letters correspond to sample estimates.
Mean, variance and standard deviation example.
𝜇, 𝜎 2 , 𝜎.
𝑥, 𝑠 2 , 𝑠.

21. (1-α)% confidence interval for the mean
Suppose we have estimated the average glucose concentration to be 𝑥 =86 mg/dL with a variance equal to 𝑠 2 =25 and we want to construct a 95% confidence interval for the true concentration. Our sample consists of 𝑛=31 people.
95% is called confidence level

22. (1-α)% confidence interval for the mean
Suppose we have estimated the average glucose concentration to be 𝑥 =86 mg/dL with a variance equal to 𝑠 2 =25 and we want to construct a 95% confidence interval for the true concentration. Our sample consists of 𝑛=31 people.
𝑥 − 𝑡 1− 𝑎 2 , 𝑛−1 𝑠 𝑛 , 𝑥 + 𝑡 1− 𝑎 2 , 𝑛−1 𝑠 𝑛 .
We have everything but the term 𝑡 1− 𝑎 2 , 𝑣−1 .

23. t distribution tables

24. (1-α)% confidence interval for the mean
𝑡 1− 𝑎 2 , 𝑣−1 = 𝑡 1− , 35−1 = 𝑡 0.975, =2.042
86 − , =
84.166,

25. (1-α)% confidence interval for the proportion
In sample of 132 women who smoke, 58 of them were found to have increased chances of getting breast cancer. 𝑝 = = or 43.94% The relevant 95% confidence interval is given by 𝑝 − Ζ 1− 𝑎 2 𝑝 (1− 𝑝 ) 𝑛 , 𝑝 + Ζ 1− 𝑎 2 𝑝 (1− 𝑝 ) 𝑛 . What is Ζ 1− 𝑎 2 ?

27. (1-α)% confidence interval for the proportion
Ζ 1− 𝑎 2 = Ζ 1− = Ζ =1.96.
− − , (1−0.4394) − − , (1−0.4394) 132
0.3547, or %, 52.41% .