P ROBABLITY S TATICS &

PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear regression model,assuming the regressor X is given (that is, not random -- this is also commonly referred to as conditioning on X = x). One must check whether the MLE of the model parameters ( and ) are the same as their LSE’s.

Solution The p.d.f. is We derive the Maximum Likelihood Estimator for the simple linear regression model:

Derive the total probability function L Derive lnL

We compute the and, namely that Note:

We also set up We will still use them in next projects.

It's easy to get the result: As we can see, they are the same as their LSE's

PROJECT. 2 Errors in Variable (EIV) regression. In this case, let's derive its distribution.

X and Y follow a bivariate normal distribution: And we need to derive the MLE of the regression slope :

In order to simplify the calculation,we also bulit a simple EIV model. In this model, are not ramdon any longer.It means that X and Y are independent.And we can also derive the MLE of the regression slope as same expression.

Solution Let's derive the MLE for simple EIV model:

Let 、 and then we have Here, stands for.

So we can get Now,we will pin-point which special cases correspond to the OR and the GMR(they are 2 kinds of way to derive the linear regression).

the Orthogonal Regression So,we have

the Geometric Mean Regression So,we have

PROJECT. 3 Our third project is to derive a class of non-parametric estimators of the EIV model for simple linear regression based on minimizing the sum of the following distance fromeach point to the line as illustrate din the figure below:

Please also show whether there is a 1-1 relationship between this class of estimators and those in Project 2(A/B). That is, try to ascertain whether there is a 1-1 relationship between c and.

We know that Therefore,we need to minimize the sum an d Here, stands for

Let and,we can get that

Solving these two equations, we can get s.t. When we have the examples, we can put them in this founction,then we can compute the.

PROJECT. 4 In the project 3,we have found that and are 1-1 relationship. In this case, we can easily derive that and are 1-1 relationship, so and are 1-1 relationship.

PROJECT. 6 For those who have finished Projects 2 & 3 & 4 above, youmay also examine how to estimate the error variance ratio, when we have two repeated measures on each sample.

Let be the material and measurement. Therefore, we have It’s easy to know these elements are independent.

In order to simplify the calculation,we also bulit a simple EIV model.

Let and,we can get an d Let and,we can get

Therefore