1. Linear Regression Summer School IFPRI
Westminster International University in Tashkent
2018

2. Regression
Regression analysis is concerned with the study of the dependence of one variable, the dependent variable, on one or more other variables, the explanatory variables, with a view of estimating and/or predicting the population mean or average values of the former in terms of the known or fixed (in repeated sampling) values of the latter.

3. Terminology and Notation
Dependent Variable
Independent variable
Explained variable
Predictand
Predictor
Regressand
Regressor
Response
Stimulus
Endogenous
Exogenous
Outcome
Covariate
Controlled variable
Control variable

4. Conditional Mean 77 89 101 149 161 173 Income 80 100 120 140 160 180
Income 80
100
120
140
160
180
200
220
240
260
Consumption 55 65 79
102
110
135
137
150 60 70 84 93
107
115
136
145
152 74 90 95
155
175 94
103
116
130
144
165
178 75 85 98
108
118
157 88
113
125
189
185
162
191
Total
325
462
445
707
678
750
685
1043
966
1211
Conditional mean 77 89
101
149
161
173

5. Population Regression
Simple Regression
Conditional expected
values E(Y|X)
Population Regression
Curve
A population regression curve is simply the locus of the conditional means of the dependent variable for the fixed values of the explanatory variable(s).

6. Simple Regression Conditional Expectation Function (CEF)
Population Regression
Population Regression Function (PRF)
Linear Population Regression Function
Regression Coefficients

7. Linear Linear in parameter functions Non-linear in parameter functionX Y X Y X Y
Linear in parameter functions
Non-linear in parameter function

8. Stochastic specification
Stochastic error term
Nonsystematic component
Systematic component

9. Sample Regression Function
SRF2
SRF1

10. Sample Regression Function
PRF
SRF
Estimate

11. Sample Regression FunctionY X A

12. Assumptions.
Linearity. The relationship between independent and dependent variable is linear.
Full Rank. There is no exact relationship among any independent variables.
Exogeneity of independent variables. The error term of the regression is not a function of independent variables.
Homoscedastisity and no Autocorrelation. Error term of the regression is independently and normally distributed with zero means and constant variance.
Normality of Error term

13. Ordinary Least Squares

14. Ordinary Least Squares

15. Ordinary Least Squares

16. Assumptions Linear Regression Model
X values are repeated in sampling – X is assumed to be nonstochastic.
Zero mean values of disturbance ui

17. Assumptions
Homoscedasticity or equal variance of ui X Y
f(u)

18. Assumptions X Y
f(u)
Heteroscedasticity

19. Assumptions No autocorrelation between the disturbances
Exogeneity. Zero covariance between Xi and ui

20. Coefficient moments
Estimator
True value
Additionally we know that

21. Coefficient moments
According to our “Exogenity” assumption. (Error term is independent from X variable.
Thus, OLS estimator is unbiased estimator.

22. Coefficient moments
According to Homoscedasticity and no auto-correlation assumptions.

23. Coefficient moments
According to Homoscedasticity and no auto-correlation assumptions.

24. Using similar argument

25. BLUE estimator Sampling distribution of β2*

26. OLS Estimation: Multiple Regression Model

27. Assumptions and estimation.
Assumptions are the same
Minimize the sum of squared residuals
The unbiased estimator of
R square
Adjusted R square

28. OLS Estimation: Multiple regression model

29. Goodness of Fit
TSS = ESS + RSS

30. Goodness of Fit
TSS = ESS + RSS