1. Statistical Inference and Regression Analysis: GB.3302.30
Professor William Greene
Stern School of Business
IOMS Department
Department of Economics

2. Statistics and Data Analysis
Part 10 – Advanced Topics

3. Advanced topics Nonlinear Least Squares
Nonlinear Models – ML Estimation
Poisson Regression
Binary Choice
End of course.

4. Statistics and Data Analysis
Nonlinear Least Squares

5. Nonlinear Least Squares

6. Lanczos 1 Data

7. Nonlinear Regression

8. Nonlinear Least Squares
There are no explicit solutions to these equations in the form of bi = a function of (y,x).

9. Strategy for Nonlinear LS

10. NLS Strategy Pick b A. Compute yi0 and xi0 B. Regress yi0 on xi0
This obtains a new b
Return to step A or exit if the new b is the same as the old b

13. Lanczos 1 First Iteration
Now, repeat the iteration using this as b

14. This is the correct answer

15. Gauss-Marquardt Algorithm
Starting with b0
A. Compute regressors xi0
Compute residuals ei0 = yi – f(xi,b0)
B. New b1 = b slopes in regression of ei0 on xi0
Return to A. or exit if estimates have converged.
This is equivalent to our earlier method.

16. Statistics and Data Analysis
Maximum Likelihood: Poisson

17. Application: Doctor Visits
German Individual Health Care data: N=27,236
Model for number of visits to the doctor:
Poisson regression
Age, Health Satisfaction, Marital Status, Income, Kids

18. Poisson Regression

19. Nonlinear Least Squares

20. Maximum Likelihood Estimation
This defines a class of estimators based on the particular distribution assumed to have generated the observed random variable.
The main advantage of ML estimators is that among all Consistent Asymptotically Normal Estimators, MLEs have optimal asymptotic properties.

21. Setting up the MLE
The distribution of the observed random variable is written as a function of the parameters to be estimated
P(yi|data,β) = Probability density | parameters.
The likelihood function is constructed from the density
Construction: Joint probability density function of the observed sample of data – generally the product when the data are a random sample.

22. Likelihood for the Poisson Regression

23. Newton’s Method

25. Properties of the MLE Consistent: Not necessarily unbiased, however
Asymptotically normally distributed: Proof based on central limit theorems
Asymptotically efficient: Among the possible estimators that are consistent and asymptotically normally distributed
Invariant: The MLE of g() is g(the MLE of )

26. Computing the Asymptotic Variance
We want to estimate {-E[H]}-1 Three ways:
(1) Just compute the negative of the actual second derivatives matrix and invert it.
(2) Insert the maximum likelihood estimates into the known expected values of the second derivatives matrix. Sometimes (1) and (2) give the same answer (for example, in the Poisson regression model).
(3) Since E[H] is the variance of the first derivatives, estimate this with the sample variance (i.e., mean square) of the first derivatives. This will almost always be different from (1) and (2).
Since they are estimating the same thing, in large samples, all three will give the same answer.

27. Poisson Regression Iterations

28. MLE
NLS

29. Using the Model. Partial Effects

30. Effect of Income Depends on Age

31. Effect of Income | Age

32. Statistics and Data Analysis
Binary Choice

33. Case Study: Credit Modeling
1992 American Express analysis of
Application process: Acceptance or rejection; Y = 0 (reject) or 1 (accept).
Cardholder behavior
Loan default (D = 0 or 1).
Average monthly expenditure (E = $/month)
General credit usage/behavior (C = number of charges)
13,444 applications in November, 1992

34. Proportion for Bernoulli
In the AmEx data, the true population acceptance rate is = 
Y = 1 if application accepted, 0 if not.
E[y] = 
E[(1/N)Σiyi] = paccept = .
This is the estimator

35. Some Evidence = Homeowners
Does the acceptance rate depend on home ownership?

36. A Test of Independence
In the credit card example, are Own/Rent and Accept/Reject independent?
Hypothesis: Prob(Ownership) and Prob(Acceptance) are independent
Formal hypothesis, based only on the laws of probability: Prob(Own,Accept) = Prob(Own)Prob(Accept) (and likewise for the other three possibilities.
Rejection region: Joint frequencies that do not look like the products of the marginal frequencies.

37. Contingency Table Analysis
The Data: Frequencies Reject Accept Total Rent , , ,214 Own , , ,630 Total , , ,444
Step 1: Convert to Actual Proportions Reject Accept Total Rent Own Total

38. Independence Test
Step 2: Expected proportions assuming independence: If the factors are independent, then the joint proportions should equal the product of the marginal proportions.
[Rent,Reject] x = [Rent,Accept] x = [Own,Reject] x = [Own,Accept] x =

39. Comparing Actual to Expected
It appears that the acceptance rate is dependent on home ownership

40. When is the Chi Squared Large?
Critical chi squared D.F
Critical values from chi squared table
Degrees of freedom = (R-1)(C-1).

41. Analyzing Default
DEFAULT
OWNRENT All
All
Do renters default more often (at a different rate) than owners?
To investigate, we study the cardholders (only)

42. Hypothesis Test

43. Central Proposition: A Utility Based Approach
Observed outcomes partially reveal underlying preferences
There exists an underlying preference scale defined over alternatives, U*(choices)
Revelation of preferences between two choices labeled 0 and 1 reveals the ranking of the underlying utility
U*(choice 1) > U*(choice 0) Choose 1
U*(choice 1) < U*(choice 0) Choose 0
Net utility = U = U*(choice 1) - U*(choice 0). U > 0 => choice 1

44. Binary Outcome: Visit Doctor In the 1984 year of the GSOEP, 1611 of individuals visited the doctor at least once.

45. More Formal Model of Acceptance and Default

46. Probability Models zi

47. Likelihood Function

48. American Express, 1992

49. Logistic Model for Acceptance

50. Probit Default Model

52. Ordered Discrete Outcomes
E.g.: Taste test, credit rating, course grade, preference scale
Underlying random preferences:
Existence of an underlying continuous preference scale
Mapping to observed choices
Strength of preferences is reflected in the discrete outcome
Censoring and discrete measurement
The nature of ordered data

53. Ordered Choices at IMDb

56. Health Satisfaction (HSAT)
Self administered survey: Health Care Satisfaction (0 – 10)
Continuous Preference Scale

57. Dueling Selection Biases – From two emails, same day.
“I am trying to find methods which can deal with data that is non-randomised and suffers from selection bias.”
“I explain the probability of answering questions using, among other independent variables, a variable which measures knowledge breadth. Knowledge breadth can be constructed only for those individuals that fill in a skill description in the company intranet. This is where the selection bias comes from.

58. The Crucial Element Selection on the unobservables
Selection into the sample is based on both observables and unobservables
All the observables are accounted for
Unobservables in the selection rule also appear in the model of interest (or are correlated with unobservables in the model of interest)
“Selection Bias”=the bias due to not accounting for the unobservables that link the equations.

59. Canonical Sample Selection Model

60. Applications Labor Supply model:
y*=wage-reservation wage
d=labor force participation
Attrition model: Clinical studies of medicines
Survival bias in financial data
Income studies – value of a college application
Treatment effects
Any survey data in which respondents self select to report
Etc…

61. Estimation of the Selection Model
Two step least squares
Inefficient
Simple – exists in current software
Simple to understand and widely used
Full information maximum likelihood
Efficient
Not so simple to understand – widely misunderstood

62. Heckman’s Model

63. Two Step Estimation
The “LAMBDA”

64. Classic Application
Mroz, T., Married women’s labor supply, Econometrica, 1987.
N =753
N1 = 428
A (my) specification
LFP=f(age,age2,family income, education, kids)
Wage=g(experience, exp2, education, city)

65. Selection Equation +---------------------------------------------+
| Binomial Probit Model |
| Dependent variable LFP |
| Number of observations |
| Log likelihood function |
|Variable| Coefficient | Standard Error |b/St.Er.|P[|Z|>z]| Mean of X|
Index function for probability
Constant|
AGE |
AGESQ |
FAMINC | D D
WE |
KIDS |

66. Heckman Estimator

67. University of Connecticut Daniel Solis University of Miami
TECHNICAL EFFICIENCY ANALYSIS CORRECTING FOR BIASES FROM OBSERVED AND UNOBSERVED VARIABLES: AN APPLICATION TO A NATURAL RESOURCE MANAGEMENT PROJECT Empirical Economics: Volume 43, Issue 1 (2012), Pages 55-72
Boris Bravo-Ureta
University of Connecticut
Daniel Solis
University of Miami
William Greene New York University

68. The MARENA Program in Honduras
 Several programs have been implemented to address resource degradation while also seeking to improve productivity, managerial performance and reduce poverty (and in some cases make up for lack of public support).
 One such effort is the Programa Multifase de Manejo de Recursos Naturales en Cuencas Prioritarias or MARENA in Honduras focusing on small scale hillside farmers.

69. OVERALL CONCEPTUAL FRAMEWORK
Training &
Financing
MARENA
More Production and Productivity
Natural, Human &
Social Capital
More Farm
Income
Off-Farm
Income
Sustainability
Working HYPOTHESIS: if farmers receive private benefits (higher income) from project activities (e.g., training, financing) then adoption is likely to be sustainable and to generate positive externalities. 69

70. The MARENA Program
 COMPONENT I: Strengthening Strategic Management Capabilities among Govt. Institutions (central and local)
 COMPONENT II: Support to Nat. Res. Management. Projects
 Module 1: Promotion and Organization
 Modulo 2: Strengthening Local Institutions & Organizations
 Module 3: Investment (farm, municipal & regional)
 COMPONENT III: Administration and Supervision

71. Component II - Module 3
 Component II - Module 3 focused on promoting investments in sustainable production systems with a budget of US $7.6 million (Bravo-Ureta, 2009).
 The major activities undertaken with beneficiaries: training in business management and sustainable farming practices; and the provision of funds to co-finance investment activities through local rural savings associations (cajas rurales).

72. Expected Impact Evaluation

73. Cornwell and Rupert Data
Cornwell and Rupert Returns to Schooling Data, 595 Individuals, 7 Years Variables in the file are
EXP = work experience WKS = weeks worked OCC = occupation, 1 if blue collar, IND = 1 if manufacturing industry SOUTH = 1 if resides in south SMSA = 1 if resides in a city (SMSA) MS = 1 if married FEM = 1 if female UNION = 1 if wage set by union contract ED = years of education LWAGE = log of wage = dependent variable in regressions
These data were analyzed in Cornwell, C. and Rupert, P., "Efficient Estimation with Panel Data: An Empirical Comparison of Instrumental Variable Estimators," Journal of Applied Econometrics, 3, 1988, pp  See Baltagi, page 122 for further analysis.  The data were downloaded from the website for Baltagi's text. 73

74. Specification

75. The Effect of Education on LWAGE

76. What Influences LWAGE?

77. An Exogenous Influence

78. Instrumental Variables
Structure
LWAGE (ED,EXP,EXPSQ,WKS,OCC, SOUTH,SMSA,UNION)
ED (MS, FEM)
Reduced Form: LWAGE[ ED (MS, FEM), EXP,EXPSQ,WKS,OCC, SOUTH,SMSA,UNION ]

79. Two Stage Least Squares Strategy
Reduced Form: LWAGE[ ED (MS, FEM,X), EXP,EXPSQ,WKS,OCC, SOUTH,SMSA,UNION ]
Strategy
(1) Purge ED of the influence of everything but MS, FEM (and the other variables). Predict ED using all exogenous information in the sample (X and Z).
(2) Regress LWAGE on this prediction of ED and everything else.
Standard errors must be adjusted for the predicted ED