Statistical Inference and Regression Analysis: GB.3302.30 Professor William Greene Stern School of Business IOMS Department Department of Economics

Statistics and Data Analysis Part 10 – Advanced Topics

Advanced topics Nonlinear Least Squares Nonlinear Models – ML Estimation Poisson Regression Binary Choice End of course.

Statistics and Data Analysis Nonlinear Least Squares

Nonlinear Least Squares

Lanczos 1 Data

Nonlinear Regression

Nonlinear Least Squares There are no explicit solutions to these equations in the form of bi = a function of (y,x).

Strategy for Nonlinear LS

NLS Strategy Pick b A. Compute yi0 and xi0 B. Regress yi0 on xi0 This obtains a new b Return to step A or exit if the new b is the same as the old b

Lanczos 1 First Iteration Now, repeat the iteration using this as b

This is the correct answer

Gauss-Marquardt Algorithm Starting with b0 A. Compute regressors xi0 Compute residuals ei0 = yi – f(xi,b0) B. New b1 = b0 + slopes in regression of ei0 on xi0 Return to A. or exit if estimates have converged. This is equivalent to our earlier method.

Statistics and Data Analysis Maximum Likelihood: Poisson

Application: Doctor Visits German Individual Health Care data: N=27,236 Model for number of visits to the doctor: Poisson regression Age, Health Satisfaction, Marital Status, Income, Kids

Poisson Regression

Nonlinear Least Squares

Maximum Likelihood Estimation This defines a class of estimators based on the particular distribution assumed to have generated the observed random variable. The main advantage of ML estimators is that among all Consistent Asymptotically Normal Estimators, MLEs have optimal asymptotic properties.

Setting up the MLE The distribution of the observed random variable is written as a function of the parameters to be estimated P(yi|data,β) = Probability density | parameters. The likelihood function is constructed from the density Construction: Joint probability density function of the observed sample of data – generally the product when the data are a random sample.

Likelihood for the Poisson Regression

Newton’s Method

Properties of the MLE Consistent: Not necessarily unbiased, however Asymptotically normally distributed: Proof based on central limit theorems Asymptotically efficient: Among the possible estimators that are consistent and asymptotically normally distributed Invariant: The MLE of g() is g(the MLE of )

Computing the Asymptotic Variance We want to estimate {-E[H]}-1 Three ways: (1) Just compute the negative of the actual second derivatives matrix and invert it. (2) Insert the maximum likelihood estimates into the known expected values of the second derivatives matrix. Sometimes (1) and (2) give the same answer (for example, in the Poisson regression model). (3) Since E[H] is the variance of the first derivatives, estimate this with the sample variance (i.e., mean square) of the first derivatives. This will almost always be different from (1) and (2). Since they are estimating the same thing, in large samples, all three will give the same answer.

Poisson Regression Iterations

MLE NLS

Using the Model. Partial Effects

Effect of Income Depends on Age

Effect of Income | Age

Statistics and Data Analysis Binary Choice

Case Study: Credit Modeling 1992 American Express analysis of Application process: Acceptance or rejection; Y = 0 (reject) or 1 (accept). Cardholder behavior Loan default (D = 0 or 1). Average monthly expenditure (E = $/month) General credit usage/behavior (C = number of charges) 13,444 applications in November, 1992

Proportion for Bernoulli In the AmEx data, the true population acceptance rate is 0.7809 =  Y = 1 if application accepted, 0 if not. E[y] =  E[(1/N)Σiyi] = paccept = . This is the estimator

Some Evidence = Homeowners Does the acceptance rate depend on home ownership?

A Test of Independence In the credit card example, are Own/Rent and Accept/Reject independent? Hypothesis: Prob(Ownership) and Prob(Acceptance) are independent Formal hypothesis, based only on the laws of probability: Prob(Own,Accept) = Prob(Own)Prob(Accept) (and likewise for the other three possibilities. Rejection region: Joint frequencies that do not look like the products of the marginal frequencies.

Contingency Table Analysis The Data: Frequencies Reject Accept Total Rent 1,845 5,469 7,214 Own 1,100 5,030 6,630 Total 2,945 10,499 13,444 Step 1: Convert to Actual Proportions Reject Accept Total Rent 0.13724 0.40680 0.54404 Own 0.08182 0.37414 0.45596 Total 0.21906 0.78094 1.00000

Independence Test Step 2: Expected proportions assuming independence: If the factors are independent, then the joint proportions should equal the product of the marginal proportions. [Rent,Reject] 0.54404 x 0.21906 = 0.11918 [Rent,Accept] 0.54404 x 0.78094 = 0.42486 [Own,Reject] 0.45596 x 0.21906 = 0.09988 [Own,Accept] 0.45596 x 0.78094 = 0.35606

Comparing Actual to Expected It appears that the acceptance rate is dependent on home ownership

When is the Chi Squared Large? Critical chi squared D.F. .05 .01 1 3.84 6.63 2 5.99 9.21 3 7.81 11.34 4 9.49 13.28 5 11.07 15.09 6 12.59 16.81 7 14.07 18.48 8 15.51 20.09 9 16.92 21.67 10 18.31 23.21 Critical values from chi squared table Degrees of freedom = (R-1)(C-1).

Analyzing Default DEFAULT OWNRENT 0 1 All 0 4854 615 5469 46.23 5.86 52.09 1 4649 381 5030 44.28 3.63 47.91 All 9503 996 10499 90.51 9.49 100.00 Do renters default more often (at a different rate) than owners? To investigate, we study the cardholders (only)

Hypothesis Test

Central Proposition: A Utility Based Approach Observed outcomes partially reveal underlying preferences There exists an underlying preference scale defined over alternatives, U*(choices) Revelation of preferences between two choices labeled 0 and 1 reveals the ranking of the underlying utility U*(choice 1) > U*(choice 0) Choose 1 U*(choice 1) < U*(choice 0) Choose 0 Net utility = U = U*(choice 1) - U*(choice 0). U > 0 => choice 1

Binary Outcome: Visit Doctor In the 1984 year of the GSOEP, 1611 of 3874 individuals visited the doctor at least once.

More Formal Model of Acceptance and Default

Probability Models zi

Likelihood Function

American Express, 1992

Logistic Model for Acceptance

Probit Default Model

Ordered Discrete Outcomes E.g.: Taste test, credit rating, course grade, preference scale Underlying random preferences: Existence of an underlying continuous preference scale Mapping to observed choices Strength of preferences is reflected in the discrete outcome Censoring and discrete measurement The nature of ordered data

Ordered Choices at IMDb

Health Satisfaction (HSAT) Self administered survey: Health Care Satisfaction (0 – 10) Continuous Preference Scale

Dueling Selection Biases – From two emails, same day. “I am trying to find methods which can deal with data that is non-randomised and suffers from selection bias.” “I explain the probability of answering questions using, among other independent variables, a variable which measures knowledge breadth. Knowledge breadth can be constructed only for those individuals that fill in a skill description in the company intranet. This is where the selection bias comes from.

The Crucial Element Selection on the unobservables Selection into the sample is based on both observables and unobservables All the observables are accounted for Unobservables in the selection rule also appear in the model of interest (or are correlated with unobservables in the model of interest) “Selection Bias”=the bias due to not accounting for the unobservables that link the equations.

Canonical Sample Selection Model

Applications Labor Supply model: y*=wage-reservation wage d=labor force participation Attrition model: Clinical studies of medicines Survival bias in financial data Income studies – value of a college application Treatment effects Any survey data in which respondents self select to report Etc…

Estimation of the Selection Model Two step least squares Inefficient Simple – exists in current software Simple to understand and widely used Full information maximum likelihood Efficient Not so simple to understand – widely misunderstood

Heckman’s Model

Two Step Estimation The “LAMBDA”

Classic Application Mroz, T., Married women’s labor supply, Econometrica, 1987. N =753 N1 = 428 A (my) specification LFP=f(age,age2,family income, education, kids) Wage=g(experience, exp2, education, city)

Selection Equation +---------------------------------------------+ | Binomial Probit Model | | Dependent variable LFP | | Number of observations 753 | | Log likelihood function -490.8478 | +--------+--------------+----------------+--------+--------+----------+ |Variable| Coefficient | Standard Error |b/St.Er.|P[|Z|>z]| Mean of X| ---------+Index function for probability Constant| -4.15680692 1.40208596 -2.965 .0030 AGE | .18539510 .06596666 2.810 .0049 42.5378486 AGESQ | -.00242590 .00077354 -3.136 .0017 1874.54847 FAMINC | .458045D-05 .420642D-05 1.089 .2762 23080.5950 WE | .09818228 .02298412 4.272 .0000 12.2868526 KIDS | -.44898674 .13091150 -3.430 .0006 .69588313

Heckman Estimator

University of Connecticut Daniel Solis University of Miami TECHNICAL EFFICIENCY ANALYSIS CORRECTING FOR BIASES FROM OBSERVED AND UNOBSERVED VARIABLES: AN APPLICATION TO A NATURAL RESOURCE MANAGEMENT PROJECT Empirical Economics: Volume 43, Issue 1 (2012), Pages 55-72 Boris Bravo-Ureta University of Connecticut Daniel Solis University of Miami William Greene New York University

The MARENA Program in Honduras  Several programs have been implemented to address resource degradation while also seeking to improve productivity, managerial performance and reduce poverty (and in some cases make up for lack of public support).  One such effort is the Programa Multifase de Manejo de Recursos Naturales en Cuencas Prioritarias or MARENA in Honduras focusing on small scale hillside farmers.

OVERALL CONCEPTUAL FRAMEWORK Training & Financing MARENA More Production and Productivity Natural, Human & Social Capital More Farm Income Off-Farm Income Sustainability Working HYPOTHESIS: if farmers receive private benefits (higher income) from project activities (e.g., training, financing) then adoption is likely to be sustainable and to generate positive externalities. 69

The MARENA Program  COMPONENT I: Strengthening Strategic Management Capabilities among Govt. Institutions (central and local)  COMPONENT II: Support to Nat. Res. Management. Projects  Module 1: Promotion and Organization  Modulo 2: Strengthening Local Institutions & Organizations  Module 3: Investment (farm, municipal & regional)  COMPONENT III: Administration and Supervision

Component II - Module 3  Component II - Module 3 focused on promoting investments in sustainable production systems with a budget of US $7.6 million (Bravo-Ureta, 2009).  The major activities undertaken with beneficiaries: training in business management and sustainable farming practices; and the provision of funds to co-finance investment activities through local rural savings associations (cajas rurales).

Expected Impact Evaluation

Cornwell and Rupert Data Cornwell and Rupert Returns to Schooling Data, 595 Individuals, 7 Years Variables in the file are EXP = work experience WKS = weeks worked OCC = occupation, 1 if blue collar, IND = 1 if manufacturing industry SOUTH = 1 if resides in south SMSA = 1 if resides in a city (SMSA) MS = 1 if married FEM = 1 if female UNION = 1 if wage set by union contract ED = years of education LWAGE = log of wage = dependent variable in regressions These data were analyzed in Cornwell, C. and Rupert, P., "Efficient Estimation with Panel Data: An Empirical Comparison of Instrumental Variable Estimators," Journal of Applied Econometrics, 3, 1988, pp. 149-155.  See Baltagi, page 122 for further analysis.  The data were downloaded from the website for Baltagi's text. 73

Specification

The Effect of Education on LWAGE

What Influences LWAGE?

An Exogenous Influence

Instrumental Variables Structure LWAGE (ED,EXP,EXPSQ,WKS,OCC, SOUTH,SMSA,UNION) ED (MS, FEM) Reduced Form: LWAGE[ ED (MS, FEM), EXP,EXPSQ,WKS,OCC, SOUTH,SMSA,UNION ]

Two Stage Least Squares Strategy Reduced Form: LWAGE[ ED (MS, FEM,X), EXP,EXPSQ,WKS,OCC, SOUTH,SMSA,UNION ] Strategy (1) Purge ED of the influence of everything but MS, FEM (and the other variables). Predict ED using all exogenous information in the sample (X and Z). (2) Regress LWAGE on this prediction of ED and everything else. Standard errors must be adjusted for the predicted ED