Part 17: Nonlinear Regression 17-1/26 Econometrics I Professor William Greene Stern School of Business Department of Economics

Part 17: Nonlinear Regression 17-2/26 Econometrics I Part 17 – Nonlinear Regression

Part 17: Nonlinear Regression 17-3/26 Nonlinear Regression  Nonlinearity and Nonlinear Models  Estimation Criterion  Iterative Algorithm

Part 17: Nonlinear Regression 17-4/26 Nonlinear Regression What makes a regression model “nonlinear?” Nonlinear functional form? Regression model: y i = f(x i,  ) +  i Not necessarily: y i = exp(α) + β 2 *log(x i )+  i β 1 = exp(α) y i = exp(α)x i β 2 exp(  i ) is “loglinear” Models can be nonlinear in the functional form of the relationship between y and x, and not be nonlinear for purposes here. We will redefine “nonlinear” shortly, as we proceed.

Part 17: Nonlinear Regression 17-5/26 Least Squares Least squares: Minimize wrt  S(  ) = ½  i {y i - E[y i |x i,  ]} 2 = ½  i [y i - f(x i,  )] 2 = ½  i e i 2 First order conditions:  S(  )/  = 0  {½  i [y i - f(x i,  )] 2 }/  = ½  i (-2)[y i - f(x i,  )]  f(x i,  )/  = -  i e i x i 0 = 0 (familiar?) There is no explicit solution, b = f(data) like LS. (Nonlinearity of the FOC defines nonlinear model)

Part 17: Nonlinear Regression 17-6/26 Example How to solve this kind of set of equations: Example, y i =  0 +  1x i  2 +  i.  [ ½  i e i 2 ]/  0 =  i (-1) (y i -  0 -  1x i  2 ) 1 = 0  [ ½  i e i 2 ]/  1 =  i (-1) (y i -  0 -  1 x i  2 ) x i  2 = 0  [ ½  i e i 2 ]/  2 =  i (-1) (y i -  0 -  1 x i  2 )  1 x i  2 lnx i = 0 Nonlinear equations require a nonlinear solution. This defines a nonlinear regression model. I.e., when the first order conditions are not linear in . (!!!) Check your understanding. What does this produce if f(x i,  ) = x i  ? (I.e., a linear model)

Part 17: Nonlinear Regression 17-7/26 The Linearized Regression Model Linear Taylor series: y = f(x i,  ) + . Expand the regression around some point,  *. f(x i,  )  f(x i,  * ) +  k [  f(x i,  * )/  k * ](  k -  k * ) = f(x i,  * ) +  k x ik 0 (  k -  k * ) = [f(x i,  * ) -  k x ik 0  k * ] +  k x i 0  k = f 0 +  k x ik 0  k which looks linear. x ik 0 = the derivative wrt  k evaluated at  * The ‘pseudo-regressors’ are the derivative functions in the linearized model.

Part 17: Nonlinear Regression 17-8/26 Estimating Asy.Var[b] Computing the asymptotic covariance matrix for the nonlinear least squares estimator using the pseudo regressors and the sum of squares.

Part 17: Nonlinear Regression 17-9/26 Gauss-Marquardt Algorithm Given a coefficient vector b(m), at step m, find the vector for step m+1 by b(m+1) = b(m) + [X 0 (m)X 0 (m)] -1 X 0 (m)e 0 (m) Columns of X 0 (m) are the derivatives,  f(x i,b(m))/  b(m) e 0 = vector of residuals, y - f[x,b(m)] “Update” vector is the slopes in the regression of the residuals on the pseudo-regressors. Update is zero when they are orthogonal. (Just like LS)

Part 17: Nonlinear Regression 17-10/26

Part 17: Nonlinear Regression 17-11/26

Part 17: Nonlinear Regression 17-12/26 The NIST Dan Wood Application Y X y =  1 x  2 +  x i 0 = [x  2,  1 x  2 logx ]

Part 17: Nonlinear Regression 17-13/26 Dan Wood Solution

Part 17: Nonlinear Regression 17-14/26 Iterations NLSQ;LHS=Y ;FCN=B1*X^B2 ;LABELS=B1,B2 ;MAXIT=500;TLF;TLB;OUTPUT=1;DFC ;START=1,5 $ Begin NLSQ iterations. Linearized regression. Iteration= 1; Sum of squares= ; Gradient = Iteration= 2; Sum of squares= ; Gradient = Iteration= 3; Sum of squares= E-01; Gradient = E-01 Iteration= 4; Sum of squares= E-02; Gradient = E-05 Iteration= 5; Sum of squares= E-02; Gradient = E-11 Iteration= 6; Sum of squares= E-02; Gradient = E-15 Iteration= 7; Sum of squares= E-02; Gradient = E-19 Iteration= 8; Sum of squares= E-02; Gradient = E-23 Convergence achieved

Part 17: Nonlinear Regression 17-15/26 Results User Defined Optimization Nonlinear least squares regression LHS=Y Mean = Standard deviation = Number of observs. = 6 Model size Parameters = 2 Degrees of freedom = 4 Residuals Sum of squares = E-02 Standard error of e = Fit R-squared = Adjusted R-squared = Model test F[ 1, 4] (prob) = (.0000) Not using OLS or no constant. Rsqrd & F may be < | Standard Prob. 95% Confidence UserFunc| Coefficient Error z |z|>Z* Interval B1|.76886*** B2| *** Note: ***, **, * ==> Significance at 1%, 5%, 10% level

Part 17: Nonlinear Regression 17-16/26 NLS Solution The pseudo regressors and residuals at the solution are: X10 X20 X30 1 x  2  1 x  2 lnx e X0e0 = D D D-10

Part 17: Nonlinear Regression 17-17/26 Application: Doctor Visits  German Individual Health Care data: N=27,236  Model for number of visits to the doctor

Part 17: Nonlinear Regression 17-18/26 Conditional Mean and Projection Notice the problem with the linear approach. Negative predictions.

Part 17: Nonlinear Regression 17-19/26 Nonlinear Model Specification Nonlinear Regression Model y=exp(β’x) + ε X =one,age,health_status, married, education, household_income, kids

Part 17: Nonlinear Regression 17-20/26 NLS Iterations --> nlsq;lhs=docvis;start=0,0,0,0,0,0,0;labels=k_b;fcn=exp(b1'x);maxit=25;out... Begin NLSQ iterations. Linearized regression. Iteration= 1; Sum of squares= ; Gradient= Iteration= 2; Sum of squares= E+11 ; Gradient= E+11 Iteration= 3; Sum of squares= E+10 ; Gradient= E+10 Iteration= 4; Sum of squares= ; Gradient= Iteration= 5; Sum of squares= ; Gradient= Iteration= 6; Sum of squares= ; Gradient= Iteration= 7; Sum of squares= ; Gradient= Iteration= 8; Sum of squares= ; Gradient= Iteration= 9; Sum of squares= ; Gradient= Iteration= 10; Sum of squares= ; Gradient= Iteration= 11; Sum of squares= ; Gradient= E-03 Iteration= 12; Sum of squares= ; Gradient= E-05 Iteration= 13; Sum of squares= ; Gradient= E-07 Iteration= 14; Sum of squares= ; Gradient= E-08 Iteration= 15; Sum of squares= ; Gradient= E-10 (The blue iterations take place within the ‘hidden digits.’)

Part 17: Nonlinear Regression 17-21/26 Nonlinear Regression Results

Part 17: Nonlinear Regression 17-22/26 Partial Effects in the Nonlinear Model

Part 17: Nonlinear Regression 17-23/26 Delta Method for Asymptotic Variance of the Slope Estimator

Part 17: Nonlinear Regression 17-24/26 Computing the Slopes Namelist;x=one,age,hsat,married,educ,hhninc,hhkids$ Calc ; k=col(x)$ Nlsq ; lhs=docvis;start=0,0,0,0,0,0,0 ; labels=k_b;fcn=exp(b1'x)$ Partials ; function=exp(b1'x) ; parameters=b ; covariance = varb ; labels=k_b ; effects : x ; summary [; means] $

Part 17: Nonlinear Regression 17-25/26 Partial Effects: PEA vs. APE

Part 17: Nonlinear Regression 17-26/26 What About Just Using LS? |Variable | Coefficient | Standard Error |b/St.Er.|P[|Z|>z] | Mean of X| Least Squares Coefficient Estimates AGE NEWHSAT MARRIED EDUC HHNINC HHKIDS Estimated Partial Effects ME_ ME_ ME_ ME_ ME_ ME_