1. Statistical Inference and Regression Analysis: GB.3302.30 Professor William Greene Stern School of Business IOMS Department Department of Economics

2. Statistics and Data Analysis Part 10 – Advanced Topics

4. Advanced topics Nonlinear Least Squares Nonlinear Models – ML Estimation Poisson Regression Binary Choice End of course. 4

5. Statistics and Data Analysis Nonlinear Least Squares

7. Lanczos 1 Data

8. Nonlinear Regression

9. Nonlinear Least Squares There are no explicit solutions to these equations in the form of b i = a function of (y,x).

10. Strategy for Nonlinear LS

11. NLS Strategy Pick b A. Compute y i 0 and x i 0 B. Regress y i 0 on x i 0 This obtains a new b Return to step A or exit if the new b is the same as the old b

14. Lanczos 1 First Iteration Now, repeat the iteration using this as b

15. This is the correct answer

16. Gauss-Marquardt Algorithm Starting with b 0 A. Compute regressors x i 0 Compute residuals e i 0 = y i – f(x i,b 0 ) B. New b 1 = b 0 + slopes in regression of e i 0 on x i 0 Return to A. or exit if estimates have converged. This is equivalent to our earlier method.

17. Statistics and Data Analysis Maximum Likelihood: Poisson

18. Application: Doctor Visits German Individual Health Care data: N=27,236 Model for number of visits to the doctor: Poisson regression Age, Health Satisfaction, Marital Status, Income, Kids

19. Poisson Regression

20. Nonlinear Least Squares

21. Maximum Likelihood Estimation This defines a class of estimators based on the particular distribution assumed to have generated the observed random variable. The main advantage of ML estimators is that among all Consistent Asymptotically Normal Estimators, MLEs have optimal asymptotic properties.

22. Setting up the MLE The distribution of the observed random variable is written as a function of the parameters to be estimated P(y i |data,β) = Probability density | parameters. The likelihood function is constructed from the density Construction: Joint probability density function of the observed sample of data – generally the product when the data are a random sample.

23. Likelihood for the Poisson Regression

24. Newton’s Method

26. Properties of the MLE Consistent: Not necessarily unbiased, however Asymptotically normally distributed: Proof based on central limit theorems Asymptotically efficient: Among the possible estimators that are consistent and asymptotically normally distributed Invariant: The MLE of g(  ) is g(the MLE of  )

27. Computing the Asymptotic Variance We want to estimate {-E[H]} -1 Three ways: (1) Just compute the negative of the actual second derivatives matrix and invert it. (2) Insert the maximum likelihood estimates into the known expected values of the second derivatives matrix. Sometimes (1) and (2) give the same answer (for example, in the Poisson regression model). (3) Since E[H] is the variance of the first derivatives, estimate this with the sample variance (i.e., mean square) of the first derivatives. This will almost always be different from (1) and (2). Since they are estimating the same thing, in large samples, all three will give the same answer.

28. Poisson Regression Iterations

29. MLE NLS

30. Using the Model. Partial Effects

31. Effect of Income Depends on Age

32. Effect of Income | Age

33. Statistics and Data Analysis Binary Choice

34. Case Study: Credit Modeling 1992 American Express analysis of Application process: Acceptance or rejection; Y = 0 (reject) or 1 (accept). Cardholder behavior Loan default (D = 0 or 1). Average monthly expenditure (E = $/month) General credit usage/behavior (C = number of charges) 13,444 applications in November, 1992

35. Proportion for Bernoulli In the AmEx data, the true population acceptance rate is 0.7809 =  Y = 1 if application accepted, 0 if not. E[y] =  E[(1/N)Σ i y i ] = p accept = . This is the estimator 35

36. Some Evidence = Homeowners Does the acceptance rate depend on home ownership?

37. A Test of Independence In the credit card example, are Own/Rent and Accept/Reject independent? Hypothesis: Prob(Ownership) and Prob(Acceptance) are independent Formal hypothesis, based only on the laws of probability: Prob(Own,Accept) = Prob(Own)Prob(Accept) (and likewise for the other three possibilities. Rejection region: Joint frequencies that do not look like the products of the marginal frequencies.

38. Contingency Table Analysis The Data: Frequencies Reject Accept Total Rent 1,845 5,469 7,214 Own 1,100 5,030 6,630 Total 2,945 10,499 13,444 Step 1: Convert to Actual Proportions Reject Accept Total Rent 0.13724 0.40680 0.54404 Own 0.08182 0.37414 0.45596 Total 0.21906 0.78094 1.00000

39. Independence Test Step 2: Expected proportions assuming independence: If the factors are independent, then the joint proportions should equal the product of the marginal proportions. [Rent,Reject] 0.54404 x 0.21906 = 0.11918 [Rent,Accept] 0.54404 x 0.78094 = 0.42486 [Own,Reject] 0.45596 x 0.21906 = 0.09988 [Own,Accept] 0.45596 x 0.78094 = 0.35606

40. Comparing Actual to Expected It appears that the acceptance rate is dependent on home ownership

41. When is the Chi Squared Large? Critical values from chi squared table Degrees of freedom = (R-1)(C-1). Critical chi squared D.F..05.01 1 3.84 6.63 2 5.99 9.21 3 7.81 11.34 4 9.49 13.28 5 11.07 15.09 6 12.59 16.81 7 14.07 18.48 8 15.51 20.09 9 16.92 21.67 10 18.31 23.21

42. Analyzing Default Do renters default more often (at a different rate) than owners? To investigate, we study the cardholders (only) DEFAULT OWNRENT 0 1 All 0 4854 615 5469 46.23 5.86 52.09 1 4649 381 5030 44.28 3.63 47.91 All 9503 996 10499 90.51 9.49 100.00

43. Hypothesis Test

44. More Formal Model of Acceptance and Default

45. Probability Models zizi

46. Likelihood Function

47. American Express, 1992

48. Logistic Model for Acceptance

49. Probit Default Model

51. Think statistically Build models Thank you.