Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd.

Topics Separation in Logistic Regression Approaches to Separation Firth’s Bias-reduced GLMs firthlogit: syntax and examples Caveats and to-do’s

Separation in Logistic Regression

Complete Separation Dataset adapted from D. W. Hosmer and S. Lemeshow, Applied Logistic Regression Second Edition. (New York: John Wiley & Sons, 2000), pp. 138–39.

Quasi-complete Separation Dataset adapted from D. W. Hosmer and S. Lemeshow, Applied Logistic Regression Second Edition. (New York: John Wiley & Sons, 2000), pp. 138–39.

Approaches to Separation Remove predictors Pool groups Remove interaction terms Gather more data Use alternatives

Exact Logistic Regression

But . . . Dataset from D. M. Potter. 2005. A permutation test for inference in logistic regression with small- and moderate-sized data sets. Statistics in Medicine 24:693–708.

[19] D. Firth. 1993. Bias reduction in maximum likelihood estimates [19] D. Firth. 1993. Bias reduction in maximum likelihood estimates. Biometrika 80:27–38.

firthlogit

But . . . redux

But . . . redux, continued

Profile Likelihood Ratio CIs

Caveats Profile Penalized Likelihood CIs Small-sample Behavior

G. Heinze and M. Ploner, A SAS macro, S-PLUS library and R package to perform logistic regression without convergence problems. Technical Report 2/2004. Medical University of Vienna. p. 36.

To-do’s Profile Penalized Likelihood CIs Modify ml d0