1. Evidence Contrary to the Statistical View of Boosting David Mease & Abraham Wyner

2. What is the Statistical View? The idea presented in – J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: A statistical view of boosting. Annals of Statistics, 28:337–374, 2000a – Ada Boost is similar to LogitBoost Regression

3. The Challenge Centers on areas where the paper’s view fails to explain important characteristics of Ada Boost – The statistical view of boosting focuses only on one aspect of the algorithm - the optimization. Does not explain why the statistical view doesn’t work, merely presents evidence to the contrary

4. Areas of Deficiency for Statistical View Stagewise nature of the algorithm Empirical variance reduction that can be observed on hold out samples – variance reduction seems to happen accidently. Strong resistance to over fitting of Ada Boost which is lost in regression model

5. Practical Advice AdaBoost is one of the most successful boosting algorithms Do not assume that newer, regularized and modified versions of boosting are necessarily better Try standard AdaBoost along with these newer algorithms If classification is the goal, monitor the misclassification error on hold out (or cross-validation) samples Much of the evidence presented is counter-intuitive – keep an open mind when experimenting with AdaBoost – If stumps are causing overfitting, be willing to try larger trees – Intuition may suggest the larger trees will overfit, but we have seen that is not necessarily true