1. BAGGING ALGORITHM, ONLINE BOOSTING AND VISION Se – Hoon Park

2. ENSEMBLE METHOD  Multiple ‘base’ models (classifiers, regressors), each covers a different part (region) of th e input space.

3. BAGGING ALGORITHM

4.  Given  Training set of N examples  A class of learning models(decision trees, neural networks, …)  Method  Train multiple(k) models on different samples(data splits)  Predict (test) by averaging or majority voting the results of k models  Goal  Improve the accuracy of one model by using its multiple copies  Average of misclassification errors on different data splits gives a better estimate of the predictive ability of a learning method

5. BAGGING ALGORITHM  Training  Randomly sample with replacement N samples from the training set  Train a chosen “base model”(neural network, decision tree) on the samples  Test  Start all trained base models  Predict by combining results of all trained models  Regression : averaging  Classification : a majority vote

6. BAGGING ALGORITHM  Bias vs variance Under fitting High bias Small variance Over fitting Small bias High variance

7. BAGGING ALGORITHM  Main property of bagging  Bagging decreases variance of the base model without changing the bias because of averaging  Bagging is useful when applied with an over-fitted base model  It does not help much  High bias, when the base model is robust to the changes in the training data

8. ONLINE BOOSTING AND VISION Helmut Grabner and Horst Bischof, CVPR 2006 Institute for Computer Graphics and Vision, Graz University of Technology

9. OFFLINE BOOSTING

11. ONLINE BOOSTING FOR FEATURE SELECTION

14. APPLICATION : BACKGROUND MODEL

16. APPLICATION : TRACKING

17. APPLICATION : OBJECT DETECTION  Detection using offline boosting  Trained classifier scan over the whole image at multiple locations and scales  Detection using online boosting  All patches where motion detection has detected an object are selected as positive examples.  10% false positives a robust reconstructive representation (PCA on appearance and shape) is computed from the output of the motion detector.  Thus, the false positives can be filtered out and may be used as negative examples

18. APPLICATION : OBJECT DETECTION  evaluation Initial classifierAfter 300 frameAfter 1200 frame TP : true positive FP : false positive nP : number of positive