1. AdaBoost Reference Yoav Freund and Robert E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119-139, 1997.

2. “Two heads are better than one.” 三人寄れば文殊の知恵 Boosting is a technique of constructing a strong classifier by combining a collection of weak classifiers.

3. T1 T2 T3 T4 目標属性 (Objective Attribute) 1 0 1 11 1 1 1 11 1 1 1 00 1 0 1 00 1 1 0 10 1 0 0 10 1 1 0 10 0 1 0 11 0 0 1 11 0 1 0 1 1 0 0 0 11 0 0 1 00 0 1 0 00 0 0 1 00 Training dataset

4. Basic Idea of AdaBoost Initially assign an equal weight to each record. Iterate the following steps: 1.Generate a hypothesis the error ratio of which is smaller than that of random guessing. 2. Increase weights of mis-predicted records relatively higher than the others. We call a classifier “a hypothesis” following the terms in the cited paper.

5. T1 T2 T3 T4 Ob Weightif T1=1 New then Ob=0 Weight else Ob=1 1 0 1 110 1 1 1 110 1 1 1 000 1 0 1 000 1 1 0 100 1 0 0 100 1 1 0 100 hypothesis represents the degree of the weight. Size of

6. T1 T2 T3 T4 Ob Weightif T3=1 New then Ob=1 Weight else Ob=0 1 0 1 111 1 1 1 111 1 1 1 001 1 0 1 001 1 1 0 100 1 0 0 100 1 1 0 100 another hypothesis

7. T1 T2 T3 T4 Ob Weightif T4=1 New then Ob=1 Weight else Ob=0 1 0 1 111 1 1 1 111 1 1 1 000 1 0 1 000 1 1 0 101 1 0 0 101 1 1 0 101 another hypothesis

8. AdaBoost performs weighted majority voting by hypotheses. Hypotheses if T1=1 if T3=1 if T4=1 Simple then Ob=0 then Ob=1 then Ob=1 Majority T1 T2 T3 T4 Ob else Ob=1 else Ob=0else Ob=0Voting 1 0 1 110 111 1 1 1 110 111 1 1 1 000 100 1 0 1 000 100 1 1 0 100 010 1 0 0 100 010 1 1 0 100 010

9. Input Training dataset Initial weight Weak learner WeakLearn that always ouputs a hypothesis whose error ratio is less than ½. T: number of iterations to generate weak hypothesis for each i =1,2, …, N Details of AdaBoost

10. 2: Call WeakLearn to generate such a weak hypothesis h t that 1: Compute the distribution p i t of each record by normalizing weights 3: Revise weights

11. Revision of weights

12. Output: Final hypothesis h f (Weighted Majority Voting among h t )

13. The error of h f for the initial distribution.

14. Lemma 2

16. Q.E.D