Shared Ensemble Learning using Multi-trees 전자전기컴퓨터공학과 G 김영제 Database Lab

Introduction What is a decision tree? Each node in the tree specifies a test for some attribute of the instance Each branch corresponds to an attribute value Each leaf node assigns a classification Decision Tree for PlayTennis

Cost Associated with Machine Learning Generation costs Computational costs i.e. computer resource consumption Give better solutions for provided resources

Cost Associated with Machine Learning Application costs First Models are accurate in average This does not mean seamless, and confident Model can be highly accurate for frequent cases Extremely inaccurate for infrequent, critical situations i.e. diagnosis, fault detection

Cost Associated with Machine Learning Application costs Second Even accurate models can be useless If the purpose is to obtain some new knowledge not expressed in the form of rules the number of rules is too high The interpretation of results significant costs it may even be impossible

Construction of Decision Tree Tree construction Driven by a splitting criterion that selects the best split The selected split is applied to generate new branches The rest of splits are discarded Algorithm stops when the examples that fall into a branch belong to the same class

Construction of Decision Tree Pruning Removal of not useful parts of the tree in order to avoid over- fitting Pre-pruning performed during the construction of the tree Post-pruning performed by analyzing the leaves once the tree has been built

Merit and Demerit of Decision Tree Merit Allows the quick construction of a model Because decision trees are built in a eager way (greedy) Demerit It may produce bad models because of bad decisions

Multi-tree Structure Rejected splits are not removed But stored as suspended nodes Two new criteria required for the construction of a single decision tree Suspended node selection To populate a multi-tree, need to specify a criterion that selects one of the suspended nodes Selection of a model Select one or more comprehensible models according to a selection criterion

Multi-tree Structure

Shared Ensembles Combination Combination of a set of classifiers improves the accuracy of simple classifiers Combination methods Boosting, Bagging, Randomization, Stacking, Windowing The large amount of memory required to store Shared the common parts of the components of the ensemble Using the multi-tree

Shared Ensembles Combination

Original Good loser Bad loser MajorityDifference {40, 10, 30} {80, 0, 0} {40, 0, 0} {1, 0, 0} {0, -60, -20} {7, 2, 10} {0, 0, 19} {0, 0, 10} {0, 0, 1} {-5, -15, 1}

Experiments #DatasetSizeClasses Nom. Attr. Num. Attr. 1Balance-scale Cars Dermatology Ecoli Iris House-votes Monks Monks Monks New-thyroid Post-operative Soybean-small Tae Tic-tac Wine Information about datasets used in the experiments.

Experiments Arit.Sum.Prod.Max.Min. #Acc.Dev.Acc.Dev.Acc.Dev.Acc.Dev.Acc.Dev Geomean Comparison between fusion techniques

Experiments Max+OrigMax+GoodMax+BadMax+Majo.Max+Diff. #Acc.DevAcc.DevAcc.DevAcc.DevAcc.Dev Gmean Comparison between vector transformation methods

Experiments #Acc.Dev.Acc.Dev.Acc.Dev.Acc.Dev Gmean Influence of the size of the multi-tree

Experiments

References 54-escicucrri.pdf 54-escicucrri.pdf Shared Ensemble Learning using Multi-trees V. Estruch, C. Ferri, J. Hernandez-Orallo, M.J. Ramirez-Quintana Wikipedia

Thank you for listening