CSE 573 Introduction to Artificial Intelligence Decision Trees

Slides:

Advertisements

Similar presentations

On the Optimality of Probability Estimation by Random Decision Trees Wei Fan IBM T.J.Watson.

Advertisements

Random Forest Predrag Radenković 3237/10

Ensemble Learning Reading: R. Schapire, A brief introduction to boosting.

Machine Learning III Decision Tree Induction

My name is Dustin Boswell and I will be presenting: Ensemble Methods in Machine Learning by Thomas G. Dietterich Oregon State University, Corvallis, Oregon.

Data Mining Classification: Alternative Techniques

Ensemble Methods An ensemble method constructs a set of base classifiers from the training data Ensemble or Classifier Combination Predict class label.

Data Mining Classification: Alternative Techniques

Boosting Ashok Veeraraghavan. Boosting Methods Combine many weak classifiers to produce a committee. Resembles Bagging and other committee based methods.

Longin Jan Latecki Temple University

Machine Learning CSE 454. Administrivia PS1 due next tues 10/13 Project proposals also due then Group meetings with Dan Signup out shortly.

Learning from Observations Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 18 Fall 2005.

Ensemble Learning: An Introduction

Machine Learning IV Ensembles CSE 473. © Daniel S. Weld 2 Machine Learning Outline Machine learning: Supervised learning Overfitting Ensembles of classifiers.

Introduction to Boosting Aristotelis Tsirigos SCLT seminar - NYU Computer Science.

Examples of Ensemble Methods

Machine Learning: Ensemble Methods

Sparse vs. Ensemble Approaches to Supervised Learning

Boosting Main idea: train classifiers (e.g. decision trees) in a sequence. a new classifier should focus on those cases which were incorrectly classified.

Ensemble Learning (2), Tree and Forest

For Better Accuracy Eick: Ensemble Learning

Ensembles of Classifiers Evgueni Smirnov

Issues with Data Mining

By Wang Rui State Key Lab of CAD&CG

Machine Learning Chapter 3. Decision Tree Learning

ENSEMBLE LEARNING David Kauchak CS451 – Fall 2013.

CS 391L: Machine Learning: Ensembles

Benk Erika Kelemen Zsolt

Learning from Observations Chapter 18 Through

Ensembles. Ensemble Methods l Construct a set of classifiers from training data l Predict class label of previously unseen records by aggregating predictions.

CLASSIFICATION: Ensemble Methods

ISQS 6347, Data & Text Mining1 Ensemble Methods. ISQS 6347, Data & Text Mining 2 Ensemble Methods Construct a set of classifiers from the training data.

CS 8751 ML & KDDDecision Trees1 Decision tree representation ID3 learning algorithm Entropy, Information gain Overfitting.

Machine Learning II Decision Tree Induction CSE 573.

CS 5751 Machine Learning Chapter 3 Decision Tree Learning1 Decision Trees Decision tree representation ID3 learning algorithm Entropy, Information gain.

Ensemble Methods in Machine Learning

Random Forests Ujjwol Subedi. Introduction What is Random Tree? ◦ Is a tree constructed randomly from a set of possible trees having K random features.

Classification Ensemble Methods 1

Ensemble Methods Construct a set of classifiers from the training data Predict class label of previously unseen records by aggregating predictions made.

Decision Trees IDHairHeightWeightLotionResult SarahBlondeAverageLightNoSunburn DanaBlondeTallAverageYesnone AlexBrownTallAverageYesNone AnnieBlondeShortAverageNoSunburn.

1 Universidad de Buenos Aires Maestría en Data Mining y Knowledge Discovery Aprendizaje Automático 4-Inducción de árboles de decisión (1/2) Eduardo Poggi.

Learning From Observations Inductive Learning Decision Trees Ensembles.

CSE 446: Ensemble Learning Winter 2012 Daniel Weld Slides adapted from Tom Dietterich, Luke Zettlemoyer, Carlos Guestrin, Nick Kushmerick, Padraig Cunningham.

Tree and Forest Classification and Regression Tree Bagging of trees Boosting trees Random Forest.

1 Machine Learning Lecture 8: Ensemble Methods Moshe Koppel Slides adapted from Raymond J. Mooney and others.

Machine Learning Reading: Chapter Classification Learning Input: a set of attributes and values Output: discrete valued function Learning a continuous.

Ensemble Classifiers.

Machine Learning: Ensemble Methods

Data Mining Practical Machine Learning Tools and Techniques

Reading: R. Schapire, A brief introduction to boosting

Bagging and Random Forests

6-3The Central Limit Theorem.

Machine Learning: Ensembles

Ensemble Learning Introduction to Machine Learning and Data Mining, Carla Brodley.

Issues in Decision-Tree Learning Avoiding overfitting through pruning

Data Mining Lecture 11.

CSE P573 Applications of Artificial Intelligence Decision Trees

A “Holy Grail” of Machine Learing

Data Mining Practical Machine Learning Tools and Techniques

Introduction to Data Mining, 2nd Edition

Machine Learning Chapter 3. Decision Tree Learning

Machine Learning Chapter 3. Decision Tree Learning

Machine Learning CSE 454.

Statistical Learning Dong Liu Dept. EEIS, USTC.

Ensemble learning.

Model Combination.

Ensemble learning Reminder - Bagging of Trees Random Forest

CS 391L: Machine Learning: Ensembles

Machine Learning: Lecture 5

Presentation transcript:

CSE 573 Introduction to Artificial Intelligence Decision Trees Henry Kautz Autumn 2003

Variable-sized hypothesis space Number of possible hypotheses grows with depth of tree

How can this algorithm be viewed as a state-space search problem?

Ensembles of Classifiers Idea: instead of training one classifier (decision tree) Train k classifiers and let them vote Only helps if classifiers disagree with each other Trained on different data Use different learning methods Amazing fact: can help a lot!

How voting helps If individual area is 0.3 Assume errors are independent Assume majority vote Probability majority is wrong = area under bionomial dist Prob 0.2 0.1 Number of classifiers in error If individual area is 0.3 Area under curve for 11 wrong is 0.026 Order of magnitude improvement!

Constructing Ensembles Bagging Run classifier k times on m examples drawn randomly with replacement from the original set of n examples Cross-validated committees Divide examples into k disjoint sets Train on k sets corresponding to original minus 1/k-th Boosting (Shapire) Maintain a probability distribution over set of training examples On each iteration, use distribution to sample Use error rate to modify distribution Create harder and harder learning problems

Summary Inductive learning Decision trees Ensembles Representation Tree growth Heuristics Overfitting and pruning Scaling up Ensembles