Lecture 05: Decision Trees

Slides:

Advertisements

Similar presentations

Chapter 7 Classification and Regression Trees

Advertisements

Decision Trees Decision tree representation ID3 learning algorithm

CHAPTER 9: Decision Trees

Machine Learning III Decision Tree Induction

CPSC 502, Lecture 15Slide 1 Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 15 Nov, 1, 2011 Slide credit: C. Conati, S.

Decision Tree Algorithm (C4.5)

Bab /44 Bab 4 Classification: Basic Concepts, Decision Trees & Model Evaluation Part 1 Classification With Decision tree.

Classification Techniques: Decision Tree Learning

Chapter 7 – Classification and Regression Trees

Chapter 7 – Classification and Regression Trees

Decision Tree Rong Jin. Determine Milage Per Gallon.

Decision Tree Algorithm

Lecture 5 (Classification with Decision Trees)

Decision Trees (2). Numerical attributes Tests in nodes are of the form f i > constant.

Example of a Decision Tree categorical continuous class Splitting Attributes Refund Yes No NO MarSt Single, Divorced Married TaxInc NO < 80K > 80K.

Machine Learning Lecture 10 Decision Trees G53MLE Machine Learning Dr Guoping Qiu1.

Comp 540 Chapter 9: Additive Models, Trees, and Related Methods

Ensemble Learning (2), Tree and Forest

Decision Tree Learning

INTRODUCTION TO MACHINE LEARNING David Kauchak CS 451 – Fall 2013.

Lecture Notes 4 Pruning Zhangxi Lin ISQS

1 Statistics 202: Statistical Aspects of Data Mining Professor David Mease Tuesday, Thursday 9:00-10:15 AM Terman 156 Lecture 11 = Finish ch. 4 and start.

Chapter 9 – Classification and Regression Trees

Lecture 7. Outline 1. Overview of Classification and Decision Tree 2. Algorithm to build Decision Tree 3. Formula to measure information 4. Weka, data.

Decision Tree Learning Debapriyo Majumdar Data Mining – Fall 2014 Indian Statistical Institute Kolkata August 25, 2014.

Business Intelligence and Decision Modeling Week 9 Customer Profiling Decision Trees (Part 2) CHAID CRT.

Longin Jan Latecki Temple University

CS690L Data Mining: Classification

Decision Trees Example of a Decision Tree categorical continuous class Refund MarSt TaxInc YES NO YesNo Married Single, Divorced < 80K> 80K Splitting.

DECISION TREE Ge Song. Introduction ■ Decision Tree: is a supervised learning algorithm used for classification or regression. ■ Decision Tree Graph:

Random Forests Ujjwol Subedi. Introduction What is Random Tree? ◦ Is a tree constructed randomly from a set of possible trees having K random features.

Lecture Notes for Chapter 4 Introduction to Data Mining

ECE 471/571 – Lecture 20 Decision Tree 11/19/15. 2 Nominal Data Descriptions that are discrete and without any natural notion of similarity or even ordering.

1 Illustration of the Classification Task: Learning Algorithm Model.

Classification and Regression Trees

Combining multiple learners Usman Roshan. Decision tree From Alpaydin, 2010.

COM24111: Machine Learning Decision Trees Gavin Brown

Eco 6380 Predictive Analytics For Economists Spring 2016 Professor Tom Fomby Department of Economics SMU.

Tree and Forest Classification and Regression Tree Bagging of trees Boosting trees Random Forest.

Review of Decision Tree Learning Bamshad Mobasher DePaul University Bamshad Mobasher DePaul University.

CSE573 Autumn /11/98 Machine Learning Administrative –Finish this topic –The rest of the time is yours –Final exam Tuesday, Mar. 17, 2:30-4:20.

Combining Models Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya.

Machine Learning Inductive Learning and Decision Trees

DECISION TREES An internal node represents a test on an attribute.

Decision Trees an introduction.

Classification Algorithms

Decision Trees: Another Example

Ch9: Decision Trees 9.1 Introduction A decision tree:

Lecture 04: Logistic Regression

Decision Tree Saed Sayad 9/21/2018.

Introduction to Data Mining, 2nd Edition by

ECE 471/571 – Lecture 12 Decision Tree.

Introduction to Data Mining, 2nd Edition by

Decision Trees.

Machine Learning: Lecture 3

Lecture 18: Bagging and Boosting

Classification with Decision Trees

Play Tennis ????? Day Outlook Temperature Humidity Wind PlayTennis

Decision Trees.

Statistical Learning Dong Liu Dept. EEIS, USTC.

COMP61011 : Machine Learning Decision Trees

Lecture 06: Bagging and Boosting

Classification with CART

Artificial Intelligence 9. Perceptron

Junheng, Shengming, Yunsheng 10/19/2018

INTRODUCTION TO Machine Learning 2nd Edition

A task of induction to find patterns

A task of induction to find patterns

STT : Intro. to Statistical Learning

COP5577: Principles of Data Mining Fall 2008 Lecture 4 Dr

Presentation transcript:

Lecture 05: Decision Trees CS480/680: Intro to ML Lecture 05: Decision Trees 09/25/18 Yao-Liang Yu

Trees Recalled 09/25/18 Yao-Liang Yu

Example: EnjoySport? Outlook Humidity Wind Yes No Overcast Sunny Rain No High Normal Strong Weak Decision trees can represent any boolean function 09/25/18 Yao-Liang Yu

Classification And Regression Tree decision: splitting a variable classification: majority of label / prob. of label regression: average of responses training data 09/25/18 Yao-Liang Yu

LEARNing a Decision Tree Which variables to split in each stage? What threshold to use? When to stop? What to put at the leaves? setup a cost/objective NP-Hard… regularization: early stopping / pruning regression / classification / other 09/25/18 Yao-Liang Yu

Algorithm 09/25/18 Yao-Liang Yu

Growing a tree y X 09/25/18 Yao-Liang Yu

Growing a tree Splitting based on x Evaluation can be based on y y X 09/25/18 Yao-Liang Yu

Growing a tree y X 09/25/18 Yao-Liang Yu

Growing a tree y X 09/25/18 Yao-Liang Yu

Growing a tree y X 09/25/18 Yao-Liang Yu

Growing a tree y X 09/25/18 Yao-Liang Yu

Growing a tree y X 09/25/18 Yao-Liang Yu

Growing a tree y X 09/25/18 Yao-Liang Yu

Growing a tree 09/25/18 Yao-Liang Yu

Which and How For categorical features, simply try each assuming features are ordinal cost / loss partition training data into left and right greedily choose a variable to split greedily choose a threshold to split For categorical features, simply try each What should Tj be? 09/25/18 Yao-Liang Yu

Stopping criterion Maximum depth exceeded Maximum running time exceeded All children nodes are sufficiently homogeneous All children nodes have too few training examples Cross-validation Reduction in cost is small 09/25/18 Yao-Liang Yu

Regression cost Can of course use other loss than least-squares Can also fit any regression model on D average of yi in D stored in leaves 09/25/18 Yao-Liang Yu

Classification cost Misclassification error: Entropy: Gini index: majority vote Misclassification error: Entropy: Gini index: 09/25/18 Yao-Liang Yu

Comparison 09/25/18 Yao-Liang Yu

Example 09/25/18 Yao-Liang Yu

Type or Patrons A better feature split should lead to nearly all positives or all negatives 09/25/18 Yao-Liang Yu

Result 09/25/18 Yao-Liang Yu

Pruning Early stopping can be myopic Grow a full tree and then prune in bottom-up 09/25/18 Yao-Liang Yu

Decision Stump A binary tree with depth 1 Eye features low high No Yes A binary tree with depth 1 Performs classification based on one feature Easy to train but underfits; interprettable 09/25/18 Yao-Liang Yu

Questions? 09/25/18 Yao-Liang Yu