INTRODUCTION TO MACHINE LEARNING 3RD EDITION ETHEM ALPAYDIN © The MIT Press, 2014 Lecture.

Slides:

Advertisements

Similar presentations

CHAPTER 9: Decision Trees

Advertisements

CPSC 502, Lecture 15Slide 1 Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 15 Nov, 1, 2011 Slide credit: C. Conati, S.

Paper By - Manish Mehta, Rakesh Agarwal and Jorma Rissanen

Huffman code and ID3 Prof. Sin-Min Lee Department of Computer Science.

RIPPER Fast Effective Rule Induction

C4.5 - pruning decision trees

Classification Techniques: Decision Tree Learning

Chapter 7 – Classification and Regression Trees

Chapter 7 – Classification and Regression Trees

Decision Trees Instructor: Qiang Yang Hong Kong University of Science and Technology Thanks: Eibe Frank and Jiawei Han.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Example of a Decision Tree categorical continuous class Splitting Attributes Refund Yes No NO MarSt Single, Divorced Married TaxInc NO < 80K > 80K.

Classification.

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Machine Learning Lecture 10 Decision Trees G53MLE Machine Learning Dr Guoping Qiu1.

Decision Tree Learning

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Fall 2004 TDIDT Learning CS478 - Machine Learning.

Lecture Notes 4 Pruning Zhangxi Lin ISQS

Machine Learning Chapter 3. Decision Tree Learning

Mohammad Ali Keyvanrad

For Wednesday No new reading Homework: –Chapter 18, exercises 3, 4, 7.

INTRODUCTION TO MACHINE LEARNING 3RD EDITION ETHEM ALPAYDIN © The MIT Press, Lecture.

Chapter 9 – Classification and Regression Trees

Review - Decision Trees

For Friday No reading No homework. Program 4 Exam 2 A week from Friday Covers 10, 11, 13, 14, 18, Take home due at the exam.

Decision Trees Jyh-Shing Roger Jang ( 張智星 ) CSIE Dept, National Taiwan University.

Comparing Univariate and Multivariate Decision Trees Olcay Taner Yıldız Ethem Alpaydın Department of Computer Engineering Bogazici University

Decision Tree Learning Debapriyo Majumdar Data Mining – Fall 2014 Indian Statistical Institute Kolkata August 25, 2014.

Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, a Machine Learning.

Data Mining Practical Machine Learning Tools and Techniques Chapter 4: Algorithms: The Basic Methods Section 4.3: Decision Trees Rodney Nielsen Many of.

For Wednesday No reading Homework: –Chapter 18, exercise 6.

CS690L Data Mining: Classification

For Monday No new reading Homework: –Chapter 18, exercises 3 and 4.

MACHINE LEARNING 10 Decision Trees. Motivation  Parametric Estimation  Assume model for class probability or regression  Estimate parameters from all.

CS 5751 Machine Learning Chapter 3 Decision Tree Learning1 Decision Trees Decision tree representation ID3 learning algorithm Entropy, Information gain.

1 Decision Tree Learning Original slides by Raymond J. Mooney University of Texas at Austin.

INTRODUCTION TO MACHINE LEARNING 3RD EDITION ETHEM ALPAYDIN © The MIT Press, Lecture.

INTRODUCTION TO MACHINE LEARNING 3RD EDITION ETHEM ALPAYDIN © The MIT Press, Lecture.

Data Mining Practical Machine Learning Tools and Techniques By I. H. Witten, E. Frank and M. A. Hall Chapter 6.2: Classification Rules Rodney Nielsen Many.

Decision Trees Example of a Decision Tree categorical continuous class Refund MarSt TaxInc YES NO YesNo Married Single, Divorced < 80K> 80K Splitting.

DECISION TREE Ge Song. Introduction ■ Decision Tree: is a supervised learning algorithm used for classification or regression. ■ Decision Tree Graph:

Decision Tree Learning

Machine Learning: Decision Trees Homework 4 assigned courtesy: Geoffrey Hinton, Yann LeCun, Tan, Steinbach, Kumar.

ECE 471/571 – Lecture 20 Decision Tree 11/19/15. 2 Nominal Data Descriptions that are discrete and without any natural notion of similarity or even ordering.

Classification and Regression Trees

Outline Decision tree representation ID3 learning algorithm Entropy, Information gain Issues in decision tree learning 2.

Data Mining CH6 Implementation: Real machine learning schemes(2) Reporter: H.C. Tsai.

Decision Tree Learning DA514 - Lecture Slides 2 Modified and expanded from: E. Alpaydin-ML (chapter 9) T. Mitchell-ML.

Review of Decision Tree Learning Bamshad Mobasher DePaul University Bamshad Mobasher DePaul University.

CSE573 Autumn /11/98 Machine Learning Administrative –Finish this topic –The rest of the time is yours –Final exam Tuesday, Mar. 17, 2:30-4:20.

Chapter 6 Decision Tree.

DECISION TREES An internal node represents a test on an attribute.

Decision Trees an introduction.

C4.5 - pruning decision trees

Artificial Intelligence

Decision Trees (suggested time: 30 min)

Ch9: Decision Trees 9.1 Introduction A decision tree:

Data Science Algorithms: The Basic Methods

Introduction to Data Mining, 2nd Edition by

Introduction to Data Mining, 2nd Edition by

INTRODUCTION TO Machine Learning

INTRODUCTION TO Machine Learning 2nd Edition

INTRODUCTION TO Machine Learning 3rd Edition

INTRODUCTION TO Machine Learning

INTRODUCTION TO Machine Learning

Presentation transcript:

INTRODUCTION TO MACHINE LEARNING 3RD EDITION ETHEM ALPAYDIN © The MIT Press, Lecture Slides for

CHAPTER 9: DECISION TREES

Tree Uses Nodes and Leaves 3

Divide and Conquer 4  Internal decision nodes  Univariate: Uses a single attribute, x i Numeric x i : Binary split : x i > w m Discrete x i : n-way split for n possible values  Multivariate: Uses all attributes, x  Leaves  Classification: Class labels, or proportions  Regression: Numeric; r average, or local fit  Learning is greedy; find the best split recursively (Breiman et al, 1984; Quinlan, 1986, 1993)

Classification Trees (ID3,CART,C4.5) 5  For node m, N m instances reach m, N i m belong to C i  Node m is pure if p i m is 0 or 1  Measure of impurity is entropy

 If node m is pure, generate a leaf and stop, otherwise split and continue recursively  Impurity after split: N mj of N m take branch j. N i mj belong to C i  Find the variable and split that min impurity (among all variables -- and split positions for numeric variables) Best Split 6

7

 Error at node m:  After splitting: Regression Trees 8

9 Model Selection in Trees

Pruning Trees 10  Remove subtrees for better generalization (decrease variance)  Prepruning: Early stopping  Postpruning: Grow the whole tree then prune subtrees that overfit on the pruning set  Prepruning is faster, postpruning is more accurate (requires a separate pruning set)

Rule Extraction from Trees 11 C4.5Rules (Quinlan, 1993 )

Learning Rules 12  Rule induction is similar to tree induction but  tree induction is breadth-first,  rule induction is depth-first; one rule at a time  Rule set contains rules; rules are conjunctions of terms  Rule covers an example if all terms of the rule evaluate to true for the example  Sequential covering: Generate rules one at a time until all positive examples are covered  IREP (Fürnkrantz and Widmer, 1994), Ripper (Cohen, 1995)

13

14

Multivariate Trees 15