DECISION TREE Ge Song. Introduction ■ Decision Tree: is a supervised learning algorithm used for classification or regression. ■ Decision Tree Graph:

Slides:

Advertisements

Similar presentations

COMP3740 CR32: Knowledge Management and Adaptive Systems

Advertisements

Data Mining Lecture 9.

CHAPTER 9: Decision Trees

CPSC 502, Lecture 15Slide 1 Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 15 Nov, 1, 2011 Slide credit: C. Conati, S.

IT 433 Data Warehousing and Data Mining

Hunt’s Algorithm CIT365: Data Mining & Data Warehousing Bajuna Salehe

Decision Tree Approach in Data Mining

Data Mining Classification: Basic Concepts, Decision Trees, and Model Evaluation Lecture Notes for Chapter 4 Part I Introduction to Data Mining by Tan,

Decision Tree Algorithm (C4.5)

Deriving rules from data Decision Trees a.j.m.m (ton) weijters.

Classification Techniques: Decision Tree Learning

A Quick Overview By Munir Winkel. What do you know about: 1) decision trees 2) random forests? How could they be used?

Decision Trees IDHairHeightWeightLotionResult SarahBlondeAverageLightNoSunburn DanaBlondeTallAverageYesnone AlexBrownTallAverageYesNone AnnieBlondeShortAverageNoSunburn.

Decision Tree under MapReduce Week 14 Part II. Decision Tree.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Decision Tree Rong Jin. Determine Milage Per Gallon.

ID3 Algorithm Abbas Rizvi CS157 B Spring What is the ID3 algorithm? ID3 stands for Iterative Dichotomiser 3 Algorithm used to generate a decision.

Decision Tree Algorithm

Decision tree LING 572 Fei Xia 1/10/06. Outline Basic concepts Main issues Advanced topics.

More on Decision Trees. Numerical attributes Tests in nodes can be of the form x j > constant Divides the space into rectangles.

1 Classification with Decision Trees I Instructor: Qiang Yang Hong Kong University of Science and Technology Thanks: Eibe Frank and Jiawei.

Three kinds of learning

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

LEARNING DECISION TREES

Decision Trees (2). Numerical attributes Tests in nodes are of the form f i > constant.

Microsoft Enterprise Consortium Data Mining Concepts Introduction to Directed Data Mining: Decision Trees Prepared by David Douglas, University of ArkansasHosted.

DATA MINING AND MACHINE LEARNING Addison Euhus and Dan Weinberg.

Introduction to Directed Data Mining: Decision Trees

Decision tree LING 572 Fei Xia 1/16/06.

Chapter 7 Decision Tree.

Math – Getting Information from the Graph of a Function 1.

INTRODUCTION TO MACHINE LEARNING. $1,000,000 Machine Learning  Learn models from data  Three main types of learning :  Supervised learning  Unsupervised.

Mohammad Ali Keyvanrad

Decision Trees & the Iterative Dichotomiser 3 (ID3) Algorithm David Ramos CS 157B, Section 1 May 4, 2006.

LEARNING DECISION TREES Yılmaz KILIÇASLAN. Definition - I Decision tree induction is one of the simplest, and yet most successful forms of learning algorithm.

Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, a Machine Learning.

Decision Trees. Decision trees Decision trees are powerful and popular tools for classification and prediction. The attractiveness of decision trees is.

Classification and Prediction Compiled By: Umair Yaqub Lecturer Govt. Murray College Sialkot Readings: Chapter 6 – Han and Kamber.

Decision Trees. MS Algorithms Decision Trees The basic idea –creating a series of splits, also called nodes, in the tree. The algorithm adds a node to.

Machine Learning BY:Vatsal J. Gajera (09BCE010).  What is Machine Learning? It is a branch of artificial intelligence.It is a scientfic discipline concerned.

CS690L Data Mining: Classification

For Monday No new reading Homework: –Chapter 18, exercises 3 and 4.

Decision Trees. What is a decision tree? Input = assignment of values for given attributes –Discrete (often Boolean) or continuous Output = predicated.

Copyright © 2010 SAS Institute Inc. All rights reserved. Decision Trees Using SAS Sylvain Tremblay SAS Canada – Education SAS Halifax Regional User Group.

MACHINE LEARNING 10 Decision Trees. Motivation  Parametric Estimation  Assume model for class probability or regression  Estimate parameters from all.

Decision Trees Binary output – easily extendible to multiple output classes. Takes a set of attributes for a given situation or object and outputs a yes/no.

1 CSCI 3202: Introduction to AI Decision Trees Greg Grudic (Notes borrowed from Thomas G. Dietterich and Tom Mitchell) Intro AIDecision Trees.

Exercises Decision Trees In decision tree learning, the information gain criterion helps us select the best attribute to split the data at every node.

Decision Tree Learning Presented by Ping Zhang Nov. 26th, 2007.

COMP 2208 Dr. Long Tran-Thanh University of Southampton Decision Trees.

Presentation on Decision trees Presented to: Sir Marooof Pasha.

1 Classification: predicts categorical class labels (discrete or nominal) classifies data (constructs a model) based on the training set and the values.

Data Mining By Farzana Forhad CS 157B. Agenda Decision Tree and ID3 Rough Set Theory Clustering.

Classification and Regression Trees

CIS 335 CIS 335 Data Mining Classification Part I.

DECISION TREES Asher Moody, CS 157B. Overview  Definition  Motivation  Algorithms  ID3  Example  Entropy  Information Gain  Applications  Conclusion.

Outline Decision tree representation ID3 learning algorithm Entropy, Information gain Issues in decision tree learning 2.

1 By: Ashmi Banerjee (125186) Suman Datta ( ) CSE- 3rd year.

INTRODUCTION TO MACHINE LEARNING 3RD EDITION ETHEM ALPAYDIN © The MIT Press, Lecture.

Machine Learning Reading: Chapter Classification Learning Input: a set of attributes and values Output: discrete valued function Learning a continuous.

10. Decision Trees and Markov Chains for Gene Finding.

Decision Trees.

Ch9: Decision Trees 9.1 Introduction A decision tree:

Decision Trees Greg Grudic

Classification and Prediction

Statistical Learning Dong Liu Dept. EEIS, USTC.

INTRODUCTION TO Machine Learning

INTRODUCTION TO Machine Learning 2nd Edition

Presentation transcript:

DECISION TREE Ge Song

Introduction ■ Decision Tree: is a supervised learning algorithm used for classification or regression. ■ Decision Tree Graph: is a graph that uses a branching method to illustrate every possible outcome of a decision. input output Each internal node: test one discrete-valued attribute Xi Each branch from a node: selects one value for Xi Each leaf node: predict Y (or P(Y|X ∈ leaf))

ID3 Algorithm ■Calculate the entropy of every attribute using the data set ■Split the set into subsets using the attribute for which entropy is minimum (or, equivalently, information gain is maximum) ■Make a decision tree node containing that attribute ■Recurse on subsets using remaining attributes.

Result and Limitations ■ Result: [18+/13-] H01420 = yes: [2+/12-] | D12765 = yes: [2+/0-] | D12765 = no: [0+/12-] H01420 = no: [16+/1-] | D14662 = yes: [0+/1-] | D14662 = no: [16+/0-] testError: 0.4 ■ Limitation: 1) The rule (average max and min) that converts attribute values to binary values; 2) The threshold (0.1) that used to check whether to split or not ; 3) The depth of the tree and the robustness; 4) The cross validation step.