Radosław Wesołowski Tomasz Pękalski, Michal Borkowicz, Maciej Kopaczyński 12-03-2008.

Slides:

Advertisements

Similar presentations

Decision Tree Learning - ID3

Advertisements

Decision Trees Decision tree representation ID3 learning algorithm

1er. Escuela Red ProTIC - Tandil, de Abril, Decision Tree Learning 3.1 Introduction –Method for approximation of discrete-valued target functions.

C4.5 algorithm Let the classes be denoted {C1, C2,…, Ck}. There are three possibilities for the content of the set of training samples T in the given node.

ICS320-Foundations of Adaptive and Learning Systems

Classification Techniques: Decision Tree Learning

Decision Tree Learning 主講人：虞台文大同大學資工所智慧型多媒體研究室.

Chapter 7 – Classification and Regression Trees

Chapter 7 – Classification and Regression Trees

Decision Trees IDHairHeightWeightLotionResult SarahBlondeAverageLightNoSunburn DanaBlondeTallAverageYesnone AlexBrownTallAverageYesNone AnnieBlondeShortAverageNoSunburn.

Decision Trees Instructor: Qiang Yang Hong Kong University of Science and Technology Thanks: Eibe Frank and Jiawei Han.

ID3 Algorithm Abbas Rizvi CS157 B Spring What is the ID3 algorithm? ID3 stands for Iterative Dichotomiser 3 Algorithm used to generate a decision.

Part 7.3 Decision Trees Decision tree representation ID3 learning algorithm Entropy, information gain Overfitting.

Decision Tree Algorithm

Induction of Decision Trees

1 Classification with Decision Trees I Instructor: Qiang Yang Hong Kong University of Science and Technology Thanks: Eibe Frank and Jiawei.

Decision Trees an Introduction.

Decision Trees Chapter 18 From Data to Knowledge.

Example of a Decision Tree categorical continuous class Splitting Attributes Refund Yes No NO MarSt Single, Divorced Married TaxInc NO < 80K > 80K.

ICS 273A Intro Machine Learning

Machine Learning Lecture 10 Decision Trees G53MLE Machine Learning Dr Guoping Qiu1.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Fall 2004 TDIDT Learning CS478 - Machine Learning.

Lecture Notes 4 Pruning Zhangxi Lin ISQS

Machine Learning Chapter 3. Decision Tree Learning

Chapter 4 Classification. 2 Classification: Definition Given a collection of records (training set ) –Each record contains a set of attributes, one of.

Lecture 7. Outline 1. Overview of Classification and Decision Tree 2. Algorithm to build Decision Tree 3. Formula to measure information 4. Weka, data.

Computational Intelligence: Methods and Applications Lecture 19 Pruning of decision trees Włodzisław Duch Dept. of Informatics, UMK Google: W Duch.

Machine Learning Lecture 10 Decision Tree Learning 1.

CpSc 810: Machine Learning Decision Tree Learning.

Comparing Univariate and Multivariate Decision Trees Olcay Taner Yıldız Ethem Alpaydın Department of Computer Engineering Bogazici University

Learning from Observations Chapter 18 Through

1 COMP3503 Inductive Decision Trees with Daniel L. Silver Daniel L. Silver.

Decision Tree Learning Debapriyo Majumdar Data Mining – Fall 2014 Indian Statistical Institute Kolkata August 25, 2014.

Business Intelligence and Decision Modeling Week 9 Customer Profiling Decision Trees (Part 2) CHAID CRT.

For Wednesday No reading Homework: –Chapter 18, exercise 6.

For Monday No new reading Homework: –Chapter 18, exercises 3 and 4.

CS 8751 ML & KDDDecision Trees1 Decision tree representation ID3 learning algorithm Entropy, Information gain Overfitting.

Copyright © 2010 SAS Institute Inc. All rights reserved. Decision Trees Using SAS Sylvain Tremblay SAS Canada – Education SAS Halifax Regional User Group.

CS 5751 Machine Learning Chapter 3 Decision Tree Learning1 Decision Trees Decision tree representation ID3 learning algorithm Entropy, Information gain.

Decision Trees Binary output – easily extendible to multiple output classes. Takes a set of attributes for a given situation or object and outputs a yes/no.

1 Decision Tree Learning Original slides by Raymond J. Mooney University of Texas at Austin.

Decision Trees, Part 1 Reading: Textbook, Chapter 6.

Decision Trees Example of a Decision Tree categorical continuous class Refund MarSt TaxInc YES NO YesNo Married Single, Divorced < 80K> 80K Splitting.

DECISION TREE Ge Song. Introduction ■ Decision Tree: is a supervised learning algorithm used for classification or regression. ■ Decision Tree Graph:

Decision Tree Learning Presented by Ping Zhang Nov. 26th, 2007.

ECE 471/571 – Lecture 20 Decision Tree 11/19/15. 2 Nominal Data Descriptions that are discrete and without any natural notion of similarity or even ordering.

Data Mining By Farzana Forhad CS 157B. Agenda Decision Tree and ID3 Rough Set Theory Clustering.

Decision Trees.

Oliver Schulte Machine Learning 726 Decision Tree Classifiers.

Chapter 18 Section 1 – 3 Learning from Observations.

1 Universidad de Buenos Aires Maestría en Data Mining y Knowledge Discovery Aprendizaje Automático 4-Inducción de árboles de decisión (1/2) Eduardo Poggi.

Learning From Observations Inductive Learning Decision Trees Ensembles.

Eco 6380 Predictive Analytics For Economists Spring 2016 Professor Tom Fomby Department of Economics SMU.

Decision Tree Learning DA514 - Lecture Slides 2 Modified and expanded from: E. Alpaydin-ML (chapter 9) T. Mitchell-ML.

Review of Decision Tree Learning Bamshad Mobasher DePaul University Bamshad Mobasher DePaul University.

DECISION TREES An internal node represents a test on an attribute.

Decision Trees an introduction.

C4.5 algorithm Let the classes be denoted {C1, C2,…, Ck}. There are three possibilities for the content of the set of training samples T in the given node.

C4.5 algorithm Let the classes be denoted {C1, C2,…, Ck}. There are three possibilities for the content of the set of training samples T in the given node.

Artificial Intelligence

Decision Trees (suggested time: 30 min)

Ch9: Decision Trees 9.1 Introduction A decision tree:

Data Science Algorithms: The Basic Methods

Machine Learning Chapter 3. Decision Tree Learning

Machine Learning: Lecture 3

Decision Trees Decision tree representation ID3 learning algorithm

Machine Learning Chapter 3. Decision Tree Learning

Decision trees.

Decision trees One possible representation for hypotheses

Presentation transcript:

Radosław Wesołowski Tomasz Pękalski, Michal Borkowicz, Maciej Kopaczyński

What is it anyway? Decision tree T – a tree with a root (in graph theory sense), in which we assign the following meanings to its elements: - inner nodes represent attributes, - edges represent values of the attribute, - leafs represent classification decisions. Using decision tree we can visualize a program with only ‘if-then’ instructions.

Testing functions Let us consider an attribute A (e.g. temperature). Let V A mean the set of all possible values of A (0K up to infinity). Let R t mean the set of all possible test results (hot, mild, cold). As a testing function we mean a map t: V A  R t We distinguish two main types of testing functions, depending on the set V A - discrete and continuous.

Quality of a decision tree (Occam's razor): - we prefer small, simple trees, - we want to gain maximum accuracy of classification (training set, test set) For example: For example: Q(T) =  *size(T) +  *accuracy(T)

Optimal tree – we are given: - a training set S, - a testing functions set TEST, - quality criterion Q. Target: T optimising Q(T). Fact: usually this is NP-hard problem. Conclusion: we have to use heuristics.

Building a decision tree: - top_down method: a. In the beginning the root includes all training examples b. We divide them recursively, choosing one attribute at a time - bottom_up: we remove subtrees or edges to gain precision for judging new cases.

Entropy – average bits amount to represent a decision d for a randomly chosen object from a given set S. Why? Because optimal binary representation assigns –log2(p) bits to a decision which probability is p. We have formula: entropy(p1,...pn)= - p1*log2(p1) pn*log2(pn)

Information gain: gain(.) = info before dividing – info after dividing

Overtraining: We say that a model H overfits if there is a model H’ such that : - training_error(H) < training_error(H’), - testing_error(H) > testing_error(H’). Avoiding overtraining: - adequate stop criterions, - posprunning, - preprunning.

Some decision trees algorithms: - R1, - ID3 (), - ID3 ( Interactive dichotomizer version 3 ), - C4.5 ( - C4.5 ( ID3 + discretization + prunning ), - CART ( - CART ( Classification and Regression Trees ), - CHAID ( - CHAID ( CHi-squared Automatic Interaction ). -Detection ).