(classification & regression trees)

Slides:

Advertisements

Similar presentations

Chapter 7 Classification and Regression Trees

Advertisements

Random Forest Predrag Radenković 3237/10

Classification Algorithms

Hunt’s Algorithm CIT365: Data Mining & Data Warehousing Bajuna Salehe

Decision Tree Approach in Data Mining

CART: Classification and Regression Trees Chris Franck LISA Short Course March 26, 2013.

Chapter 7 – Classification and Regression Trees

Chapter 7 – Classification and Regression Trees

Decision Tree Rong Jin. Determine Milage Per Gallon.

Additive Models and Trees

Modeling Gene Interactions in Disease CS 686 Bioinformatics.

© Prentice Hall1 DATA MINING Introductory and Advanced Topics Part II Margaret H. Dunham Department of Computer Science and Engineering Southern Methodist.

Notes on Measuring the Performance of a Binary Classifier David Madigan.

Comp 540 Chapter 9: Additive Models, Trees, and Related Methods

Prelude of Machine Learning 202 Statistical Data Analysis in the Computer Age (1991) Bradely Efron and Robert Tibshirani.

Decision Tree Models in Data Mining

Biostatistics Case Studies 2009 Peter D. Christenson Biostatistician Session 1: Classification Trees.

Lecture Notes 4 Pruning Zhangxi Lin ISQS

Predicting Income from Census Data using Multiple Classifiers Presented By: Arghya Kusum Das Arnab Ganguly Manohar Karki Saikat Basu Subhajit Sidhanta.

Biostatistics Case Studies 2005 Peter D. Christenson Biostatistician Session 5: Classification Trees: An Alternative to Logistic.

Basics of Regression Analysis. Determination of three performance measures Estimation of the effect of each factor Explanation of the variability Forecasting.

Chapter 9 – Classification and Regression Trees

Categorical data. Decision Tree Classification Which feature to split on? Try to classify as many as possible with each split (This is a good split)

Chapter 6 Data Mining 1. Introduction The increase in the use of data-mining techniques in business has been caused largely by three events. The explosion.

Business Intelligence and Decision Modeling Week 9 Customer Profiling Decision Trees (Part 2) CHAID CRT.

Recursive Binary Partitioning Old Dogs with New Tricks KDD Conference 2009 David J. Slate and Peter W. Frey.

CS690L Data Mining: Classification

Copyright © 2010 SAS Institute Inc. All rights reserved. Decision Trees Using SAS Sylvain Tremblay SAS Canada – Education SAS Halifax Regional User Group.

DECISION TREE Ge Song. Introduction ■ Decision Tree: is a supervised learning algorithm used for classification or regression. ■ Decision Tree Graph:

Random Forests Ujjwol Subedi. Introduction What is Random Tree? ◦ Is a tree constructed randomly from a set of possible trees having K random features.

Logistic Regression Saed Sayad 1www.ismartsoft.com.

Chapter 5: Credibility. Introduction Performance on the training set is not a good indicator of performance on an independent set. We need to predict.

Biostatistics Case Studies 2008 Peter D. Christenson Biostatistician Session 6: Classification Trees.

Classification and Regression Trees

RECITATION 4 MAY 23 DPMM Splines with multiple predictors Classification and regression trees.

Eco 6380 Predictive Analytics For Economists Spring 2016 Professor Tom Fomby Department of Economics SMU.

Tree and Forest Classification and Regression Tree Bagging of trees Boosting trees Random Forest.

Bump Hunting The objective PRIM algorithm Beam search References: Feelders, A.J. (2002). Rule induction by bump hunting. In J. Meij (Ed.), Dealing with.

CPH Dr. Charnigo Chap. 9 Notes To begin with, have a look at Figure 9.5 on page 315. One can get an intuitive feel for how a tree works by examining.

Bootstrap and Model Validation

BINARY LOGISTIC REGRESSION

Machine Learning Inductive Learning and Decision Trees

Chapter 7. Classification and Prediction

k-Nearest neighbors and decision tree

Evaluating Classifiers

Logistic Regression APKC – STATS AFAC (2016).

Heping Zhang, Chang-Yung Yu, Burton Singer, Momian Xiong

Data Mining CAS 2004 Ratemaking Seminar Philadelphia, Pa.

Notes on Logistic Regression

Decision Trees.

C4.5 algorithm Let the classes be denoted {C1, C2,…, Ck}. There are three possibilities for the content of the set of training samples T in the given node.

C4.5 algorithm Let the classes be denoted {C1, C2,…, Ck}. There are three possibilities for the content of the set of training samples T in the given node.

Lecture 17. Boosting¶ CS 109A/AC 209A/STAT 121A Data Science: Harvard University Fall 2016 Instructors: P. Protopapas, K. Rader, W. Pan.

Ch9: Decision Trees 9.1 Introduction A decision tree:

Chapter 6 Classification and Prediction

Advanced Analytics Using Enterprise Miner

Employee Turnover: Data Analysis and Exploration

ECE 471/571 – Lecture 12 Decision Tree.

MIS2502: Data Analytics Classification using Decision Trees

Lecture 05: Decision Trees

CSCI N317 Computation for Scientific Applications Unit Weka

Statistical Learning Dong Liu Dept. EEIS, USTC.

Model generalization Brief summary of methods

MIS2502: Data Analytics Classification Using Decision Trees

Decision trees MARIO REGIN.

CS639: Data Management for Data Science

Lecture 46 Section 14.5 Wed, Apr 13, 2005

Advisor: Dr.vahidipour Zahra salimian Shaghayegh jalali Dec 2017

STT : Intro. to Statistical Learning

Data Mining CSCI 307, Spring 2019 Lecture 8

Presentation transcript:

(classification & regression trees) Classification trees CART (classification & regression trees)

Tree algorithm- binary recursive partitioning The classification tree or regression tree method examines all the predictors (Xs) and finds the best value of the best predictor that splits the data into two groups (nodes) that are as different as possible on the outcome. This process is independently repeated in the two “daughter” nodes created by the split until either the final (“terminal”) nodes are homogeneous (all observations are the same value, sample size is too small (default: n <5) or the difference between the two nodes is not statistically significant.

Ex: Kidwell -Hemorrhage stroke data, n=89, Y=54 with no hemorrhage, 35 with hemorrhage (binary outcome Y) Potential predictors: Glucose level Platelet count Hematocrit Time to recanalization Coumadin used (y/n) NIH stroke score (0-42) Age Sex Weight SBP DBP Diabetes

Classification Matrix Hemorrhage model Classification Matrix Actual Category no yes total Predicted 45 11 56 Category 9 24 33 54 35 89 Sensitivity= 24/35 =69% Specificity= 45/54 =83% Accuracy=76% C = R2 (Hosmer) = 0.606 df=89-5= 84 Deviance/df=72.3/84= 0.861

JMP tree (“partition”) details JMP reports chi-square (G2) values for each node. The chi square for testing the significance of a split is computed as G2 test = G2 parent - (G2 left + G2 right). JMP reports the “Log Worth” rather than the p value (common in genetics) for G2 test Log Worth = - log10(p value) G2 test p value Log Worth 2.71 0.10 1.00 3.84 0.05 1.30 6.63 0.01 2.00 Titanic example

JMP partition details (cont) Can automate making the tree with “k fold cross validation” option (k=5 by default). Choose this option, then select “go”. May need to “prune” tree (remove non significant terminal nodes).

Tree from Titanic data- Y=survival

Titanic tree fit stats ROC Area 0.8095

Titanic tree – fit stats Measure Training Definition Entropy RSquare 0.3042 1-Loglike(model)/Loglike(0) Generalized RSquare 0.4523 (1-(L(0)/L(model))^(2/n))/(1-L(0)^(2/n)) Mean -Log p 0.4628 ∑ -Log(ρ[j])/n RMSE 0.3842 √ ∑(y[j]-ρ[j])²/n Mean Abs Dev 0.2956 ∑ |y[j]-ρ[j]|/n Misclassification Rate 0.2116 ∑ (ρ[j]≠ρMax)/n n 1309 Survived Predicted Actual No Yes Total Acc 774 35 809 95.7% 242 258 500 51.6%