Chapter 13 – Ensembles and Uplift

Slides:

Advertisements

Similar presentations

Ensemble Learning – Bagging, Boosting, and Stacking, and other topics

Advertisements

Random Forest Predrag Radenković 3237/10

Ensemble Learning Reading: R. Schapire, A brief introduction to boosting.

Ensemble Methods An ensemble method constructs a set of base classifiers from the training data Ensemble or Classifier Combination Predict class label.

Chapter 7 – Classification and Regression Trees

Chapter 7 – Classification and Regression Trees

Longin Jan Latecki Temple University

Ensemble Learning what is an ensemble? why use an ensemble?

A Brief Introduction to Adaboost

Ensemble Learning: An Introduction

Bagging and Boosting in Data Mining Carolina Ruiz

Examples of Ensemble Methods

Ensemble Learning (2), Tree and Forest

Machine Learning CS 165B Spring 2012

Zhangxi Lin ISQS Texas Tech University Note: Most slides are from Decision Tree Modeling by SAS Lecture Notes 6 Ensembles of Trees.

ENSEMBLE LEARNING David Kauchak CS451 – Fall 2013.

Chapter 9 – Classification and Regression Trees

LOGO Ensemble Learning Lecturer: Dr. Bo Yuan

Data Mining - Volinsky Columbia University 1 Topic 10 - Ensemble Methods.

Combining multiple learners Usman Roshan. Bagging Randomly sample training data Determine classifier C i on sampled data Goto step 1 and repeat m times.

Today Ensemble Methods. Recap of the course. Classifier Fusion

Ensemble Methods: Bagging and Boosting

Ensembles. Ensemble Methods l Construct a set of classifiers from training data l Predict class label of previously unseen records by aggregating predictions.

Ensemble Learning Spring 2009 Ben-Gurion University of the Negev.

CLASSIFICATION: Ensemble Methods

BAGGING ALGORITHM, ONLINE BOOSTING AND VISION Se – Hoon Park.

ISQS 6347, Data & Text Mining1 Ensemble Methods. ISQS 6347, Data & Text Mining 2 Ensemble Methods Construct a set of classifiers from the training data.

Ensemble with Neighbor Rules Voting Itt Romneeyangkurn, Sukree Sinthupinyo Faculty of Computer Science Thammasat University.

E NSEMBLE L EARNING : A DA B OOST Jianping Fan Dept of Computer Science UNC-Charlotte.

Ensemble Learning  Which of the two options increases your chances of having a good grade on the exam? –Solving the test individually –Solving the test.

Konstantina Christakopoulou Liang Zeng Group G21

Random Forests Ujjwol Subedi. Introduction What is Random Tree? ◦ Is a tree constructed randomly from a set of possible trees having K random features.

1 January 24, 2016Data Mining: Concepts and Techniques 1 Data Mining: Concepts and Techniques — Chapter 7 — Classification Ensemble Learning.

Classification and Prediction: Ensemble Methods Bamshad Mobasher DePaul University Bamshad Mobasher DePaul University.

Ensemble Methods Construct a set of classifiers from the training data Predict class label of previously unseen records by aggregating predictions made.

Decision Trees IDHairHeightWeightLotionResult SarahBlondeAverageLightNoSunburn DanaBlondeTallAverageYesnone AlexBrownTallAverageYesNone AnnieBlondeShortAverageNoSunburn.

© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Genetic Algorithms (in 1 Slide) l GA: based on an analogy to biological evolution l Each.

1 Introduction to Predictive Learning Electrical and Computer Engineering LECTURE SET 8 Combining Methods and Ensemble Learning.

Chapter 5 – Evaluating Predictive Performance Data Mining for Business Analytics Shmueli, Patel & Bruce.

1 Machine Learning Lecture 8: Ensemble Methods Moshe Koppel Slides adapted from Raymond J. Mooney and others.

Chapter 11 – Neural Nets © Galit Shmueli and Peter Bruce 2010 Data Mining for Business Intelligence Shmueli, Patel & Bruce.

Ensemble Classifiers.

Machine Learning: Ensemble Methods

Data Mining Practical Machine Learning Tools and Techniques

Bagging and Random Forests

Zaman Faisal Kyushu Institute of Technology Fukuoka, JAPAN

Eco 6380 Predictive Analytics For Economists Spring 2016

COMP61011 : Machine Learning Ensemble Models

Ensemble Learning Introduction to Machine Learning and Data Mining, Carla Brodley.

Chapter 7 – K-Nearest-Neighbor

Ungraded quiz Unit 6.

A “Holy Grail” of Machine Learing

Combining Base Learners

Data Mining Practical Machine Learning Tools and Techniques

CSC321: 2011 Introduction to Neural Networks and Machine Learning Lecture 12: Combining models Geoffrey Hinton.

Direct or Remotely sensed

Genetic Algorithms (GA)

Introduction to Data Mining, 2nd Edition

Chap. 7 Regularization for Deep Learning (7.8~7.12 )

Multiple Decision Trees ISQS7342

Siyan Gan, Pepperdine University

Decision Trees By Cole Daily CSCI 446.

Ensemble learning.

Cross-validation Brenda Thomson/ Peter Fox Data Analytics

Model Combination.

Ensemble learning Reminder - Bagging of Trees Random Forest

CS639: Data Management for Data Science

CS 391L: Machine Learning: Ensembles

Advisor: Dr.vahidipour Zahra salimian Shaghayegh jalali Dec 2017

Presentation transcript:

Chapter 13 – Ensembles and Uplift Data Mining for Business Analytics Shmueli, Patel & Bruce

Ensembles can improve predictive power Model 1 Model 2 Error e1,i e2,i Expected Error E(e1,i) = 0 E(e2,i) = 0 Variance Var(e1,i) Var(e2,i) Ensemble: E((e1,i+ e2,i)/2) = 0 ¼ Var(e1,i) + ¼ Var(e2,i) + Cov(e1,i, e1,i)

Methods used Simple averaging Weighted averaging Voting for classifiers

Bagging Bootstrap sampling Aggregating Generate multiple random samples with replacement Aggregating Run modeling algorithm on each sample Combine the results

Boosting Fit a model Generate a sample -- oversample misclassified cases Fit the model to the new sample Repeat 2 and 3 multiple times Bagging improves stability Helps avoid over fitting

Advantages and Disadvantages More precise predictions Improves stability Helps avoid over fitting Disadvantages Requires more resources Time Ensemble model non-interpretable

Uplift modeling Collect sample data including current status Randomly split data into treatment and control group Apply treatment to treatment group Measure status change for both groups Recombine sample and randomly partition into training and validation sets Develop model to training set with Status change as target and include treatment applied or not as a predictor

Uplift modeling ….. Run model to validation data set with treatment set to 1 and calculate propensity, i.e. P(Success/Treatment = 1) Repeat with treatment set to 0 and calculate propensity, i.e. P(Success/Treatment = 0) Uplift = P(Success/Treatment = 1) - P(Success/Treatment = 0)

Uplift Example

Uplift Example

Uplift Example

Uplift Example