Review Session II.

Slides:

Advertisements

Similar presentations

EE462 MLCV Lecture 5-6 Object Detection – Boosting Tae-Kyun Kim.

Advertisements

Ensemble Methods An ensemble method constructs a set of base classifiers from the training data Ensemble or Classifier Combination Predict class label.

CMPUT 466/551 Principal Source: CMU

Ensemble Learning: An Introduction

Adaboost and its application

Three kinds of learning

Examples of Ensemble Methods

Machine Learning: Ensemble Methods

Boosting Main idea: train classifiers (e.g. decision trees) in a sequence. a new classifier should focus on those cases which were incorrectly classified.

Copyright © 2004 by Jinyan Li and Limsoon Wong Rule-Based Data Mining Methods for Classification Problems in Biomedical Domains Jinyan Li Limsoon Wong.

Predicting Income from Census Data using Multiple Classifiers Presented By: Arghya Kusum Das Arnab Ganguly Manohar Karki Saikat Basu Subhajit Sidhanta.

Learning what questions to ask. 8/29/03Decision Trees2  Job is to build a tree that represents a series of questions that the classifier will ask of.

CS-424 Gregory Dudek Today’s outline Administrative issues –Assignment deadlines: 1 day = 24 hrs (holidays are special) –The project –Assignment 3 –Midterm.

BOOSTING David Kauchak CS451 – Fall Admin Final project.

1 Learning Chapter 18 and Parts of Chapter 20 AI systems are complex and may have many parameters. It is impractical and often impossible to encode all.

Combining multiple learners Usman Roshan. Bagging Randomly sample training data Determine classifier C i on sampled data Goto step 1 and repeat m times.

1 Ensembles An ensemble is a set of classifiers whose combined results give the final decision. test feature vector classifier 1classifier 2classifier.

Today Ensemble Methods. Recap of the course. Classifier Fusion

Pattern Recognition 1 Pattern recognition is: 1. The name of the journal of the Pattern Recognition Society. 2. A research area in which patterns in data.

ISQS 6347, Data & Text Mining1 Ensemble Methods. ISQS 6347, Data & Text Mining 2 Ensemble Methods Construct a set of classifiers from the training data.

Levels of Image Data Representation 4.2. Traditional Image Data Structures 4.3. Hierarchical Data Structures Chapter 4 – Data structures for.

Classification Ensemble Methods 1

1 January 24, 2016Data Mining: Concepts and Techniques 1 Data Mining: Concepts and Techniques — Chapter 7 — Classification Ensemble Learning.

Combining multiple learners Usman Roshan. Decision tree From Alpaydin, 2010.

Boosting ---one of combining models Xin Li Machine Learning Course.

AdaBoost Algorithm and its Application on Object Detection Fayin Li.

1 Kernel Machines A relatively new learning methodology (1992) derived from statistical learning theory. Became famous when it gave accuracy comparable.

1 Ensembles An ensemble is a set of classifiers whose combined results give the final decision. test feature vector classifier 1classifier 2classifier.

Machine Learning Reading: Chapter Classification Learning Input: a set of attributes and values Output: discrete valued function Learning a continuous.

Copyright © 2004 by Jinyan Li and Limsoon Wong Rule-Based Data Mining Methods for Classification Problems in Biomedical Domains Jinyan Li Limsoon Wong.

Ensemble Classifiers.

Combining Models Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya.

Machine Learning: Ensemble Methods

Neural networks and support vector machines

Bagging and Random Forests

Deep Feedforward Networks

MIRA, SVM, k-NN Lirong Xia. MIRA, SVM, k-NN Lirong Xia.

Session 7: Face Detection (cont.)

Trees, bagging, boosting, and stacking

Learning I Linda Shapiro CSE 455.

More on Learning Neural Nets Support Vectors Machines

CS 4/527: Artificial Intelligence

Generalization ..

Machine Learning Week 1.

Linear Discriminators

Data Mining Practical Machine Learning Tools and Techniques

Learning Chapter 18 and Parts of Chapter 20

Categorization by Learning and Combing Object Parts

Introduction to Data Mining, 2nd Edition

Multiple Decision Trees ISQS7342

Learning Chapter 18 and Parts of Chapter 20

Ensemble learning.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Model Combination.

Learning Chapter 18 and Parts of Chapter 20

Learning I Linda Shapiro EE/CSE 576.

Learning I Linda Shapiro CSE 455.

Recitation 10 Oznur Tastan

Chapter 7: Transformations

Artificial Intelligence 9. Perceptron

Mihir Patel and Nikhil Sardana

Learning Chapter 18 and Parts of Chapter 20

Ensembles An ensemble is a set of classifiers whose combined results give the final decision. test feature vector classifier 1 classifier 2 classifier.

Kernel Machines A relatively new learning methodology (1992) derived from statistical learning theory. Became famous when it gave accuracy comparable to.

Learning I Linda Shapiro EE/CSE 576.

Learning I Linda Shapiro ECE/CSE 576.

Ensembles An ensemble is a set of classifiers whose combined results give the final decision. test feature vector classifier 1 classifier 2 classifier.

Review Session II.

MIRA, SVM, k-NN Lirong Xia. MIRA, SVM, k-NN Lirong Xia.

Learning I Linda Shapiro ECE/CSE 576.

Presentation transcript:

Review Session II

Logic and Reasoning John likes any food Peanuts can be eaten. Anything eaten is food. Prove: John likes peanuts food(x) => likes(John,x) eatable(peanuts) eatable(x) => food(x) {¬food(x) , likes(John,x) {eatable(peanuts)} {¬eatable(x) , food(x)} Conjecture: likes(John,peanuts) Negation: {¬likes(John,peanuts)} {¬likes(John,peanuts)} {¬food(x) , likes(John,x) {eatable(peanuts)} {¬eatable(x) , food(x)} {¬food(peanuts)} {food(peanuts)} NIL

Decision Trees with Information Gain Gray Large LongNeck Class Y Y N E N Y Y G N Y N E Y Y N E Entropy of root: -3/4 log2(3/4) – ¼ log2(1/4) -.75(-.415) -.25(-2) = .811 Split on Gray: Y: {E,E} Entropy 0 N: {G,E} Entropy 1 Gain: .811 -.5(0)-.5(1) = .311 Split on Large: Always Y {E,G,E,E}, same as root. Gain of Zero. Split on LongNeck: Y: {G} Entropy 0 N: {E,E,E} Entropy 0 Gain: .811 – 0 = .811***

Idea of Boosting

ADABoost ADABoost boosts the accuracy of the original learning algorithm. If the original learning algorithm does slightly better than 50% accuracy, ADABoost with a large enough number of classifiers is guaranteed to classify the training data perfectly.

ADABoost Weight Updating (from Fig 18.34 text) /* First find the sum of the weights of the misclassified samples */ for j = 1 to N do /* go through training samples */ if h[m](xj) <> yj then error <- error + wj /* Now use the ratio of error to 1-error to change the weights of the correctly classified samples */ for j = 1 to N do if h[m](xj) = yj then w[j] <- w[j] * error/(1-error)

Example Start with 4 samples of equal weight .25. Suppose 1 is misclassified. So error = .25. The ratio comes out .25/.75 = .33 The correctly classified samples get weight of .25*.33 = .0825 .2500 .0825 .5 .165 What’s wrong? What should we do? We want them to add up to 1, not .4975. Answer: To normalize, divide each one by their sum (.4975).

Neural Nets -2 * .5 + 2 * .4 = -.2 g(-.2) = -1 .5 -2 .4 .5 -1 * .4 + 1 * .2 = -.2 g(-.2) = -1 .4 .2 2 .6 -2 * .5 + 2 * .6 = .2 g(.2) = 1

SVMs

K-means: mean mu EM: mean mu, covariance ∑, weight W

Yi Li’s EM Learning Method 1: one Gaussian model per object class Method 2: for each class, first use the positive instances to obtain Gaussian clusters in each feature space (color, texture, structure, etc) Then use the CLUSTERS to obtain fixed length feature vectors for positive and negative instances of that class and train a neural net Tree model Stadium Model Building Model …..

CNNs Convolution Pooling ReLU