Classification with reject option in gene expression data Blaise Hanczar and Edward R Dougherty BIOINFORMATICS Vol. 24 no. 17 2008, pages 1889-1895.

Slides:

Advertisements

Similar presentations

Pattern Recognition and Machine Learning

Advertisements

Pattern Recognition and Machine Learning

Support Vector Machines

SVM—Support Vector Machines

Minimum Redundancy and Maximum Relevance Feature Selection

Pattern Classification, Chapter 2 (Part 2) 0 Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R.

Chapter 4: Linear Models for Classification

Present by: Fang-Hui Chu A Survey of Large Margin Hidden Markov Model Xinwei Li, Hui Jiang York University.

Fuzzy Support Vector Machines (FSVMs) Weijia Wang, Huanren Zhang, Vijendra Purohit, Aditi Gupta.

Co-Training and Expansion: Towards Bridging Theory and Practice Maria-Florina Balcan, Avrim Blum, Ke Yang Carnegie Mellon University, Computer Science.

1 Learning to Detect Objects in Images via a Sparse, Part-Based Representation S. Agarwal, A. Awan and D. Roth IEEE Transactions on Pattern Analysis and.

Prénom Nom Document Analysis: Linear Discrimination Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.

Predictive Automatic Relevance Determination by Expectation Propagation Yuan (Alan) Qi Thomas P. Minka Rosalind W. Picard Zoubin Ghahramani.

Reduced Support Vector Machine

Support Vector Machines Based on Burges (1998), Scholkopf (1998), Cristianini and Shawe-Taylor (2000), and Hastie et al. (2001) David Madigan.

Machine Learning CMPT 726 Simon Fraser University

Face Processing System Presented by: Harvest Jang Group meeting Fall 2002.

Statistical Learning Theory: Classification Using Support Vector Machines John DiMona Some slides based on Prof Andrew Moore at CMU:

Review Rong Jin. Comparison of Different Classification Models  The goal of all classifiers Predicating class label y for an input x Estimate p(y|x)

Efficient Model Selection for Support Vector Machines

Based on: The Nature of Statistical Learning Theory by V. Vapnick 2009 Presentation by John DiMona and some slides based on lectures given by Professor.

SVM by Sequential Minimal Optimization (SMO)

Principles of Pattern Recognition

Introduction to variable selection I Qi Yu. 2 Problems due to poor variable selection: Input dimension is too large; the curse of dimensionality problem.

Statistical Decision Theory

8/25/05 Cognitive Computations Software Tutorial Page 1 SNoW: Sparse Network of Winnows Presented by Nick Rizzolo.

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 03: GAUSSIAN CLASSIFIERS Objectives: Whitening.

Classification and Ranking Approaches to Discriminative Language Modeling for ASR Erinç Dikici, Murat Semerci, Murat Saraçlar, Ethem Alpaydın 報告者：郝柏翰 2013/01/28.

Universit at Dortmund, LS VIII

Jun-Won Suh Intelligent Electronic Systems Human and Systems Engineering Department of Electrical and Computer Engineering Speaker Verification System.

Classification Heejune Ahn SeoulTech Last updated May. 03.

1 E. Fatemizadeh Statistical Pattern Recognition.

Computational Intelligence: Methods and Applications Lecture 23 Logistic discrimination and support vectors Włodzisław Duch Dept. of Informatics, UMK Google:

D. M. J. Tax and R. P. W. Duin. Presented by Mihajlo Grbovic Support Vector Data Description.

Statistical Decision Theory Bayes’ theorem: For discrete events For probability density functions.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

An Introduction to Support Vector Machine (SVM)

Linear Models for Classification

Class 23, 2001 CBCl/AI MIT Bioinformatics Applications and Feature Selection for SVMs S. Mukherjee.

Guest lecture: Feature Selection Alan Qi Dec 2, 2004.

Feature Selection with Kernel Class Separability 指導教授：王振興電機所 N 林哲偉電機所 N 曾信輝電機所 N 吳俐瑩 Date: Lei Wang, “Feature selection.

Chapter 20 Classification and Estimation Classification – Feature selection Good feature have four characteristics: –Discrimination. Features.

Final Exam Review CS479/679 Pattern Recognition Dr. George Bebis 1.

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 04: GAUSSIAN CLASSIFIERS Objectives: Whitening.

Discriminative Training and Machine Learning Approaches Machine Learning Lab, Dept. of CSIE, NCKU Chih-Pin Liao.

METU Informatics Institute Min720 Pattern Classification with Bio-Medical Applications Part 9: Review.

Support Vector Machines Reading: Ben-Hur and Weston, “A User’s Guide to Support Vector Machines” (linked from class web page)

Giansalvo EXIN Cirrincione unit #4 Single-layer networks They directly compute linear discriminant functions using the TS without need of determining.

Introduction to Classifiers Fujinaga. Bayes (optimal) Classifier (1) A priori probabilities: and Decision rule: given and decide if and probability of.

Psychology and Neurobiology of Decision-Making under Uncertainty Angela Yu March 11, 2010.

On the Optimality of the Simple Bayesian Classifier under Zero-One Loss Pedro Domingos, Michael Pazzani Presented by Lu Ren Oct. 1, 2007.

The Chinese University of Hong Kong Learning Larger Margin Machine Locally and Globally Dept. of Computer Science and Engineering The Chinese University.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Predictive Automatic Relevance Determination by Expectation Propagation Y. Qi T.P. Minka R.W. Picard Z. Ghahramani.

Optimal Decision-Making in Humans & Animals Angela Yu March 05, 2009.

CS 9633 Machine Learning Support Vector Machines

LECTURE 09: BAYESIAN ESTIMATION (Cont.)

LECTURE 03: DECISION SURFACES

Asymmetric Gradient Boosting with Application to Spam Filtering

Pattern Recognition CS479/679 Pattern Recognition Dr. George Bebis

Classification Discriminant Analysis

LECTURE 05: THRESHOLD DECODING

Classifiers Fujinaga.

Minimax Probability Machine (MPM)

Discriminative Frequent Pattern Analysis for Effective Classification

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Pattern Recognition and Machine Learning

Classifiers Fujinaga.

LECTURE 23: INFORMATION THEORY REVIEW

LECTURE 05: THRESHOLD DECODING

Presentation transcript:

Classification with reject option in gene expression data Blaise Hanczar and Edward R Dougherty BIOINFORMATICS Vol. 24 no , pages

Outline Introduction Theory of classification with reject option Implementation for bioinformatics Result and Discussion

Introduction Our method consists of defining a rejection region, which contains the examples for which classification is ambiguous. These are rejected by the classifier. The accuracy of the classifier becomes a user- defined parameter of the classification rule. The task of the classification rule is to minimize the rejection region with the constraint that the error rate of the classifier be bounded by the chosen target error.

Theory of classification with reject option A classifier is a function which divides the feature space into two regions, R 1, R 2. Error rate: Accuracy of a classifier:

Theory of classification with reject option Bayes classifier: rejection region R reject : Rejection rate: Acceptance rate:

Theory of classification with reject option Two types of error: – –Conditional error : Basic properties:

Theory of classification with reject option The error rate decreases monotonically while the rejection rate increases. Different thresholds for each class : – Our approach : Given the conditional error rate of the classifier for each class, design a classifier with the smallest rejection rate.

Implementation for bioinformatics Classifier model: –A classifier can be defined via a discriminant function : d(x)≤0 implies f(x)=C 1 d(x)>0 implies f(x)=C 2 the distance represent the confidence of the classification.

Implementation for bioinformatics

Threshold selection –Minimize the rejection rate under the constraints –The problem can be formalized as an optimization problem with three constraint :

Implementation for bioinformatics t 1 =t 2, classifiers with no rejection option.

Implementation for bioinformatics –We propose an iterative procedure: For a give value, let the function give the value of t 1. For a give value, let the function give the value of t 2. Alternately minimize t 1 with respect to the constraint and maximize t 2 with respect the constraint. Until t 1 cannot be decreased and t 2 cannot be increased.

Implementation for bioinformatics Feature selection –For feature selection we adapt sequential forward search (SFS) to classification with reject option. –We select the feature providing the lowest rejection rate under the conditional error constraints and.

Result and Discussion Error rate of the classifier with no rejection option whose the target error rate is 0.1. Error rate of the classifier with no rejection option. Synthetic data

Result and Discussion Error rate of classifier using posterior probabilities.

Result and Discussion Number of selected features is fixed to 10.

Result and Discussion

The target error of each class is fixed to 0.05 Fisher discriminant Real data

Result and Discussion SVM with linear kernel