Naive Bayes Classifiers (NBC)

Slides:

Advertisements

Similar presentations

Feature Selection for Pattern Recognition J.-S. Roger Jang ( 張智星 ) CSIE Dept., National Taiwan University ( 台灣大學資訊工程系 )

Advertisements

Probabilistic Generative Models Rong Jin. Probabilistic Generative Model Classify instance x into one of K classes Class prior Density function for class.

Shallow Copy Jyh-Shing Roger Jang ( 張智星 ) CSIE Dept, National Taiwan University.

Onset Detection in Audio Music J.-S Roger Jang ( 張智星 ) MIR LabMIR Lab, CSIE Dept. National Taiwan University.

Assuming normally distributed data! Naïve Bayes Classifier.

Classification Dr Eamonn Keogh Computer Science & Engineering Department University of California - Riverside Riverside,CA Who.

Logistic Regression Rong Jin. Logistic Regression Model  In Gaussian generative model:  Generalize the ratio to a linear model Parameters: w and c.

Logistic Regression Rong Jin. Logistic Regression Model  In Gaussian generative model:  Generalize the ratio to a linear model Parameters: w and c.

Performance Evaluation: Estimation of Recognition rates J.-S. Roger Jang ( 張智星 ) CSIE Dept., National Taiwan Univ.

Simple Bayesian Supervised Models Saskia Klein & Steffen Bollmann 1.

Thanks to Nir Friedman, HU

Results Comparison with existing approaches on benchmark datasets Evaluation on a uveal melanoma datasetEvaluation on the two-spiral dataset Evaluation.

Jeff Howbert Introduction to Machine Learning Winter Classification Bayesian Classifiers.

CSCI 347 / CS 4206: Data Mining Module 06: Evaluation Topic 01: Training, Testing, and Tuning Datasets.

PCA & LDA for Face Recognition

Principal Component Analysis (PCA)

2015/9/131 Stress Detection J.-S. Roger Jang ( 張智星 ) MIR LabMIR Lab, CSIE Dept., National Taiwan Univ.

Decision Trees Jyh-Shing Roger Jang ( 張智星 ) CSIE Dept, National Taiwan University.

Computational Intelligence: Methods and Applications Lecture 12 Bayesian decisions: foundation of learning Włodzisław Duch Dept. of Informatics, UMK Google:

Singly Linked Lists Jyh-Shing Roger Jang ( 張智星 ) CSIE Dept, National Taiwan University 1.

Sparse Vectors & Matrices Jyh-Shing Roger Jang ( 張智星 ) CSIE Dept, National Taiwan University.

Binary Search Jyh-Shing Roger Jang ( 張智星 ) CSIE Dept, National Taiwan University.

Quadratic Classifiers (QC) J.-S. Roger Jang ( 張智星 ) CS Dept., National Taiwan Univ Scientific Computing.

Sep 10th, 2001Copyright © 2001, Andrew W. Moore Learning Gaussian Bayes Classifiers Andrew W. Moore Associate Professor School of Computer Science Carnegie.

Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, a Machine Learning.

STL: Maps Jyh-Shing Roger Jang ( 張智星 ) CSIE Dept, National Taiwan University.

A Brief Introduction and Issues on the Classification Problem Jin Mao Postdoc, School of Information, University of Arizona Sept 18, 2015.

Chapter 6. Classification and Prediction Classification by decision tree induction Bayesian classification Rule-based classification Classification by.

Linear Models (II) Rong Jin. Recap  Classification problems Inputs x  output y y is from a discrete set Example: height 1.8m  male/female?  Statistical.

Maximum Likelihood Estimate Jyh-Shing Roger Jang ( 張智星 ) CSIE Dept, National Taiwan University.

Discussions on Audio Melody Extraction (AME) J.-S. Roger Jang ( 張智星 ) MIR Lab, CSIE Dept. National Taiwan University.

Simulation of Stock Trading J.-S. Roger Jang ( 張智星 ) MIR Lab, CSIE Dept. National Taiwan University.

Linear Classifiers (LC) J.-S. Roger Jang ( 張智星 ) MIR Lab, CSIE Dept. National Taiwan University.

Bayesian Classification 1. 2 Bayesian Classification: Why? A statistical classifier: performs probabilistic prediction, i.e., predicts class membership.

From C to C++ Jyh-Shing Roger Jang (張智星)

Onset Detection, Tempo Estimation, and Beat Tracking

CSIE Dept., National Taiwan Univ., Taiwan

Matt Gormley Lecture 3 September 7, 2016

Maximum Likelihood Estimate

Quadratic Classifiers (QC)

DP for Optimum Strategies in Games

Query by Singing/Humming via Dynamic Programming

Discrete Fourier Transform (DFT)

Singing Voice Separation via Active Noise Cancellation 使用主動式雜訊消除於歌聲分離

Introduction to Machine Learning

Intro to Machine Learning

National Taiwan University

Reading: Pedro Domingos: A Few Useful Things to Know about Machine Learning source: /cacm12.pdf reading.

Feature Selection for Pattern Recognition

CHAPTER 10 Correlation and Regression (Objectives)

Intro to Machine Learning

Data Mining: Concepts and Techniques (3rd ed.) — Chapter 8 —

Deep Neural Networks (DNN)

How where first 3 displays generated?

Circularly Linked Lists and List Reversal

National Taiwan University

Endpoint Detection ( 端點偵測)

Applications of Heaps J.-S. Roger Jang (張智星) MIR Lab, CSIE Dept.

Query by Singing/Humming via Dynamic Programming

Insertion Sort Jyh-Shing Roger Jang (張智星)

Examples of Time Complexity

Scientific Computing: Closing 科學計算：結語

Prediction in Stock Trading

Selection Algorithm Jyh-Shing Roger Jang (張智星)

Game Trees and Minimax Algorithm

Logistic Regression [Many of the slides were originally created by Prof. Dan Jurafsky from Stanford.]

MAS 622J Course Project Classification of Affective States - GP Semi-Supervised Learning, SVM and kNN Hyungil Ahn

Duration & Pitch Modification via WSOLA

Internet Traffic Classification Using Bayesian Analysis Techniques

Pre and Post-Processing for Pitch Tracking

Presentation transcript:

Naive Bayes Classifiers (NBC) J.-S. Roger Jang (張智星) jang@mirlab.org http://mirlab.org/jang MIR Lab, CSIE Dept. National Taiwan University

Assumptions & Characteristics Statistical independency between features Statistical independency between samples Each feature governed by a feature-wise parameterized PDF (usually a 1D Gaussian) Characteristics Simple and easy (That’s why it’s named “naive”.) Highly successful in real-world applications regardless of the strong assumptions

Training and Test Stages of NBC Quiz! Training stage Identify class PDF, as follows. Identify feature PDF by MLE for 1D Gaussians Class PDF is the product of all the corresponding feature PDFs Test stage Assign a sample to the class by taking class prior into consideration:

NBC for Gender Dataset (1/2) Scatter plot of Gender dataset PDF on each features and each class ds=prData('gender'); figure; dsScatterPlot(ds); [nbcPrm, logLike, recogRate, hitIndex]=nbcTrain(ds); figure; nbcPlot(ds, nbcPrm, '1dPdf'); ds=prData('gender'); figure; dsScatterPlot(ds); [nbcPrm, logLike, recogRate, hitIndex]=nbcTrain(ds); figure; nbcPlot(ds, nbcPrm, '1dPdf');

NBC for Gender Dataset (2/2) PDF for each class Decision boundary ds=prData('gender'); [nbcPrm, logLike, recogRate, hitIndex]=nbcTrain(ds); figure; nbcPlot(ds, nbcPrm, '2dPdf'); figure; nbcPlot(ds, nbcPrm, 'decBoundary'); ds=prData('gender'); [nbcPrm, logLike, recogRate, hitIndex]=nbcTrain(ds); figure; nbcPlot(ds, nbcPrm, '2dPdf'); figure; nbcPlot(ds, nbcPrm, 'decBoundary');

NBC for Iris Dataset (1/2) Scatter plot of Iris dataset (with only the last two dim.) PDF on each features and each class ds=prData('iris'); ds.input=ds.input(3:4, :); figure; dsScatterPlot(ds); [nbcPrm, logLike, recogRate, hitIndex]=nbcTrain(ds); figure; nbcPlot(ds, nbcPrm, '1dPdf'); ds=prData('iris'); ds.input=ds.input(3:4, :); figure; dsScatterPlot(ds); [nbcPrm, logLike, recogRate, hitIndex]=nbcTrain(ds); figure; nbcPlot(ds, nbcPrm, '1dPdf');

NBC for Iris Dataset (2/2) PDF for each class Dec. boundaries ds=prData('iris'); ds.input=ds.input(3:4, :); [nbcPrm, logLike, recogRate, hitIndex]=nbcTrain(ds); figure; nbcPlot(ds, nbcPrm, '2dPdf'); ds=prData('iris'); ds.input=ds.input(3:4, :); [nbcPrm, logLike, recogRate, hitIndex]=nbcTrain(ds); figure; nbcPlot(ds, nbcPrm, '2dPdf'); ds.hitIndex=hitIndex; % For plotting figure; nbcPlot(ds, nbcPrm, 'decBoundary');

Strength and Weakness of NBC Quiz! Strength Fast computation during training and evaluation Robust than QC Fast method for computing leave-one-out cross validation Weakness Not able to deal with bi-modal data Class boundary not as complex as QC