Bayesian Decision Theory

Slides:

Advertisements

Similar presentations

Bayes rule, priors and maximum a posteriori

Advertisements

Pattern Recognition and Machine Learning

ECE 8443 – Pattern Recognition LECTURE 05: MAXIMUM LIKELIHOOD ESTIMATION Objectives: Discrete Features Maximum Likelihood Resources: D.H.S: Chapter 3 (Part.

LECTURE 11: BAYESIAN PARAMETER ESTIMATION

Lecture 20 Object recognition I

Classification and risk prediction

Decision Theory Naïve Bayes ROC Curves

METU Informatics Institute Min 720 Pattern Classification with Bio-Medical Applications PART 2: Statistical Pattern Classification: Optimal Classification.

Principles of Pattern Recognition

Speech Recognition Pattern Classification. 22 September 2015Veton Këpuska2 Pattern Classification  Introduction  Parametric classifiers  Semi-parametric.

Computational Intelligence: Methods and Applications Lecture 12 Bayesian decisions: foundation of learning Włodzisław Duch Dept. of Informatics, UMK Google:

Learning Theory Reza Shadmehr Linear and quadratic decision boundaries Kernel estimates of density Missing data.

1 E. Fatemizadeh Statistical Pattern Recognition.

Optimal Bayes Classification

1 Bayesian Decision Theory Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking and.

ECE 471/571 – Lecture 2 Bayesian Decision Theory 08/25/15.

ECE 471/571 – Lecture 6 Dimensionality Reduction – Fisher’s Linear Discriminant 09/08/15.

Bayesian Decision Theory Basic Concepts Discriminant Functions The Normal Density ROC Curves.

Chapter 20 Classification and Estimation Classification – Feature selection Good feature have four characteristics: –Discrimination. Features.

ECE 471/571 - Lecture 19 Review 11/12/15. A Roadmap 2 Pattern Classification Statistical ApproachNon-Statistical Approach SupervisedUnsupervised Basic.

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 04: GAUSSIAN CLASSIFIERS Objectives: Whitening.

METU Informatics Institute Min720 Pattern Classification with Bio-Medical Applications Part 9: Review.

Basic Technical Concepts in Machine Learning Introduction Supervised learning Problems in supervised learning Bayesian decision theory.

ECE 471/571 – Lecture 21 Syntactic Pattern Recognition 11/19/15.

ECE 471/571 – Lecture 3 Discriminant Function and Normal Density 08/27/15.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

Introduction to Classifiers Fujinaga. Bayes (optimal) Classifier (1) A priori probabilities: and Decision rule: given and decide if and probability of.

Objectives: Loss Functions Risk Min. Error Rate Class. Resources: DHS – Chap. 2 (Part 1) DHS – Chap. 2 (Part 2) RGO - Intro to PR MCE for Speech MCE for.

Lecture 2. Bayesian Decision Theory

Basic Technical Concepts in Machine Learning

Lecture 1.31 Criteria for optimal reception of radio signals.

Syntactic Pattern Recognition 04/28/17

ECE 471/571 - Lecture 19 Review 02/24/17.

Nonparametric Density Estimation – k-nearest neighbor (kNN) 02/20/17

Performance Evaluation 02/15/17

ICS 280 Learning in Graphical Models

Ch3: Model Building through Regression

ECE 471/571 – Lecture 18 Classifier Fusion 04/12/17.

Unsupervised Learning - Clustering 04/03/17

Special Topics In Scientific Computing

Unsupervised Learning - Clustering

Classifiers Fujinaga.

Lecture 26: Faces and probabilities

Course Outline MODEL INFORMATION COMPLETE INCOMPLETE

REMOTE SENSING Multispectral Image Classification

REMOTE SENSING Multispectral Image Classification

ECE 471/571 – Lecture 12 Perceptron.

Where did we stop? The Bayes decision rule guarantees an optimal classification… … But it requires the knowledge of P(ci|x) (or p(x|ci) and P(ci)) We.

Statistical NLP: Lecture 4

ECE 471/571 – Review 1.

Parametric Estimation

Pattern Recognition and Machine Learning

Hairong Qi, Gonzalez Family Professor

Classifiers Fujinaga.

LECTURE 07: BAYESIAN ESTIMATION

ECE – Pattern Recognition Lecture 16: NN – Back Propagation

ECE – Pattern Recognition Lecture 15: NN - Perceptron

ECE – Pattern Recognition Lecture 13 – Decision Tree

Mathematical Foundations of BME

Hairong Qi, Gonzalez Family Professor

Machine Learning – a Probabilistic Perspective

Hairong Qi, Gonzalez Family Professor

ECE – Lecture 1 Introduction.

ECE – Pattern Recognition Lecture 10 – Nonparametric Density Estimation – k-nearest-neighbor (kNN) Hairong Qi, Gonzalez Family Professor Electrical.

Hairong Qi, Gonzalez Family Professor

ECE – Pattern Recognition Lecture 8 – Performance Evaluation

ECE – Pattern Recognition Lecture 4 – Parametric Estimation

Hairong Qi, Gonzalez Family Professor

ECE – Pattern Recognition Lecture 14 – Gradient Descent

ECE – Pattern Recognition Midterm Review

Presentation transcript:

Bayesian Decision Theory ECE 471/571 – Lecture 2 Bayesian Decision Theory

Pattern Classification Statistical Approach Non-Statistical Approach Supervised Unsupervised Decision-tree Basic concepts: Baysian decision rule (MPP, LR, Discri.) Basic concepts: Distance Agglomerative method Syntactic approach Parameter estimate (ML, BL) k-means Non-Parametric learning (kNN) Winner-takes-all LDF (Perceptron) Kohonen maps NN (BP) Mean-shift Support Vector Machine Deep Learning (DL) Dimensionality Reduction FLD, PCA Performance Evaluation ROC curve (TP, TN, FN, FP) cross validation Stochastic Methods local opt (GD) global opt (SA, GA) Classifier Fusion majority voting NB, BKS

Bayes’ formula (Bayes’ rule) From domain knowledge a-priori probability (prior probability) Conditional probability density (pdf – probability density function) (likelihood) a-posteriori probability (posterior probability) normalization constant (evidence)

pdf examples Gaussian distribution Uniform distribution Bell curve Normal distribution Uniform distribution Rayleigh distribution

Bayes decision rule x Maximum a-posteriori Probability (MAP): Stock market x Maximum a-posteriori Probability (MAP):

Conditional probability of error

Decision regions The effect of any decision rule is to partition the feature space into c decision regions

Overall probability of error Or unconditional risk, unconditional probability of error 8

The conditional risk Given x, the conditional risk of taking action ai is: lij is the loss when decide x belongs to class i while it should be j lij

Likelihood ratio - two category classification

Zero-One loss

Recap Bayes decision rule  maximum a-posteriori probability Conditional risk Likelihood ratio Decision regions  How to calculate the overall probability of error

Recap Maximum a-posteriori Probability Likelihood ratio Overall probability of error