Hairong Qi, Gonzalez Family Professor

Slides:

Advertisements

Similar presentations

Pattern Recognition and Machine Learning

Advertisements

Pattern Classification. Chapter 2 (Part 1): Bayesian Decision Theory (Sections ) Introduction Bayesian Decision Theory–Continuous Features.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

Classification and risk prediction

0 Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Chapter 2 (part 3) Bayesian Decision Theory Discriminant Functions for the Normal Density Bayes Decision Theory – Discrete Features All materials used.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

CHAPTER 4: Parametric Methods. Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 2 Parametric Estimation X = {

METU Informatics Institute Min 720 Pattern Classification with Bio-Medical Applications PART 2: Statistical Pattern Classification: Optimal Classification.

Principles of Pattern Recognition

Speech Recognition Pattern Classification. 22 September 2015Veton Këpuska2 Pattern Classification  Introduction  Parametric classifiers  Semi-parametric.

Ch 4. Linear Models for Classification (1/2) Pattern Recognition and Machine Learning, C. M. Bishop, Summarized and revised by Hee-Woong Lim.

1 E. Fatemizadeh Statistical Pattern Recognition.

1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 12 Sept 30, 2005 Nanjing University of Science & Technology.

ECE 471/571 – Lecture 2 Bayesian Decision Theory 08/25/15.

ECE 471/571 – Lecture 6 Dimensionality Reduction – Fisher’s Linear Discriminant 09/08/15.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

Introduction to Machine Learning Multivariate Methods 姓名 : 李政軒.

ECE 471/571 - Lecture 19 Review 11/12/15. A Roadmap 2 Pattern Classification Statistical ApproachNon-Statistical Approach SupervisedUnsupervised Basic.

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 04: GAUSSIAN CLASSIFIERS Objectives: Whitening.

METU Informatics Institute Min720 Pattern Classification with Bio-Medical Applications Part 9: Review.

ECE 471/571 – Lecture 21 Syntactic Pattern Recognition 11/19/15.

ECE 471/571 – Lecture 3 Discriminant Function and Normal Density 08/27/15.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

Pattern Classification Chapter 2(Part 3) 0 Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O.

Part 3: Estimation of Parameters. Estimation of Parameters Most of the time, we have random samples but not the densities given. If the parametric form.

Lecture 2. Bayesian Decision Theory

Syntactic Pattern Recognition 04/28/17

ECE 471/571 - Lecture 19 Review 02/24/17.

Nonparametric Density Estimation – k-nearest neighbor (kNN) 02/20/17

LECTURE 04: DECISION SURFACES

Performance Evaluation 02/15/17

LECTURE 10: DISCRIMINANT ANALYSIS

CH 5: Multivariate Methods

ECE 471/571 – Lecture 18 Classifier Fusion 04/12/17.

Unsupervised Learning - Clustering 04/03/17

Unsupervised Learning - Clustering

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Course Outline MODEL INFORMATION COMPLETE INCOMPLETE

REMOTE SENSING Multispectral Image Classification

REMOTE SENSING Multispectral Image Classification

ECE 471/571 – Lecture 12 Perceptron.

ECE 599/692 – Deep Learning Lecture 9 – Autoencoder (AE)

Pattern Recognition and Image Analysis

ECE 471/571 – Review 1.

Parametric Estimation

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Pattern Recognition and Machine Learning

Hairong Qi, Gonzalez Family Professor

Generally Discriminant Analysis

LECTURE 09: DISCRIMINANT ANALYSIS

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

ECE – Pattern Recognition Lecture 16: NN – Back Propagation

ECE – Pattern Recognition Lecture 15: NN - Perceptron

ECE – Pattern Recognition Lecture 13 – Decision Tree

Multivariate Methods Berlin Chen

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Multivariate Methods Berlin Chen, 2005 References:

Hairong Qi, Gonzalez Family Professor

Linear Discrimination

Test #1 Thursday September 20th

Hairong Qi, Gonzalez Family Professor

ECE – Lecture 1 Introduction.

ECE – Pattern Recognition Lecture 10 – Nonparametric Density Estimation – k-nearest-neighbor (kNN) Hairong Qi, Gonzalez Family Professor Electrical.

Bayesian Decision Theory

Hairong Qi, Gonzalez Family Professor

ECE – Pattern Recognition Lecture 8 – Performance Evaluation

ECE – Pattern Recognition Lecture 4 – Parametric Estimation

ECE – Pattern Recognition Lecture 14 – Gradient Descent

ECE – Pattern Recognition Midterm Review

Presentation transcript:

ECE471-571 – Pattern Recognition Lecture 3 – Discriminant Function and Normal Density Hairong Qi, Gonzalez Family Professor Electrical Engineering and Computer Science University of Tennessee, Knoxville http://www.eecs.utk.edu/faculty/qi Email: hqi@utk.edu

Pattern Classification Statistical Approach Non-Statistical Approach Supervised Unsupervised Decision-tree Basic concepts: Baysian decision rule (MPP, LR, Discri.) Basic concepts: Distance Agglomerative method Syntactic approach Parameter estimate (ML, BL) k-means Non-Parametric learning (kNN) Winner-takes-all LDF (Perceptron) Kohonen maps NN (BP) Mean-shift Support Vector Machine Deep Learning (DL) Dimensionality Reduction FLD, PCA Performance Evaluation ROC curve (TP, TN, FN, FP) cross validation Stochastic Methods local opt (GD) global opt (SA, GA) Classifier Fusion majority voting NB, BKS

Bayes Decision Rule Maximum Posterior Probability Likelihood If , then x belongs to class 1, otherwise, 2. Likelihood Ratio

Discrimimant Function One way to represent pattern classifier- use discriminant functions gi(x) For two-class cases, g_i(x) = P(\omega_i|x) g_i(x) = p(x|\omega_i)P(\omega_i) g_i(x) = \ln p(x|\omega_i)+\ln P(\omega_i)

Multivariate Normal Density

Discriminant Function for Normal Density

Case 1: Si=s2I The features are statistically independent, and have the same variance Geometrically, the samples fall in equal-size hyperspherical clusters Decision boundary: hyperplane of d-1 dimension

Linear Discriminant Function and Linear Machine

Minimum-Distance Classifier When P(wi) are the same for all c classes, the discriminant function is actually measuring the minimum distance from each x to each of the c mean vectors

Case 2: Si = S The covariance matrices for all the classes are identical but not a scalar of identity matrix. Geometrically, the samples fall in hyperellipsoidal Decision boundary: hyperplane of d-1 dimension Squared Mahalanobis distance

Case 3: Si = arbitrary The covariance matrices are different from each category Quadratic classifier Decision boundary: hyperquadratic for 2-D Gaussian

Case Study Calculate m Calculate S a1 3.1 11.70 a a2 3.0 1.30 a a3 1.9 0.10 a a4 3.8 0.04 a a5 4.1 1.10 a a6 1.9 0.40 a b1 8.3 1.00 b b2 3.8 0.20 b b3 3.9 0.60 b b4 7.8 1.20 b b5 9.1 0.60 b b6 15.4 3.60 b b7 7.7 1.60 b b8 6.5 0.40 b b9 5.7 0.40 b b10 13.6 1.60 b c1 10.2 6.40 c c2 9.2 7.90 c c3 9.6 3.10 c c4 53.8 2.50 c c5 15.8 7.60 c u1 5.1 0.4 b u2 12.9 5.0 c u3 13.0 0.8 b u4 2.6 0.1 a u5 30.0 0.1 o u6 20.5 0.8 o Calculate m Calculate S Derive the discriminant function gi(x)