PatReco: Discriminant Functions for Gaussians Alexandros Potamianos Dept of ECE, Tech. Univ. of Crete Fall

PatReco: Problem Solving 1.Data Collection 2.Data Analysis 3.Feature Selection 4.Model Selection 5.Model Training 6.Classification 7.Classifier Evaluation

Discriminant Functions  Define class boundaries (instead of class characteristics)  Dualism: Parametric class description  Bayes classifier  Decision boundary  Parametric Discriminant Functions

Normal Density  1D  Multi-D Full covariance Diagonal covariance Diagonal covariance + univariate  Mixture of Gaussians Usually diagonal covariance

Gaussian Discriminant Functions  Same variance ALL classes Hyper-planes  Different variance among classes Hyper-quadratics (hyper-parabolas, hyper- ellipses etc.)

Hyper-Planes  When the covariance matrix is common across Gaussian classes The decision boundary is a hyper-plane that is vertical to the line connecting the means of the Gaussian distributions If the a-priori probabilities of classes are equal the hyper-planes cuts the line connecting the Gaussian means in the middle  Euclidean classifier

Gaussian Discriminant Functions  Same variance ALL classes Hyper-planes  Different variance among classes Hyper-quadratics (hyper-parabolas, hyper- ellipses etc.)

Hyper-Quadratics  When the Gaussian class variances are different the boundary can be hyper-plane, multiple hyper-planes, hyper-sphere, hyper- parabola, hyper-elipsoid etc. The boundary in general in NOT vertical to the Gaussian mean connecting line If the a-priori probabilities of classes are equal the resulting classifier is a Mahalanobois classifier

Conclusions  Parametric statistical models describe class characteristics x by modeling the observation probabilities p(x|class)  Discriminant functions describe class boundaries parametrically  Parametric statistical models have an equivalent parametric discriminant function  For Gaussian p(x|class) distributions the decision boundaries are hyper-planes or hyper-quadratics