Download presentation
Presentation is loading. Please wait.
1
Parametric Estimation
ECE 471/571 – Lecture 4 Parametric Estimation
2
Pattern Classification
Statistical Approach Non-Statistical Approach Supervised Unsupervised Decision-tree Basic concepts: Baysian decision rule (MPP, LR, Discri.) Basic concepts: Distance Agglomerative method Syntactic approach Parameter estimate (ML, BL) k-means Non-Parametric learning (kNN) Winner-takes-all LDF (Perceptron) Kohonen maps NN (BP) Mean-shift Support Vector Machine Deep Learning (DL) Dimensionality Reduction FLD, PCA Performance Evaluation ROC curve (TP, TN, FN, FP) cross validation Stochastic Methods local opt (GD) global opt (SA, GA) Classifier Fusion majority voting NB, BKS
3
Bayes Decision Rule Maximum Posterior Probability Likelihood Ratio
Discriminant Function
4
Normal/Gaussian Density
The rule
5
Multivariate Normal Density
6
Estimating Normal Densities
Calculate m, S
7
Covariance
8
*Why? – Maximum Likelihood Estimation
Compare “likelihood”
9
*Derivation
10
* Derivative of a Quadratic Form
11
**Why? – Baysian Estimation
Maximum likelihood estimation The parameters are fixed Find value for q that best agrees with or supports the actually observed training samples – likelihood of q w.r.t. the set of samples Baysian estimation Treat parameters as random variable themselves
12
** The pdf of the parameter (m) is Gaussian
13
** Derivation
14
** mn and sn Behavior of Bayesian learning
Our best guess for m after observing n samples Measures our uncertainty about this guess Behavior of Bayesian learning The larger the n, the smaller the sn – each additional observation decreases our uncertainty about the true value of m As n approaches infinity, p(m|D) becomes more and more sharply peaked, approaching a Dirac delta function. mn is a linear combination between the sample mean and m0
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.