Presentation is loading. Please wait.

Presentation is loading. Please wait.

Parametric Estimation

Similar presentations


Presentation on theme: "Parametric Estimation"— Presentation transcript:

1 Parametric Estimation
ECE 471/571 – Lecture 4 Parametric Estimation

2 Pattern Classification
Statistical Approach Non-Statistical Approach Supervised Unsupervised Decision-tree Basic concepts: Baysian decision rule (MPP, LR, Discri.) Basic concepts: Distance Agglomerative method Syntactic approach Parameter estimate (ML, BL) k-means Non-Parametric learning (kNN) Winner-takes-all LDF (Perceptron) Kohonen maps NN (BP) Mean-shift Support Vector Machine Deep Learning (DL) Dimensionality Reduction FLD, PCA Performance Evaluation ROC curve (TP, TN, FN, FP) cross validation Stochastic Methods local opt (GD) global opt (SA, GA) Classifier Fusion majority voting NB, BKS

3 Bayes Decision Rule Maximum Posterior Probability Likelihood Ratio
Discriminant Function

4 Normal/Gaussian Density
The rule

5 Multivariate Normal Density

6 Estimating Normal Densities
Calculate m, S

7 Covariance

8 *Why? – Maximum Likelihood Estimation
Compare “likelihood”

9 *Derivation

10 * Derivative of a Quadratic Form

11 **Why? – Baysian Estimation
Maximum likelihood estimation The parameters are fixed Find value for q that best agrees with or supports the actually observed training samples – likelihood of q w.r.t. the set of samples Baysian estimation Treat parameters as random variable themselves

12 ** The pdf of the parameter (m) is Gaussian

13 ** Derivation

14 ** mn and sn Behavior of Bayesian learning
Our best guess for m after observing n samples Measures our uncertainty about this guess Behavior of Bayesian learning The larger the n, the smaller the sn – each additional observation decreases our uncertainty about the true value of m As n approaches infinity, p(m|D) becomes more and more sharply peaked, approaching a Dirac delta function. mn is a linear combination between the sample mean and m0


Download ppt "Parametric Estimation"

Similar presentations


Ads by Google