Introduction to Machine Learning Nir Ailon Lecture 11: Probabilistic Models

Most of the Course So Far: Discriminative Approach “Bayes Optimal”

ERM Can Sometimes Be Viewed as Discriminative Approach for a ``Made Up’’ Probabilistic Method Gaussian Density

Class-Conditional Density Class Prior

Why Not Generative Approach

Why Generative Approach?

Stats 101: Maximum Likelihood Estimator (MLE)

Example: MLE For Biased Coin

Abuse of notation! Should be density… MLE for Continuous R.V.’s

Naïve Bayes Approach Conditional Independence

Naïve Bayes Classifier (Binary Case) It’s a linear model!

Depends on coordinate only Depends on coordinate & label Naïve Bayes Classifier (Gaussian Case) It’s a linear model!

(Gaussian) Naïve Bayes vs Linear Regression

Bayesian Reasoning

Bayesian Priors vs SRM

Because of conditional independence Posterior Bayesian Reasoning Bayes Average Laplace Smoothing

Difficulties in Bayes Reasoning

MAP

Summary