1. Introduction to Machine Learning 236756 Nir Ailon Lecture 11: Probabilistic Models

2. Most of the Course So Far: Discriminative Approach “Bayes Optimal”

3. ERM Can Sometimes Be Viewed as Discriminative Approach for a ``Made Up’’ Probabilistic Method Gaussian Density

4. Class-Conditional Density Class Prior

5. Why Not Generative Approach

6. Why Generative Approach?

7. Stats 101: Maximum Likelihood Estimator (MLE)

8. Example: MLE For Biased Coin

9. Abuse of notation! Should be density… MLE for Continuous R.V.’s

10. Naïve Bayes Approach Conditional Independence

11. Naïve Bayes Classifier (Binary Case) It’s a linear model!

12. Depends on coordinate only Depends on coordinate & label Naïve Bayes Classifier (Gaussian Case) It’s a linear model!

13. (Gaussian) Naïve Bayes vs Linear Regression

14. Bayesian Reasoning

15. Bayesian Priors vs SRM

16. Because of conditional independence Posterior Bayesian Reasoning Bayes Average Laplace Smoothing

17. Difficulties in Bayes Reasoning

18. MAP

19. Summary