1. Introduction

2. Some labeled training examples

3. Bag-of-words bit vector
4 USENET groups
comp
rec
sci
comp,
talk
Bag-of-words bit vector
No single threshold value will serve to unambiguously discriminate between the two categories;
The value marked l∗ will lead to the smallest number of errors, on average.

4. Three types of iris flowers
setosa
versicolor
virginica

5. red: setosa green: versicolor blue: virginica
Which flower is easiest to classify?

6. Features permuted

7. Face detection

8. Regression

9. Unsupervised learning

10. Principal components dimensionality reduction
2D linear subspace embedded in 3D
2D representation
of the data

11. 25 individual faces

12. Eigenfaces

13. Missing data A noisy image with an occluder.
An estimate of the underlying pixel intensities, based on a pairwise Markov random field model.

14. Voronoi Tessellation
Euclidean distance
Manhattan distance

15. 3-NN

16. 10-nearest neighbors: red class

17. 10-nearest neighbors: blue class

18. Maximum a posteriori of class labels
blue: class 2

19. Polynomial Regression
degree 14
degree 20

20. Sigmoid or logistic function

21. Sigmoid or logistic function

22. Logistic regression Solid black dots are SAT scores.
accept?
Solid black dots are SAT scores.
The open red circles are the predicted probabilities of acceptance.
The green crosses denote two students with the same SAT score of 525
logistic regression is a form of classification, not regression!
SAT scores

23. KNN
K=1
K=5