1. Tying up loose ends

2.  Understand your data

3.  No answers available, only data

4.  Clustering, SOM, Hebbian learning, PCA…

5.  Training includes inputs and correct answers

6.  Perceptron, Backprop, POS tagging

7.  Probability of Y given X

8.  Or the most likely Y given X

9.  Probability of Y given X  Or the most likely Y given X  Collaborative Filtering – people who like X probably like Y

10.  Probability of Y given X  Or the most likely Y given X  Collaborative Filtering – people who like X probably like Y  Neural Networks – input X triggers Y output (behaviorism)

11.  Input retrieves similarities or correlations as output

12.  X is a…

13.  X is A or B or C or D

14.  X is a…  X is A or B or C or D  X is 1 or 0

15.  X is a…  X is A or B or C or D  X is face or not-face

16.  Goal is prediction

17.  Classification is a type of association

18.  Goal is prediction  Classification is a type of association  Includes pattern recognition: OCR, faces, diagnosis, speech, NLP…

19.  Goal is prediction  Classification is a type of association  Includes pattern recognition: OCR, faces, diagnosis, speech, NLP…  Includes compression

20.  If the output is a continuous number

21.  Ex. Automatic steering  inputs: sensors (video, GPS, proximity…)  output: degree of rotation of the wheel  Ex. ALVINN Ex. ALVINN

22.  Backprop Neural Nets work for both

23.  Different algorithms use different error calculations

24.  Simplest : # wrong / # total  ie. 2/5 =.4 or 40%

25.  Different algorithms use different error calculations  Simplest : # wrong / # total  ie. 2/5 =.4 or 40%  Other examples:  WER  Mean Squared Error

26. Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output

27. Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Validation Training

28. Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Fold 1 Fold 2 Fold 3 Fold 4 Fold 5

29. Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Fold 1 Fold 2 Fold 3 Fold 4 Fold 5 Train Test -> Learner 1 error =.01

30. Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Fold 1 Fold 2 Fold 3 Fold 5 Fold 4 Train Test -> Learner 2 error =.012

31. Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Input Output Fold 1 Fold 2 Fold 3 Fold 5 Fold 3 Train Test -> Learner 3 error =.011

32.  If errors between folds vary greatly this indicated bias in training

33.  Over-fitting – too much training

34.  Over-fitting  Misrepresentative data