1. Introduction
Machine Learning
14/02/2017

2. Machine Learning
How can we design a computer system whose performance improves by learning from experience?

3. Spam filtering

4. Face/person recognition
demo

5. Recommendation systems

6. Robotics

7. Natural Language Processing

8. other application areas
Biometrics
Object recognition on images
DNA seqencing
Financial data mining/prediction
Process mining and optimisation
Pattern Classification, Chapter 1

9. Big Data

10. Rule-based systems vs. Machine learning10
Rule-based systems vs. Machine learning
Domain expert is needed for
writing rules OR
giving training sample
Which one is better?
Can the expert design rule-based systems?
Is the problem specific or general?

13. Most of the materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley & Sons, with the permission of the authors and the publisher

14. 14
Definition
Machine Learning (Mitchell): „a computer program said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.”

15. Example Classify fishes see bass Classes salmon15
Example
Classify fishes
see bass
Classes
salmon
Goal: to learn a modell from training data which can categorise fishes
(eg. salmons are shorter)
Pattern Classification, Chapter 1

16. 16
Classification(T)
Supervised learning: Based on training examples (E), learn a modell which works fine on previously unseen examples.
Classification: a supervised learning task of categorisation of entities into predefined set of classes
Pattern Classification, Chapter 1

17. 17
Pattern Classification, Chapter 1 17

18. Basic definitions Feature (or attribute) Instance Class label ID
Length (cm)
Lightness
Type 1 28
0.5
salmon 2 23
0.7 3 17
sea bass
Instance
(or entity, sample)
Class label

19. Example - Preprocessing Image processing steps Feature extraction19
Example -
Preprocessing
Image processing steps
E.g segmentation of fish contour and background
Feature extraction
Extraction of features/attributes from images which are atomic variables
Typically numerical or categorical
Pattern Classification, Chapter 1

20. Example features length lightness width number of paddles20
Example features
length
lightness
width
number of paddles
position of mouth
Pattern Classification, Chapter 1

21. Length is a weak discriminator of fish types.21
Length is a weak discriminator of fish types.
Pattern Classification, Chapter 1

22. 22
Lightness is better
Pattern Classification, Chapter 1

23. Performance evaluation (P)23
Performance evaluation (P)
most simple: accuracy (correct rate)
False positive/negative errors
E.g. if the threshold is decreased the number of sea basses falsly classified to salmon decreases
Decision theory
Pattern Classification, Chapter 1

24. Feature vector A vector of features describing a particular instance.24
Feature vector
A vector of features describing a particular instance.
InstanceA xT = [x1, x2]
Lightness
Width
Pattern Classification, Chapter 1

25. 25
Pattern Classification, Chapter 1

26. Feature space We need features which might have discriminative power.26
Feature space
Be careful by adding to many features
noisy features (eg. measurement errors)
Unnecessary (pl. information content is similar to other feature)
We need features which might have discriminative power.
Feature set engineering is highly task-specific!
Pattern Classification, Chapter 1

27. This is not ideal. Remember supervised learning principle!27
This is not ideal. Remember supervised learning principle!
Pattern Classification, Chapter 1

28. 28
Pattern Classification, Chapter 1

29. Modell selection Number of features? Complexity of the task?29
Modell selection
Number of features?
Complexity of the task?
Classifier speed?
Task and data-dependent!
Pattern Classification, Chapter 1

30. The machine learning lifecycle30
The machine learning lifecycle
Data preparation
Feature engineering
Modell selection
Modell training
Performance evaluation
Pattern Classification, Chapter 1

31. 31
Data preparation
Do we know whether we collected enough and representative sample for training a system?
Pattern Classification, Chapter 1

32. Modell selection and training32
Modell selection and training
These topics are the foci of this course
Investigate the data for modell selection!
No free lunch!
Pattern Classification, Chapter 1

33. Performance evaluation33
Performance evaluation
There are various evaluation metrics
Simulation of supervised learning:
split your data into two parts
train your modell on the training set
predict and evaluate your modell on the test set (unknow during training)
Pattern Classification, Chapter 1

34. Topics of the course Classification Regression Clustering34
Topics of the course
Classification
Regression
Clustering
Recommendation systems
Learning to rank
Structure prediction
Reinforcement learning