1. Machine learning: What is it?
Machine learning is programming computers to optimize a performance criterion using example data or past experience. 1
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)

2. Machine learning: Who needs it?
No need to “learn” to calculate payroll; program once
Humans are unable to explain their expertise (speech recognition)
Solution changes in time (routing on a computer network)
Solution needs to be adapted to particular cases (user biometrics)

3. Face Recognition Training examples of a person Very good at
Test images
Very good at
recognizing faces
but can’t say exactly
how it works
How do we get
the data?
What features are
important?
ORL dataset,
AT&T Laboratories, Cambridge UK 3
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0) 3

4. Machine Leaning = Data Mining?
Retail: Market basket analysis
Finance: Credit scoring, fraud detection
Bioinformatics: Motifs, sequence alignment
Web mining: Search engines
Medicine: Medical diagnosis
Telecommunications: Spam filters
What do these problems have in common?
What do we want from the data? 4
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)

5. Machine Leaning = Data Mining?
Navigating a car: Kinematics of a robot arm as a function location (x,y)
α1= g1(x,y)
α2= g2(x,y) α1 α2
(x,y)
What is the data?
What do we want from the data? 5
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)

6. Machine Leaning = Data Mining?
Predictive-model control: train neural network to
represent manufacturing dynamics
Predict future plant output from input attributes

7. Components of Machine Learning
Inductive bias: hypothesis about data
Statistics: inference from a sample
Optimization: refine hypothesis with data
Assessment: accuracy of generalization 7
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)

8. Types of Machine Learning
Association
Supervised Learning
Classification
Regression
Unsupervised Learning
Reinforcement Learning 8
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)

9. Learning Associations
Estimate a conditioned probability Example: Basket analysis P (Y | X ) probability that somebody who buys X also buys Y P ( chips | beer ) = 0.7 Hidden variables: customer is male, she’s married, chips are salty, etc 9
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)

10. Learning Associations
Goal of machine learning is the pattern, Not the reason for the pattern Reason for pattern may not be as useful as the pattern itself Example: theory of evolution 10
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)

11. Supervised Learning: Mapping input (data) to output (correct value provided by expert)
Uses
Prediction of future cases: Use the rule to predict the output for future inputs (generalization)
Knowledge extraction: The rule suggests a mechanism
Compression: The rule is simpler than the data it explains
Outlier detection: Exceptions that are not covered by the rule, e.g., fraud 11
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)

12. Unsupervised Learning
Learning “what normally happens”
Only input --- No output to map to
Clustering: Grouping similar instances
PCA: quantitative segregation of data
Example applications
Customer segmentation in retailing
Image compression: Color quantization
Bioinformatics: Learning motifs 12
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)

13. Unsupervised learning sometimes precedes supervised learning
Unsupervised learning (clustering, PCA, etc.)
discovers the attribute space where it is possible
to assign a label
family car
C denotes training set
t is index of training set member
xt is vector of attributes (dimensionality of problem)
rt is label
Label can be either class (classification problem) or real number (regression problem)
This classification problem has only one class (family car) “dichotomizer”
Defined by 2 attributes (price and engine size)
Examples belonging to class are called “positive” and have r = 1
Examples not in class are called “negative” and have r = 0
For some machine learning techniques (SVM) better to assign r = -1 to negative examples
Unsupervised learning: finding attribute space were class members “cluster” 13
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)

14. Reinforcement Learning
Learning a strategy by reward and penalty in a sequence of outputs.
No one “correct” output.
Examples
Credit assignment problem (bank loans)
Game playing
Robot in a maze
Multiple agents, partial observability, ... 14
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)

15. Can we model machine learning after human learning?
Brain has processing units (neurons) with connections (synapses) between them
Large number of neurons: 1010
High connectitivity: 105
Capable of parallel processing
Distributed reasoning and memory
Robust to noise and failures
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0) 15

16. Brief History of Artificial Neural Networks (ANN)
1st mathematical model of ANN (McCulloch and Pit 1943) shows ANN could, in principle, compute arithmetic and logical functions
Rosenblatt proposes perceptron and learning method. Publishes Principles of Neurodynamics (Spartan press 1961)
Minsky and Papert (1969) show limitation of Rosenblatt perceptron. Delays ANN development for years. 16
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)

17. Paul Werbos (1974) describes back propagation training method in PhD thesis. No one takes notice
Hopfield (1982) proposes multi-layer perceptron (MLP) trained by back propagation
Stanford group publishes Parallel Distributed Processing (1986) and restarts ANN research
Volume 1 of Parallel Distributed Processing shows that limitations of Rosenblatt’s perceptron pointed out by Minsky and Papert can be overcome by MLP and popularizes back-propagation as training method
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0) 17

18. ANNs are de-mystified
ANN joins ranks of non-parametric statistical methods
Back-propagation recognized as non-biological
Genome sequencing stimulates vast data-mining
New methods of data mining start replacing ANN

19. Rise and fall of supervised machine learning techniques, Jensen and Bateman, Bioinformatics 2011
ANN still exceeds the
sum of other methods

20. Weka allows many different
algorithms to easily be
applied to the same dataset
Some techniques
(Decision tree) are
more “interpretable”
than ANN