1. Learning from observations
Inductive Learning - learning from examples
Machine Learning

2. What Is Machine Learning?
“Logic is not the end of wisdom, it is just the beginning” --- Spock
time
Environment
changed
same
Knowledge
Environment
System
System
Action1
Action2
Knowledge

3. Learning & Adaptation
”Modification of a behavioral tendency by expertise.”
(Webster 1984)
”A learning machine, broadly defined is any device whose
actions are influenced by past experiences.” (Nilsson 1965)
”Any change in a system that allows it to perform better
the second time on repetition of the same task or on another
task drawn from the same population.” (Simon 1983)
”An improvement in information processing ability that results
from information processing activity.” (Tanimoto 1990)

4. Ways humans learn things
…talking, walking, running…
Learning by mimicking, reading or being told facts
Tutoring
Being informed when one is correct
Experience
Feedback from the environment
Analogy
Comparing certain features of existing knowledge to new problems
Self-reflection
Thinking things in ones own mind, deduction, discovery

7. A few achievements Programs that can: Recognize spoken words
Predict recovery rates of pneumonia patients
Detect fraudulent use of credit cards
Drive autonomous vehicles
Play games like backgammon – approaching the human champion!

8. Machine Learning
Machine learning involves automatic procedures that learn a task from a series of examples
Most convenient source of examples is data

9. Learning
Definition:
A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience.

10. Machine Learning Models
Classification
Clustering
Regression
Time series analysis
Association Analysis
Sequence Discovery ….

11. Text Classification Example
Yes No
About the Middle East
conflict?
Text
Classification
Assign items to one of a set of predefined classes of objects based on a set of observed features
Text

12. Text Classification Example
Yes No
About the Middle East
conflict?
Text
Classification
Assign items to one of a set of predefined classes of objects based on a set of observed features

13. Clustering
Seeks to place objects into meaningful groups automatically, based on their similarity. Does not require the groups to be predefined. The hope in applying clustering algorithms is that they will discover useful but unknown classes of items.

14. Classification example
New data
Train set
Test set
Loan
Yes/No
Model
Learning system

15. Inductive learning Simplest form: learn a function from examples
f is the target function
An example is a pair (x, f(x))
Problem: find a hypothesis h
such that h ≈ f
given a training set of examples

16. Inductive learning method
Construct/adjust h to agree with f on training set
(h is consistent if it agrees with f on all examples)
E.g., curve fitting:

17. Inductive learning method
Construct/adjust h to agree with f on training set
(h is consistent if it agrees with f on all examples)
E.g., curve fitting:

18. Inductive learning method
Construct/adjust h to agree with f on training set
(h is consistent if it agrees with f on all examples)
E.g., curve fitting:

19. Inductive learning method
Construct/adjust h to agree with f on training set
(h is consistent if it agrees with f on all examples)
E.g., curve fitting:

20. Inductive learning method
Construct/adjust h to agree with f on training set
(h is consistent if it agrees with f on all examples)
E.g., curve fitting:

21. Inductive learning method
Construct/adjust h to agree with f on training set
(h is consistent if it agrees with f on all examples)
E.g., curve fitting:
Ockham’s razor: prefer the simplest hypothesis consistent with data

22. Machine Learning Methods
Instance Based Methods (CBR, k-NN)
Decision Trees
Artificial Neural Networks
Bayesian Networks
Naïve Base
Evolutionary Strategies
Support Vector Machines ..

23. Classification example
Features: height, weight x x o x x o x o x o o
Height x o x o o x x o x x o o o x
x - weight-lifters
o - ballet dancers x
Weight

24. Classification example - Simple Model
Features: height, weight
Decision boundary x x o x x o x o x o o
Height x o x o o x x o x x o o o x
x - weight-lifters
o - ballet dancers x
Weight

25. Classification example - Complex model
Features: height, weight
Complex Decision boundary x x o x x o x o x o o
Height x o x o o x x o x x o o o x
x - weight-lifters
o - ballet dancers x
Weight
Note: A simple decision boundary is better than a complex one - It GENERALIZES better.

26. Learning Paradigms Supervised learning - with teacher
inputs and correct outputs are provided by the teacher
Reinforced learning - with reward or punishment
an action is evaluated
Unsupervised learning - with no teacher
no hint about correct output is given

27. Nearest Neighbor
Simple effective approach for supervised learning problems
Envision each example as a point in n-dimensional space
- Picture with 2 of them
Classify test point same as nearest training point (Euclidean distance)

28. k-Nearest Neighbor Nearest Neighbor can be subject to noise
Incorrectly classified training points
Training anomalies
k-Nearest Neighbor
Find k nearest training points (k odd) and vote on which classification
Works on numerical data