1. Machine Learning CMPUT 466/551
Nilanjan Ray
Department of Computing Science
University of Alberta

2. Introduction What is machine learning (ML)? Taxonomy in ML
Applications/Examples
Related disciplines
References
Resources

3. What is machine learning (ML)?
Definition of “learning” from Merriam-Webster: “To gain knowledge or understanding of or skill in by study, instruction, or experience”
ML = Learning in machines (computers)
ML techniques are algorithms that enable the machines to improve its performance at some task through experience
Tasks: recognition, diagnosis, prediction, planning, data mining, robot control, and so on.

4. Why Machine Learning?
Some tasks can be specified only by training data/examples
Human expertise may be scarce and/or very costly
Amount of knowledge might be too large for explicit encoding by humans
Modeling/Hidden parameter estimation: Often only data from measurements are available
Computational power is ever increasing
Growing data pool and storage capacity

5. A Brief History about ML
Samuel’s checker program
Rosenblatt’s perceptron
1960’s:
Neural network
Pattern recognition
1970’s:
Winston’s ARCH
Buchanan and Mitchell’s Meta-Dendral: mass spectrometry prediction rules
Quinlan’s ID3: Chess end-game rules
Michalski’s AQ11: Soybean disease diagnosis rules
MACROPS: macro operators in block world planning

6. A Brief History about ML…
1980’s: Gains momentum
Learning theory
Symbolic learning algorithms
Connectionist learning algorithm
Clustering
Explanation-based learning
Knowledge guided inductive learning
Genetic algorithm
1990’s: Maturity
Data mining
Ensemble learning: bagging, boosting etc.
Kernel methods
Reinforcement learning
Theoretical analysis

7. Typical Taxonomy for ML
Supervised learning
Unsupervised learning
Semi-supervised learning
Reinforcement learning

8. Supervised Learning
Training data available in the form of (input, output) pairs
When output is continuous valued the problem is called regression; if the output is qualitative or categorical the problem is called classification
The goal here is to estimate the output for a novel (never-seen-before) input, after learning the training data

9. A Supervised Learning Example (From [DHS] book)
“Sorting incoming Fish on a conveyor according to species using optical sensing”
Sea bass
Species
Salmon

10. Problem Analysis
Set up a camera and take some sample images to extract features
Length
Lightness
Width
Number and shape of fins
Position of the mouth, etc…

11. Preprocessing
Use a segmentation operation to isolate fishes from one another and from the background
Information from a single fish is sent to a feature extractor whose purpose is to reduce the data by measuring certain features
The features are passed to a classifier

13. Feature for Classification
Can we select the length of the fish as a possible
feature for discrimination?

14. Let’s Try Another Feature
Lightness (Intensity of image pixels)

15. Yet Another Feature: Width
Adopt the lightness and add the width of the fish
Fish xT = [x1, x2]
Lightness
Width

16. A Classifier
So, how does the “Width” feature help?

17. Another Classifier

18. Unsupervised Learning
No training data in the form of (input, output) pair is available
Applications:
Dimensionality reduction
Data compression
Outlier detection
Classification
Segmentation/clustering
Probability density estimation …

19. Example: Unsupervised Learning
DNA microarray data (taken from [HTF])

20. Example: Unsupervised Learning (contd..)
Applications on DNA microarray
Clustering: Group genes or samples into similar expression profiles
Bi-clustering: Subset of genes exhibiting similar expression pattern along a subset of samples
Dimension reduction …

21. Semi-supervised Learning
Uses both labeled data (in the form (input, output) pairs) and unlabelled data for learning
When labeling of data is a costly affair semi-supervised techniques could be very useful
Examples: Generative models, self-training, co-training

22. Example: Semi-supervised Learning
Source: Semi-supervised literature survey by X. Zhu, Technical Report

23. Reinforcement Learning
Reinforcement learning is the problem faced by an agent that must learn behavior through trial-and-error interactions with a dynamic environment.
There is no teacher telling the agent wrong or right
There is critic that gives a reward / penalty for the agent’s action
Applications:
Robotics
Combinatorial search problems, such as games
Industrial manufacturing
Many others!

24. Example: Reinforcement Learning
Tic Tac Toe
TD-Gammon
Goal
Learn to play optimal game
Learn to play game at master level
States
All possible board states - 9
All possible board states
Action
A new X in an empty field
21 dice combinations & avg. 20 legal moves
Reinforcement Signal
+10 winning
-1 for every move that did not win
‘1 ‘ for a reward
‘0‘ for a penalty

25. Related Disciplines Statistics Artificial Intelligence Psychology
Vision and Neuroscience
Control Theory
Signal and Image Processing
…..

26. References and Journals
Text: The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman (book website:
Reference books:
Pattern Classification by Duda, Hart and Stork
Pattern Recognition and Machine Learning by C.M. Bishop
Machine Learning by T. Mitchell
Introduction to Machine Learning by E. Alpaydin
Some related journals / associations:
Machine Learning (Kluwer).
Journal of Machine Learning Research.
Journal of AI Research (JAIR).
Data Mining and Knowledge Discovery - An International Journal.
Journal of Experimental and Theoretical Artificial Intelligence (JETAI).
Evolutionary Computation.
Artificial Life.
Fuzzy Sets and Systems
IEEE Intelligent Systems (Formerly IEEE Expert)
IEEE Transactions on Knowledge and Data Engineering
IEEE Transactions on Pattern Analysis and Machine Intelligence
IEEE Transactions on Systems, Man and Cybernetics
Journal of AI Research
Journal of Intelligent Information Systems
Journal of the American Statistical Association
Journal of the Royal Statistical Society

27. References and Journals…
Pattern Recognition
Pattern Recognition Letters
Pattern Analysis and Applications.
Computational Intelligence .
Journal of Intelligent Systems .
Annals of Mathematics and Artificial Intelligence.
IDEAL, the online scientific journal library by Academic Press.
ECCAI (European Coordinating Committee on Artificial Intelligence).
AAAI (American Association for Artificial Intelligence).
IJCAI (International Joint Conferences on Artificial Intelligence, Inc.).
ACM (Association for Computing Machinery).
Association for Uncertainty in Artificial Intelligence.
ACM SIGAR
ACM SIGMOD
American Statistical Association.
Artificial Intelligence
Artificial Intelligence in Engineering
Artificial Intelligence in Medicine
Artificial Intelligence Review
Bioinformatics
Data and Knowledge Engineering
Evolutionary Computation

28. Some Conferences & Workshops
Congress on Evolutionary Computation
European Conference on Machine Learning and Principles and Practice of Knowledge Discovery
The ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
National Conference on Artificial Intelligence
Genetic and Evolutionary Computation Conference
International Conference on Machine Learning
Conference on Autonomous Agents and Multiagent Systems
European Symposium on Artificial Neural Networks Advances in Computational Intelligence and Learning
Artificial and Ambient Intelligence
Computational Intelligene in Biomedical Engineering
IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning
International Joint Conference on Artificial Intelligence