Machine Learning CMPUT 466/551 Nilanjan Ray Department of Computing Science University of Alberta

Introduction What is machine learning (ML)? Taxonomy in ML Applications/Examples Related disciplines References Resources

What is machine learning (ML)? Definition of “learning” from Merriam-Webster: “To gain knowledge or understanding of or skill in by study, instruction, or experience” ML = Learning in machines (computers) ML techniques are algorithms that enable the machines to improve its performance at some task through experience Tasks: recognition, diagnosis, prediction, planning, data mining, robot control, and so on.

Why Machine Learning? Some tasks can be specified only by training data/examples Human expertise may be scarce and/or very costly Amount of knowledge might be too large for explicit encoding by humans Modeling/Hidden parameter estimation: Often only data from measurements are available Computational power is ever increasing Growing data pool and storage capacity

A Brief History about ML Samuel’s checker program Rosenblatt’s perceptron 1960’s: Neural network Pattern recognition 1970’s: Winston’s ARCH Buchanan and Mitchell’s Meta-Dendral: mass spectrometry prediction rules Quinlan’s ID3: Chess end-game rules Michalski’s AQ11: Soybean disease diagnosis rules MACROPS: macro operators in block world planning

A Brief History about ML… 1980’s: Gains momentum Learning theory Symbolic learning algorithms Connectionist learning algorithm Clustering Explanation-based learning Knowledge guided inductive learning Genetic algorithm 1990’s: Maturity Data mining Ensemble learning: bagging, boosting etc. Kernel methods Reinforcement learning Theoretical analysis

Typical Taxonomy for ML Supervised learning Unsupervised learning Semi-supervised learning Reinforcement learning

Supervised Learning Training data available in the form of (input, output) pairs When output is continuous valued the problem is called regression; if the output is qualitative or categorical the problem is called classification The goal here is to estimate the output for a novel (never-seen-before) input, after learning the training data

A Supervised Learning Example (From [DHS] book) “Sorting incoming Fish on a conveyor according to species using optical sensing” Sea bass Species Salmon

Problem Analysis Set up a camera and take some sample images to extract features Length Lightness Width Number and shape of fins Position of the mouth, etc…

Preprocessing Use a segmentation operation to isolate fishes from one another and from the background Information from a single fish is sent to a feature extractor whose purpose is to reduce the data by measuring certain features The features are passed to a classifier

Feature for Classification Can we select the length of the fish as a possible feature for discrimination?

Let’s Try Another Feature Lightness (Intensity of image pixels)

Yet Another Feature: Width Adopt the lightness and add the width of the fish Fish xT = [x1, x2] Lightness Width

A Classifier So, how does the “Width” feature help?

Another Classifier

Unsupervised Learning No training data in the form of (input, output) pair is available Applications: Dimensionality reduction Data compression Outlier detection Classification Segmentation/clustering Probability density estimation …

Example: Unsupervised Learning DNA microarray data (taken from [HTF])

Example: Unsupervised Learning (contd..) Applications on DNA microarray Clustering: Group genes or samples into similar expression profiles Bi-clustering: Subset of genes exhibiting similar expression pattern along a subset of samples Dimension reduction …

Semi-supervised Learning Uses both labeled data (in the form (input, output) pairs) and unlabelled data for learning When labeling of data is a costly affair semi-supervised techniques could be very useful Examples: Generative models, self-training, co-training

Example: Semi-supervised Learning Source: Semi-supervised literature survey by X. Zhu, Technical Report

Reinforcement Learning Reinforcement learning is the problem faced by an agent that must learn behavior through trial-and-error interactions with a dynamic environment. There is no teacher telling the agent wrong or right There is critic that gives a reward / penalty for the agent’s action Applications: Robotics Combinatorial search problems, such as games Industrial manufacturing Many others!

Example: Reinforcement Learning   Tic Tac Toe TD-Gammon Goal Learn to play optimal game Learn to play game at master level States All possible board states - 9 All possible board states - 1020 Action A new X in an empty field 21 dice combinations & avg. 20 legal moves Reinforcement Signal +10 winning -1 for every move that did not win ‘1 ‘ for a reward ‘0‘ for a penalty

Related Disciplines Statistics Artificial Intelligence Psychology Vision and Neuroscience Control Theory Signal and Image Processing …..

References and Journals Text: The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman (book website: http://www-stat.stanford.edu/~tibs/ElemStatLearn/) Reference books: Pattern Classification by Duda, Hart and Stork Pattern Recognition and Machine Learning by C.M. Bishop Machine Learning by T. Mitchell Introduction to Machine Learning by E. Alpaydin Some related journals / associations: Machine Learning (Kluwer). Journal of Machine Learning Research. Journal of AI Research (JAIR). Data Mining and Knowledge Discovery - An International Journal. Journal of Experimental and Theoretical Artificial Intelligence (JETAI). Evolutionary Computation. Artificial Life. Fuzzy Sets and Systems IEEE Intelligent Systems (Formerly IEEE Expert) IEEE Transactions on Knowledge and Data Engineering IEEE Transactions on Pattern Analysis and Machine Intelligence IEEE Transactions on Systems, Man and Cybernetics Journal of AI Research Journal of Intelligent Information Systems Journal of the American Statistical Association Journal of the Royal Statistical Society

References and Journals… Pattern Recognition Pattern Recognition Letters Pattern Analysis and Applications. Computational Intelligence . Journal of Intelligent Systems . Annals of Mathematics and Artificial Intelligence. IDEAL, the online scientific journal library by Academic Press. ECCAI (European Coordinating Committee on Artificial Intelligence). AAAI (American Association for Artificial Intelligence). IJCAI (International Joint Conferences on Artificial Intelligence, Inc.). ACM (Association for Computing Machinery). Association for Uncertainty in Artificial Intelligence. ACM SIGAR ACM SIGMOD American Statistical Association. Artificial Intelligence Artificial Intelligence in Engineering Artificial Intelligence in Medicine Artificial Intelligence Review Bioinformatics Data and Knowledge Engineering Evolutionary Computation

Some Conferences & Workshops Congress on Evolutionary Computation European Conference on Machine Learning and Principles and Practice of Knowledge Discovery The ACM SIGKDD International Conference on Knowledge Discovery and Data Mining National Conference on Artificial Intelligence Genetic and Evolutionary Computation Conference International Conference on Machine Learning Conference on Autonomous Agents and Multiagent Systems European Symposium on Artificial Neural Networks Advances in Computational Intelligence and Learning Artificial and Ambient Intelligence Computational Intelligene in Biomedical Engineering IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning International Joint Conference on Artificial Intelligence