Machine Learning Lecture 11 Summary G53MLE | Machine Learning | Dr Guoping Qiu1

Topics Covered G53MLE | Machine Learning | Dr Guoping Qiu2  Topic 1: Introduction  Learning and learning systems  Design of learning systems - 5 steps approach: training sample collection/preparation, data representation, choose a learning model/paradigm, learning, testing  Basic maths – vector, matrix, calculus

Topics Covered G53MLE | Machine Learning | Dr Guoping Qiu3  Topic 2: Neural networks  Perceptron – model operation, perceptron learning rule, decision boundary (surface), limitations, linearly separable classes  ADLINE – model operation, delta rule (gradient descent learning), batch mode, online mode  Multi-layer perceptron (MLP) - model operation, nonlinear unit (sigmoid function), principles of learning rule (back-propagation algorithm), model capability (solve linearly non-separable problems)  Generalization, over fitting, stopping criteria

Topics Covered G53MLE | Machine Learning | Dr Guoping Qiu4  Topic 3: Bayesian learning  Basic probability theory  Bayesian theorem  Estimation of probabilities  Maximum a posteriori (MAP)  Maximum Likelihood (ML)  Bayesian classifiers  Naïve Bayesian classifier

Topics Covered G53MLE | Machine Learning | Dr Guoping Qiu5  Topic 4: Instance based learning  K nearest neighbour classifier (K-NN) – feature space neighbourhood concept, classifier construction and operation, choose a suitable values of K

Topics Covered G53MLE | Machine Learning | Dr Guoping Qiu6  Topic 5: Clustering analysis  Basic concept of data clustering and why it is useful  How to do data clustering (K-means algorithm) – operation of the algorithm  Link between K-means algorithm and gradient descent (the concept of clustering cost function or objective function)  Limitations/weaknesses of the basic K-means algorithm – sensitive to initial cluster centres, local minima

Topics Covered G53MLE | Machine Learning | Dr Guoping Qiu7  Topic 6: Data processing and representation  Concepts of correlation and redundancy  Covariance matrix  Concepts of feature extraction/dimensionality reduction  Principle and application of Principal Component Analysis (PCA)

Topics Covered G53MLE | Machine Learning | Dr Guoping Qiu8  Topic 7: Support vector machines  Model operation (how it works)  Support vectors  Max margin classifier  Linear SVMs  Concept of Soft margin classification  Principle of Non-linear SVMs and the “Kernel Trick”

Topics Covered G53MLE | Machine Learning | Dr Guoping Qiu9  Topic 8: Decision tree learning  Information gain  Decision tree construction (design) – picking the root node, recursive branching