1. What is Learning All about ?  Get knowledge of by study, experience, or being taught  Become aware by information or from observation  Commit to memory  Be informed of or receive instruction

2. A Possible Definition of Learning Things learn when they change their behavior in a way that makes them perform better in the future. Have your shoes learned the shape of your foot ? In learning the purpose is the learner’s, whereas in training it is the teacher’s

3. Our Learning Tasks in the Class  Classification (Supervised learning)  binary classification problem  multi-class classification problem  Regression (Supervised learning)  Does your machine learn anything from you?  Who/which is the better teacher/algorithm?

4. The Mathematical Background Material in the Class  Calculus (Multi-variable)  What is the gradient of function  Linear Algebra  How to compute the distance between two parallel hyperplanes in ?  eigenvalue, positive definite matrix, inner product, projection matrix etc.

5. Basic Concepts of Probability and Statistics Basic Concepts of Probability and Statistics  Probability:  Statistics: Random variables, probability distribution, expected value (mean), variance … Confidence interval, testing hypothesis …

6. Classification Problem 2-Category Linearly Separable Case Classification Problem 2-Category Linearly Separable Case A- A+ Malignant Benign

7. Support Vector Machines Maximizing the Margin between Bounding Planes A+ A-

8. Why Use Support Vector Machines (SVMs)? Powerful tools for Data Mining  SVM classifier is an optimally defined surface  SVMs have a good geometric interpretation  SVMs can be generated very efficiently  Can be extended from linear to nonlinear case  Typically nonlinear in the input space  Linear in a higher dimensional “feature” space  Implicitly defined by a kernel function  Have a sound theoretical foundation  Based on Statistical Learning Theory

9. Why We Maximize the Margin? (Based on Statistical Learning Theory)  The Structural Risk Minimization (SRM):   The expected risk will be less than or equal to empirical risk (training error)+ VC (error) bound 

10. Two-spiral Dataset (94 White Dots & 94 Red Dots)