Pattern Recognition ->Machine Learning- >Data Analytics Supervised Learning Unsupervised Learning Semi-supervised Learning Reinforcement Learning

Bayes Theorem Conditional probability: P(A|B), P(B|A) Marginal probability: P(A), P(B) Joint probability: P(A,B) P(AB) Bayes theorem: P(A), P(B), P(A|B) →P(B|A) P(A), P(B), P(B|A) →P(A|B)

Example P(C) = 0.01 (1%) P(pos|C) = 0.90 (90%) → 90% test is positive if you have C Sensitivity P(neg|~C) = 0.90 (90%) → 90% test is negative if you don’t have C Prior Specificity Question: if test is positive, the probability of having C ? P(C|pos) = ?

All people C

Naïve Bayes (Example: Text learning) Chris Sara LifeWorkFamily LifeWorkFamily P(Chris) = 0.5 P(Sara) = 0.5 Life Family Chris or Sara? Life Work Chris or Sara?

Naïve Bayes Prior: P(y 1 ), P(y 2 )…., P(y m ) Conditional prob: P(x 1,x 2,…,x n |y 1 ),…. P(x 1,x 2,…,x n |y m ) Solution: argmax(P(y j |x 1,x 2,…,x n ) ) j Naïve: assume independence of x 1, x 2,…, x n

Gaussian Naïve Bayes Conditional prob: P(x i |y) ~ N(µ y, σ y ) Implementation: Scikit learning-sklearn

Gaussian Naïve Bayes Self-driving car dataset

Gaussian Naïve Bayes

Support Vector Machine (SVM) Basic idea

Support Vector Machine (SVM) Basic idea

Support Vector Machine (SVM) Basic idea

Support Vector Machine (SVM) Basic idea Maximize distance to nearest point Maximize margin Support vector

Support Vector Machine (SVM) Basic idea Maximize robustness of classifier

Support Vector Machine (SVM) Basic idea

Support Vector Machine (SVM) Basic idea 1.Lower classifier error ! 2.Maximize margin

Support Vector Machine (SVM) Outlier outlier

Support Vector Machine (SVM) Outlier outlier

Support Vector Machine (SVM) Will SVMs work ? 1.Yes 2.No

Support Vector Machine (SVM) Trick Features x y SVM Label

Support Vector Machine (SVM) Is this linearly separable? x y SVM Label X 2 +y 2

Support Vector Machine (SVM) x y SVM Label Z=X 2 +y 2

Support Vector Machine (SVM) x y SVM Label Z=X 2 +y 2

Support Vector Machine (SVM) Add one more feature to make linearly separable 1.x 2 +y 2 2.|x| 3.|y|

Support Vector Machine (SVM) Add one or more non-linear features to make linearly separable Kernel Trick x, y x 1 x 2 x 3 x 4 x 5 Kernel Not separableSeparable Solution(linear boundary) Non-linear separable

Support Vector Machine (SVM) Nonlinear Kernels Sigmoid function

Implementation (sklearn SVM)

Naïve Bayes, Accuracy: Linear SVM, Accuracy: 0.92RBF SVM, Accuracy: 0.94