1. Dual Problem of Linear Program subject to Primal LP Dual LP subject to ※ All duality theorems hold and work perfectly!

2. PrimalDual Nonnegative variableInequality constraint Free variableEquality constraint Inequality constraint Nonnegative variable Equality constraintFree variable

3. Primal s. t. Dual s. t.

4. Primal Problem Feasible Region

5. Dual Problem of Strictly Convex Quadratic Program subject to Primal QP With strictly convex assumption, we have Dual QP subject to

6. Classification Problem 2-Category Linearly Separable Case A- A+ Malignant Benign

7. Support Vector Machines Maximizing the Margin between Bounding Planes A+ A-

8. Algebra of the Classification Problem 2-Category Linearly Separable Case  Given m points in the n dimensional real space  Represented by an matrix or  Membership of each point in the classes is specified by an diagonal matrix D : if and if  Separate and by two bounding planes such that:  More succinctly:, where

9. Support Vector Classification (Linearly Separable Case) Letbe a linearly separable training sample and represented by matrices

10. Support Vector Classification (Linearly Separable Case, Primal) The hyperplanethat solves the minimization problem: realizes the maximal margin hyperplane with geometric margin

11. Support Vector Classification (Linearly Separable Case, Dual Form) The dual problem of previous MP: subject to Applying the KKT optimality conditions, we have. But where is Don ’ t forget

12. Dual Representation of SVM (Key of Kernel Methods: ) The hypothesis is determined by

13. Compute the Geometric Margin via Dual Solution  The geometric marginand, hence we can compute by using.  Use KKT again (in dual)!  Don ’ t forget 