Dual Problem of Linear Program subject to Primal LP Dual LP subject to ※ All duality theorems hold and work perfectly!

PrimalDual Nonnegative variableInequality constraint Free variableEquality constraint Inequality constraint Nonnegative variable Equality constraintFree variable

Primal s. t. Dual s. t.

Primal Problem Feasible Region

Dual Problem of Strictly Convex Quadratic Program subject to Primal QP With strictly convex assumption, we have Dual QP subject to

Classification Problem 2-Category Linearly Separable Case A- A+ Malignant Benign

Support Vector Machines Maximizing the Margin between Bounding Planes A+ A-

Algebra of the Classification Problem 2-Category Linearly Separable Case  Given m points in the n dimensional real space  Represented by an matrix or  Membership of each point in the classes is specified by an diagonal matrix D : if and if  Separate and by two bounding planes such that:  More succinctly:, where

Support Vector Classification (Linearly Separable Case) Letbe a linearly separable training sample and represented by matrices

Support Vector Classification (Linearly Separable Case, Primal) The hyperplanethat solves the minimization problem: realizes the maximal margin hyperplane with geometric margin

Support Vector Classification (Linearly Separable Case, Dual Form) The dual problem of previous MP: subject to Applying the KKT optimality conditions, we have. But where is Don ’ t forget

Dual Representation of SVM (Key of Kernel Methods: ) The hypothesis is determined by

Compute the Geometric Margin via Dual Solution  The geometric marginand, hence we can compute by using.  Use KKT again (in dual)!  Don ’ t forget 