1. 1-norm Support Vector Machines Good for Feature Selection  Solve the quadratic program for some : min s. t.,, denotes where or membership. Equivalent to solve a Linear Program as follows:

2. -Support Vector Regression (Linear Case:)  Given the training set:  Motivated by SVM:  should be as small as possible  Some tiny error should be discarded  Represented by an matrix and a vector  Try to find such that that is where

3. -Insensitive Loss Function  -insensitive loss function:  The loss made by the estimation function, at the data point is  If then is defined as: (Tiny Error Should Be Discarded)

4. -Insensitive Linear Regression Find with the smallest overall error

5. Five Popular Loss Functions

6. -Insensitive Loss Regression  Linear -insensitive loss function: where is a real function  Quadratic -insensitive loss function:

7. - insensitive Support Vector Regression Model Motivated by SVM:  should be as small as possible  Some tiny error should be discarded where

8. Why minimize ? probably approximately correct (pac) Consider performing linear regression for any training data distribution and then  Occam’s razor : the simplest is the best

9. Reformulated - SVR as a Constrained Minimization Problem subject to n+1+2m variables and 2m constrains minimization problem Enlarge the problem size and computational complexity for solving the problem

10. SV Regression by Minimizing Quadratic -Insensitive Loss  We have the following problem: where

11. Primal Formulation of SVR for Quadratic -Insensitive Loss  Extremely important: At the solution subject to

12. Dual Formulation of SVR for Quadratic -Insensitive Loss subject to

13. KKT Complementarity Conditions  KKT conditions are :  Don ’ t forget we have:

14. Simplify Dual Formulation of SVR subject to  The case, problem becomes to the least squares linear regression with a weight decay factor

15. Kernel in Dual Formulation for SVR  Then the regression function is defined by  Supposesolves the QP problem: where is chosen such that with subject to