Robust Optimization and Applications in Machine Learning

Part 2: Robust Classification

Data matrix

Classification problems

What is a linear classifier?

Separable data

Non-separable data

Loss functions

Two specific loss functions

Generalization error and regularization

Regularization and Sparsity

Robust classification

Formulation of robustness approach

Non-separable case

Link with worst-case loss minimization

Box uncertainty model

Formulation

Link with worst-case loss minimization

Our findings so far

Part 2: Robust Classification

Classification with interval data

Robust classification: main idea

Main results

Part 2: Robust Classification

Robust classification with hinge loss

Bound on robust SVM

Part 2: Robust Classification

Robust LR classification

Robust LR: dual

Moment matching

Part 2: Robust Classification

Minimax probability machine

Problem statement

Problem formulation

Marhsall and Olkin’s result ? ?

SOCP formulation

Dual problem

Geometric interpretation

Solving the problem

Robustness to estimation errors

Robust MPM

Formulation of Robust MPM Lemma

R-MPM: A Specific Uncertainty Model (1)

R-MPM: A Specific Uncertainty Model (2)

Robust MPM: Estimation Errors in Means

Rost MPM: Estimation Errors in Covariance

R-MPM: putting everything together

Part 2: summary