Robust Optimization and Applications in Machine Learning

Time-series prediction via linear least-squares

Predicted output

Properties of solution

Non-linear prediction and kernels

Properties of solution

What is a kernel, anyway? SVM, LR, LS, MPM, PCA, CCA, FDA…

Example: 2 nd -order polynomial kernel

A classical way to use kernels

Transduction framework

Important property of kernel matrices

Kernel optimization in least-squares

Kernel optimization for least-squares

Kernel optimization via SDP or SOCP

A non-classical way to use kernels

Kernel optimization in other problems

Kernel optimization in SVM classifiers

Kernel optimization in SVM classifiers (cont’d)

Link with robust optimization

Kernel optimization and data fusion mRNA expression data upstream region data (TF binding sites) protein-protein interaction data hydrophobicity data sequence data (gene, protein)

Challenge

Example of a Kernel for Genomic Data: Pairwise Comparison Kernel

protein 2 Example of a Kernel for Genomic Data: Linear Interaction Kernel

Exampe of a Kernel for Genomic Data: Diffusion Kernel

Learning the Optimal Kernel K

Integrate constructed kernels Learn a linear mix Large margin classifier (SVM) Maximize the margin

Yeast Protein Function Prediction

MRF SDP/SVM (binary) SDP/SVM (enriched) Yeast Protein Function Prediction

MRF SDP/SVM (binary) SDP/SVM (enriched) Yeast Protein Function Prediction

Part 3: summary