A Casual Chat on Convex Optimization in Machine Learning Data Mining at Iowa Group Qihang Lin 02/09/2014.

Slides:



Advertisements
Similar presentations
Scaling Multivariate Statistics to Massive Data Algorithmic problems and approaches Alexander Gray Georgia Institute of Technology
Advertisements

A brief review of non-neural-network approaches to deep learning
Accelerated, Parallel and PROXimal coordinate descent IPAM February 2014 APPROX Peter Richtárik (Joint work with Olivier Fercoq - arXiv: )
Regularization David Kauchak CS 451 – Fall 2013.
INTRODUCTION TO Machine Learning 2nd Edition
ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.
Semi-Stochastic Gradient Descent Methods Jakub Konečný (joint work with Peter Richtárik) University of Edinburgh.
Theano Md. Iftekhar Tanveer Rafayet Ali Md. Iftekhar Tanveer Rafayet Ali.
Semi-Stochastic Gradient Descent Methods Jakub Konečný University of Edinburgh BASP Frontiers Workshop January 28, 2014.
Optimization Tutorial
Continuous optimization Problems and successes
Lecture 13 – Perceptrons Machine Learning March 16, 2010.
Middle Term Exam 03/01 (Thursday), take home, turn in at noon time of 03/02 (Friday)
Lecture 4: Embedded methods
RECITATION 1 APRIL 9 Polynomial regression Ridge regression Lasso.
Coefficient Path Algorithms Karl Sjöstrand Informatics and Mathematical Modelling, DTU.
Lasso, Support Vector Machines, Generalized linear models Kenneth D. Harris 20/5/15.
1 Econometrics 1 Lecture 1 Classical Linear Regression Analysis.
Linear Programming Special Cases Alternate Optimal Solutions No Feasible Solution Unbounded Solutions.
Linear Regression  Using a linear function to interpolate the training set  The most popular criterion: Least squares approach  Given the training set:
Proximal Support Vector Machine Classifiers KDD 2001 San Francisco August 26-29, 2001 Glenn Fung & Olvi Mangasarian Data Mining Institute University of.
The Widrow-Hoff Algorithm (Primal Form) Repeat: Until convergence criterion satisfied return: Given a training set and learning rate Initial:  Minimize.
The Perceptron Algorithm (Dual Form) Given a linearly separable training setand Repeat: until no mistakes made within the for loop return:
Unconstrained Optimization Problem
Lecture 10: Support Vector Machines
Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, a Machine Learning.
Semi-Stochastic Gradient Descent Methods Jakub Konečný (joint work with Peter Richtárik) University of Edinburgh SIAM Annual Meeting, Chicago July 7, 2014.
Blitz: A Principled Meta-Algorithm for Scaling Sparse Optimization Tyler B. Johnson and Carlos Guestrin University of Washington.
An Introduction to Optimization Theory. Outline Introduction Unconstrained optimization problem Constrained optimization problem.
Optimization Theory Primal Optimization Problem subject to: Primal Optimal Value:
Peter Richtárik (joint work with Martin Takáč) Distributed Coordinate Descent Method AmpLab All Hands Meeting - Berkeley - October 29, 2013.
Machine Learning Seminar: Support Vector Regression Presented by: Heng Ji 10/08/03.
Mathematical formulation XIAO LIYING. Mathematical formulation.
BrainStorming 樊艳波 Outline Several papers on icml15 & cvpr15 PALM Information Theory Learning.
Sparse Inverse Covariance Estimation with Graphical LASSO J. Friedman, T. Hastie, R. Tibshirani Biostatistics, 2008 Presented by Minhua Chen 1.
Efficient and Numerically Stable Sparse Learning Sihong Xie 1, Wei Fan 2, Olivier Verscheure 2, and Jiangtao Ren 3 1 University of Illinois at Chicago,
CS Statistical Machine learning Lecture 18 Yuan (Alan) Qi Purdue CS Oct
The Group Lasso for Logistic Regression Lukas Meier, Sara van de Geer and Peter Bühlmann Presenter: Lu Ren ECE Dept., Duke University Sept. 19, 2008.
1  Problem: Consider a two class task with ω 1, ω 2   LINEAR CLASSIFIERS.
Recitation4 for BigData Jay Gu Feb LASSO and Coordinate Descent.
CS Statistical Machine learning Lecture 12 Yuan (Alan) Qi Purdue CS Oct
Logistic Regression & Elastic Net
Support-Vector Networks C Cortes and V Vapnik (Tue) Computational Models of Intelligence Joon Shik Kim.
Robust Optimization and Applications in Machine Learning.
Experimental Perspectives on Lasso-related Algorithms on Parallel Computing Frameworks
Linear Programming Chapter 1 Introduction.
COMPUTER GRAPHICS AND LINEAR ALGEBRA AN INTRODUCTION.
Optimal Control.
StingyCD: Safely Avoiding Wasteful Updates in Coordinate Descent
Machine Learning & Deep Learning
Jeremy Watt and Aggelos Katsaggelos Northwestern University
Zhu Han University of Houston Thanks for Dr. Mingyi Hong’s slides
Boosting and Additive Trees (2)
Project #5: Generating Privacy and Security Threat Summary for Internet of Things REU Student: Ray Yan Graduate mentors: Logan Lebanoff Faculty Mentor(s):
Chapter 1. Introduction Mathematical Programming (Optimization) Problem: min/max
Probabilistic Models for Linear Regression
Support Vector Machines Introduction to Data Mining, 2nd Edition by
CSCI B609: “Foundations of Data Science”
Logistic Regression & Parallel SGD
What is Regression Analysis?
Multi-layer perceptron
CRISP: Consensus Regularized Selection based Prediction
Support Vector Machine I
How do we find the best linear regression line?
Least squares linear classifier
The Math of Machine Learning
Part 3. Linear Programming
ADMM and DSO.
Linear regression with one variable
Machine Learning in Business John C. Hull
Presentation transcript:

A Casual Chat on Convex Optimization in Machine Learning Data Mining at Iowa Group Qihang Lin 02/09/2014

Optimization Models in Machine Learning LASSO Graphical LASSO Matrix Completion Robust PCA SVM ……

LASSO Linear Regression Ridge Regression L1-Regularized Regression Elastic Net Regression Fused LASSO Group LASSO Multi-Task LASSO

Graphical LASSO

Matrix Completion

Robust PCA

SVM

Convex Optimization Algorithms Proximal Gradient Frank-Wolfe (Conditional Gradient) Alternating Direction Method of Multipliers Coordinate Descent