CS639: Data Management for Data Science

Slides:

Advertisements

Similar presentations

INTRODUCTION TO Machine Learning 2nd Edition

Advertisements

SOFT LARGE MARGIN CLASSIFIERS David Kauchak CS 451 – Fall 2013.

Support Vector Machines (and Kernel Methods in general)

Linear Learning Machines  Simplest case: the decision function is a hyperplane in input space.  The Perceptron Algorithm: Rosenblatt, 1956  An on-line.

The Perceptron Algorithm (Primal Form) Repeat: until no mistakes made within the for loop return:. What is ?

Linear Learning Machines  Simplest case: the decision function is a hyperplane in input space.  The Perceptron Algorithm: Rosenblatt, 1956  An on-line.

Classification Problem 2-Category Linearly Separable Case A- A+ Malignant Benign.

Unconstrained Optimization Problem

1 Computational Learning Theory and Kernel Methods Tianyi Jiang March 8, 2004.

Lecture outline Support vector machines. Support Vector Machines Find a linear hyperplane (decision boundary) that will separate the data.

Support Vector Machines

Lecture 10: Support Vector Machines

SVM (Support Vector Machines) Base on statistical learning theory choose the kernel before the learning process.

Greg GrudicIntro AI1 Support Vector Machine (SVM) Classification Greg Grudic.

Optimization Theory Primal Optimization Problem subject to: Primal Optimal Value:

Machine Learning Week 4 Lecture 1. Hand In Data Is coming online later today. I keep test set with approx test images That will be your real test.

CS Statistical Machine learning Lecture 18 Yuan (Alan) Qi Purdue CS Oct

Text Classification 2 David Kauchak cs459 Fall 2012 adapted from:

Recognition II Ali Farhadi. We have talked about Nearest Neighbor Naïve Bayes Logistic Regression Boosting.

CS 478 – Tools for Machine Learning and Data Mining SVM.

Linear Models for Classification

Online Learning Rong Jin. Batch Learning Given a collection of training examples D Learning a classification model from D What if training examples are.

Support vector machine LING 572 Fei Xia Week 8: 2/23/2010 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A 1.

CSE4334/5334 DATA MINING CSE4334/5334 Data Mining, Fall 2014 Department of Computer Science and Engineering, University of Texas at Arlington Chengkai.

Text Classification using Support Vector Machine Debapriyo Majumdar Information Retrieval – Spring 2015 Indian Statistical Institute Kolkata.

6.S093 Visual Recognition through Machine Learning Competition Image by kirkh.deviantart.com Joseph Lim and Aditya Khosla Acknowledgment: Many slides from.

CSSE463: Image Recognition Day 14 Lab due Weds. Lab due Weds. These solutions assume that you don't threshold the shapes.ppt image: Shape1: elongation.

SUPPORT VECTOR MACHINES Presented by: Naman Fatehpuria Sumana Venkatesh.

Introduction to Machine Learning Prof. Nir Ailon Lecture 5: Support Vector Machines (SVM)

Support Vector Machines Reading: Textbook, Chapter 5 Ben-Hur and Weston, A User’s Guide to Support Vector Machines (linked from class web page)

Linear machines márc Decison surfaces We focus now on the decision surfaces Linear machines = linear decision surface Non-optimal solution but.

Support vector machines

CS 9633 Machine Learning Support Vector Machines

PREDICT 422: Practical Machine Learning

Support Vector Machine

Gradient descent David Kauchak CS 158 – Fall 2016.

Fun with Hyperplanes: Perceptrons, SVMs, and Friends

Large Margin classifiers

ECE 5424: Introduction to Machine Learning

MIRA, SVM, k-NN Lirong Xia. MIRA, SVM, k-NN Lirong Xia.

Support Vector Machines

CS 4/527: Artificial Intelligence

Pawan Lingras and Cory Butz

Machine Learning Week 1.

Support Vector Machines

Machine Learning Week 2.

Statistical Learning Dong Liu Dept. EEIS, USTC.

CS 2750: Machine Learning Support Vector Machines

Online Learning Kernels

Large Scale Support Vector Machines

CSSE463: Image Recognition Day 14

Support Vector Machine

CSSE463: Image Recognition Day 14

Support vector machines

Machine Learning Week 3.

CSSE463: Image Recognition Day 14

Lecture 18. SVM (II): Non-separable Cases

CSSE463: Image Recognition Day 14

Support Vector Machine I

Support vector machines

Based on slides by William Cohen, Andrej Karpathy, Piyush Rai

CSSE463: Image Recognition Day 14

Artificial Intelligence 9. Perceptron

CS639: Data Management for Data Science

Based on slides by William Cohen, Andrej Karpathy, Piyush Rai

Based on slides by William Cohen, Andrej Karpathy, Piyush Rai

Based on slides by William Cohen, Andrej Karpathy, Piyush Rai

Based on slides by William Cohen, Andrej Karpathy, Piyush Rai

SVMs for Document Ranking

MIRA, SVM, k-NN Lirong Xia. MIRA, SVM, k-NN Lirong Xia.

Presentation transcript:

CS639: Data Management for Data Science Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by Ankur Goswami many slides from David Sontag)

Today’s Lecture Linear classifiers The Perceptron Algorithm Support Vector Machines

Linear classifier Let’s simplify life by assuming: Every instance is a vector of real numbers, x=(x1,…,xn). (Notation: boldface x is a vector.) There are only two classes, y=(+1) and y=(-1) A linear classifier is vector w of the same dimension as x that is used to make this prediction:

Example: Linear classifier

The Perceptron Algorithm to learn a Linear Classifier

Definition: Linearly separable data

Does the perceptron algorithm work?

Properties of the perceptron algorithm

Problems with the perceptron algorithm [We will see next week]

Linear separators

Support Vector Machines

Normal to a plane

Scale invariance

Scale invariance

What is γ as a function of w?

Support Vector Machines (SVMs)

What if the data is not separable?

Allowing for slack: “Soft margin” SVM

Allowing for slack: “Soft margin” SVM

Equivalent Hinge Loss Formulation

Hinge Loss vs 0-1 Loss

Multiclass SVM

One versus all classification

Multiclass SVM

Multiclass SVM

What you need to know