1 Introduction to Support Vector Machines for Data Mining Mahdi Nasereddin Ph.D. Pennsylvania State University School of Information Sciences and Technology.

Slides:

Advertisements

Similar presentations

Introduction to Support Vector Machines (SVM)

Advertisements

Generative Models Thus far we have essentially considered techniques that perform classification indirectly by modeling the training data, optimizing.

Rerun of machine learning Clustering and pattern recognition.

1 Image Classification MSc Image Processing Assignment March 2003.

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

SVM - Support Vector Machines A new classification method for both linear and nonlinear data It uses a nonlinear mapping to transform the original training.

Classification / Regression Support Vector Machines

Support Vector Machine Classification Computation & Informatics in Biology & Medicine Madison Retreat, November 15, 2002 Olvi L. Mangasarian with G. M.

An Introduction of Support Vector Machine

Support Vector Machines

SVM—Support Vector Machines

CSCI 347 / CS 4206: Data Mining Module 07: Implementations Topic 03: Linear Models.

The Disputed Federalist Papers : SVM Feature Selection via Concave Minimization Glenn Fung and Olvi L. Mangasarian CSNA 2002 June 13-16, 2002 Madison,

The Nature of Statistical Learning Theory by V. Vapnik

COMP 328: Midterm Review Spring 2010 Nevin L. Zhang Department of Computer Science & Engineering The Hong Kong University of Science & Technology

Support Vector Machines Kernel Machines

Sparse Kernels Methods Steve Gunn.

CS 4700: Foundations of Artificial Intelligence

A Study of the Relationship between SVM and Gabriel Graph ZHANG Wan and Irwin King, Multimedia Information Processing Laboratory, Department of Computer.

Support Vector Machines

Statistical Learning: Pattern Classification, Prediction, and Control Peter Bartlett August 2002, UC Berkeley CIS.

Statistical Learning Theory: Classification Using Support Vector Machines John DiMona Some slides based on Prof Andrew Moore at CMU:

Optimization Theory Primal Optimization Problem subject to: Primal Optimal Value:

Neural Networks in Data Mining “An Overview”

Machine Learning Usman Roshan Dept. of Computer Science NJIT.

An Introduction to Support Vector Machines Martin Law.

Classification III Tamara Berg CS Artificial Intelligence Many slides throughout the course adapted from Svetlana Lazebnik, Dan Klein, Stuart Russell,

Ch. Eick: Support Vector Machines: The Main Ideas Reading Material Support Vector Machines: 1.Textbook 2. First 3 columns of Smola/Schönkopf article on.

July 11, 2001Daniel Whiteson Support Vector Machines: Get more Higgs out of your data Daniel Whiteson UC Berkeley.

Machine Learning Queens College Lecture 13: SVM Again.

Machine Learning1 Machine Learning: Summary Greg Grudic CSCI-4830.

© N. Kasabov Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering, MIT Press, 1996 INFO331 Machine learning. Neural networks. Supervised.

Support Vector Machines in Data Mining AFOSR Software & Systems Annual Meeting Syracuse, NY June 3-7, 2002 Olvi L. Mangasarian Data Mining Institute University.

1 SUPPORT VECTOR MACHINES İsmail GÜNEŞ. 2 What is SVM? A new generation learning system. A new generation learning system. Based on recent advances in.

An Introduction to Support Vector Machine (SVM) Presenter : Ahey Date : 2007/07/20 The slides are based on lecture notes of Prof. 林智仁 and Daniel Yeung.

Machine Learning Using Support Vector Machines (Paper Review) Presented to: Prof. Dr. Mohamed Batouche Prepared By: Asma B. Al-Saleh Amani A. Al-Ajlan.

Support vector machines for classification Radek Zíka

An Introduction to Support Vector Machines (M. Law)

1 Chapter 6. Classification and Prediction Overview Classification algorithms and methods Decision tree induction Bayesian classification Lazy learning.

Nonlinear Data Discrimination via Generalized Support Vector Machines David R. Musicant and Olvi L. Mangasarian University of Wisconsin - Madison

Linear Discrimination Reading: Chapter 2 of textbook.

Handwritten digit recognition

Support Vector Machine Data Mining Olvi L. Mangasarian with Glenn M. Fung, Jude W. Shavlik & Collaborators at ExonHit – Paris Data Mining Institute University.

Support Vector Machines Project מגישים : גיל טל ואורן אגם מנחה : מיקי אלעד נובמבר 1999 הטכניון מכון טכנולוגי לישראל הפקולטה להנדסת חשמל המעבדה לעיבוד וניתוח.

Ohad Hageby IDC Support Vector Machines & Kernel Machines IP Seminar 2008 IDC Herzliya.

An Introduction to Support Vector Machine (SVM)

Support Vector Machines in Marketing Georgi Nalbantov MICC, Maastricht University.

SVM – Support Vector Machines Presented By: Bella Specktor.

CSSE463: Image Recognition Day 14 Lab due Weds, 3:25. Lab due Weds, 3:25. My solutions assume that you don't threshold the shapes.ppt image. My solutions.

A TUTORIAL ON SUPPORT VECTOR MACHINES FOR PATTERN RECOGNITION ASLI TAŞÇI Christopher J.C. Burges, Data Mining and Knowledge Discovery 2, , 1998.

CSSE463: Image Recognition Day 15 Announcements: Announcements: Lab 5 posted, due Weds, Jan 13. Lab 5 posted, due Weds, Jan 13. Sunset detector posted,

Chapter 6. Classification and Prediction Classification by decision tree induction Bayesian classification Rule-based classification Classification by.

Chapter 6. Classification and Prediction Classification by decision tree induction Bayesian classification Rule-based classification Classification by.

Supervised Machine Learning: Classification Techniques Chaleece Sandberg Chris Bradley Kyle Walsh.

Feature Selction for SVMs J. Weston et al., NIPS 2000 오장민 (2000/01/04) Second reference : Mark A. Holl, Correlation-based Feature Selection for Machine.

Support-Vector Networks C Cortes and V Vapnik (Tue) Computational Models of Intelligence Joon Shik Kim.

Machine Learning and Data Mining: A Math Programming- Based Approach Glenn Fung CS412 April 10, 2003 Madison, Wisconsin.

FUZZ-IEEE Kernel Machines and Additive Fuzzy Systems: Classification and Function Approximation Yixin Chen and James Z. Wang The Pennsylvania State.

SUPPORT VECTOR MACHINES Presented by: Naman Fatehpuria Sumana Venkatesh.

CS 9633 Machine Learning Support Vector Machines

Introduction to Pattern Recognition

An Introduction to Support Vector Machines

An Introduction to Support Vector Machines

Machine Learning Week 1.

Support Vector Machines

Other Classification Models: Support Vector Machine (SVM)

Computational Intelligence: Methods and Applications

CSE 802. Prepared by Martin Law

Presentation transcript:

1 Introduction to Support Vector Machines for Data Mining Mahdi Nasereddin Ph.D. Pennsylvania State University School of Information Sciences and Technology

2 Agenda Introduction Support Vector Machines Preliminary Experimentation Conclusion Questions?

3 Data Mining Techniques: Neural Networks Decision Trees Multivariate Adaptive Regression Splines (MARS) Rule Induction Nearest Neighbor Method and discriminant analysis Genetic Algorithms Support Vector Machines

4 First introduced by Vapnik and Chervonenkis in COLT-92 Bases on Statistical Learning Theory Applications Basic Theory Classification Regression

5 Successful Applications of SVMS Protein Structure Prediction e.pdf e.pdf Intrusion Detection Handwriting Recognition Detecting Steganography in digital images ns/ih02.html ns/ih02.html

6 Successful Applications of SVMS Breast Cancer Prognosis: Chemotherapy Effect on Survival Rate (Lee, Mangasarian and Wolberg, 2001) Particle and Quark-Flavour Identification in High Energy Physics ( freiburg.de/preprints/EHEP9901.ps) freiburg.de/preprints/EHEP9901.ps Function Approximation

7 Support Vector Machines (Linearly separable case)

8

9

10 Non-Linearly separable case

11 SVM for Regression In case of regression, the goal is to construct a hyperplane that is close to as many points as possible. For both classification and regression, learning is done via quadratic programming (one optimum point)

12 Strengths and Weaknesses of SVM Strengths Training is relatively easy No local optimal, unlike in neural networks It scales relatively well to high dimensional data Weaknesses Need a “good” kernel function

13 Preliminary Experimentation: Forecasting GDP using Oil Prices (with F. Malik) Forecasting model Objective: To predict the Gross Domestic Product (GDP) for the next quarter using Oil prices (including time lag) GDP time

14 Data Set We looked at quarterly Oil prices and GDP data January 1947 – December 2002 Oil price data were obtained from Bureau of Labor Statistics GDP data were obtained from the Bureau of Economic Analysis. We used the growth rate of GDP and the growth rate of oil prices.

15 Models Neural Networks Back-propagation One hidden layer Delta rule was used for training LS-SVM (Van Gestel, 2001) Matlab toolbox

16 Experimentation Created the training data to predict the last 40 quarters GDP (test data) Trained the neural network and the SVM Used the model to predict GDP, and calculated the error of prediction

17 Results ModelMAE Neural Network LS-SVM0.0052

18 Good References Introductions Martin Law, “An Introduction to Support Vector Machines” Andrew More, “Support Vector Machines” N. Cristianini In depth Support Vector Machines book

19 Questions Presentation will be posted (by Friday) at