Chapter 9 Perceptrons and their generalizations. Rosenblatt ’ s perceptron Proofs of the theorem Method of stochastic approximation and sigmoid approximation.

Slides:

Advertisements

Similar presentations

Artificial Neural Networks

Advertisements

Introduction to Support Vector Machines (SVM)

Support Vector Machine

Support Vector Machines

Pattern Recognition and Machine Learning: Kernel Methods.

Support vector machine

CSCI 347 / CS 4206: Data Mining Module 07: Implementations Topic 03: Linear Models.

Lecture 13 – Perceptrons Machine Learning March 16, 2010.

Separating Hyperplanes

Face Recognition & Biometric Systems Support Vector Machines (part 2)

Classification and Prediction: Regression Via Gradient Descent Optimization Bamshad Mobasher DePaul University.

Methods of Pattern Recognition chapter 5 of: Statistical learning methods by Vapnik Zahra Zojaji.

Lecture 14 – Neural Networks

Support Vector Machines (and Kernel Methods in general)

November 2, 2010Neural Networks Lecture 14: Radial Basis Functions 1 Cascade Correlation Weights to each new hidden node are trained to maximize the covariance.

Sparse Kernels Methods Steve Gunn.

Greg GrudicIntro AI1 Introduction to Artificial Intelligence CSCI 3202: The Perceptron Algorithm Greg Grudic.

2806 Neural Computation Support Vector Machines Lecture Ari Visa.

Lecture 10: Support Vector Machines

Artificial Neural Networks

CHAPTER 11 Back-Propagation Ming-Feng Yeh.

Statistical Learning Theory: Classification Using Support Vector Machines John DiMona Some slides based on Prof Andrew Moore at CMU:

Optimization Theory Primal Optimization Problem subject to: Primal Optimal Value:

CS 4700: Foundations of Artificial Intelligence

Collaborative Filtering Matrix Factorization Approach

Machine Learning CUNY Graduate Center Lecture 3: Linear Regression.

Radial Basis Function Networks

Based on: The Nature of Statistical Learning Theory by V. Vapnick 2009 Presentation by John DiMona and some slides based on lectures given by Professor.

Artificial Neural Networks

Natural Gradient Works Efficiently in Learning S Amari (Fri) Computational Modeling of Intelligence Summarized by Joon Shik Kim.

Machine Learning Lecture 11 Summary G53MLE | Machine Learning | Dr Guoping Qiu1.

Radial Basis Function Networks:

Support Vector Machine With Adaptive Parameters in Financial Time Series Forecasting by L. J. Cao and Francis E. H. Tay IEEE Transactions On Neural Networks,

An Introduction to Support Vector Machine (SVM) Presenter : Ahey Date : 2007/07/20 The slides are based on lecture notes of Prof. 林智仁 and Daniel Yeung.

Machine Learning Using Support Vector Machines (Paper Review) Presented to: Prof. Dr. Mohamed Batouche Prepared By: Asma B. Al-Saleh Amani A. Al-Ajlan.

Kernel Methods A B M Shawkat Ali 1 2 Data Mining ¤ DM or KDD (Knowledge Discovery in Databases) Extracting previously unknown, valid, and actionable.

1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 24 Nov 2, 2005 Nanjing University of Science & Technology.

Ohad Hageby IDC Support Vector Machines & Kernel Machines IP Seminar 2008 IDC Herzliya.

An Introduction to Support Vector Machine (SVM)

Survey of Kernel Methods by Jinsan Yang. (c) 2003 SNU Biointelligence Lab. Introduction Support Vector Machines Formulation of SVM Optimization Theorem.

CSSE463: Image Recognition Day 14 Lab due Weds, 3:25. Lab due Weds, 3:25. My solutions assume that you don't threshold the shapes.ppt image. My solutions.

Back-Propagation Algorithm AN INTRODUCTION TO LEARNING INTERNAL REPRESENTATIONS BY ERROR PROPAGATION Presented by: Kunal Parmar UHID:

1  The Problem: Consider a two class task with ω 1, ω 2   LINEAR CLASSIFIERS.

1  Problem: Consider a two class task with ω 1, ω 2   LINEAR CLASSIFIERS.

Chapter 18 Connectionist Models

Chapter 8: Adaptive Networks

Neural Networks Vladimir Pleskonjić 3188/ /20 Vladimir Pleskonjić General Feedforward neural networks Inputs are numeric features Outputs are in.

Support-Vector Networks C Cortes and V Vapnik (Tue) Computational Models of Intelligence Joon Shik Kim.

Chapter 6 Neural Network.

Chapter 10 The Support Vector Method For Estimating Indicator Functions Intelligent Information Processing Laboratory, Fudan University.

Support Vector Machine: An Introduction. (C) by Yu Hen Hu 2 Linear Hyper-plane Classifier For x in the side of o : w T x + b  0; d = +1; For.

Machine Learning Artificial Neural Networks MPλ ∀ Stergiou Theodoros 1.

1 Support Vector Machines: Maximum Margin Classifiers Machine Learning and Pattern Recognition: September 23, 2010 Piotr Mirowski Based on slides by Sumit.

Machine Learning Supervised Learning Classification and Regression

CSSE463: Image Recognition Day 14

CS 9633 Machine Learning Support Vector Machines

One-layer neural networks Approximation problems

Support Vector Machines and Kernels

Announcements HW4 due today (11:59pm) HW5 out today (due 11/17 11:59pm)

Support Vector Machines Introduction to Data Mining, 2nd Edition by

Collaborative Filtering Matrix Factorization Approach

Online Learning Kernels

CSSE463: Image Recognition Day 14

Neural Networks Chapter 5

CSSE463: Image Recognition Day 14

CSSE463: Image Recognition Day 14

CSSE463: Image Recognition Day 14

Neural networks (1) Traditional multi-layer perceptrons

CSSE463: Image Recognition Day 14

CSSE463: Image Recognition Day 15

Presentation transcript:

Chapter 9 Perceptrons and their generalizations

Rosenblatt ’ s perceptron Proofs of the theorem Method of stochastic approximation and sigmoid approximation of indicator functions Method of potential functions and Radial basis functions Three theorem of optimization theory Neural Networks

Perceptrons (Rosenblatt, 1950s)

Recurrent Procedure

Proofs of the theorems

Method of stochastic approximation and sigmoid approximation of indicator functions

Method of Stochastic Approximation

Sigmoid Approximation of Indicator Functions

Basic Frame for learning process Use the sigmoid approximation at the stage of estimating the coefficients Use the indicator functions at the stage of recognition.

Method of potential functions and Radial Basis Functions

Potential function On-line Only one element of the training data RBFs (mid-1980s) Off-line

Method of potential functions in asymptotic learning theory Separable condition Deterministic setting of the PR Non-separable condition Stochastic setting of the PR problem

Deterministic Setting

Stochastic Setting

RBF Method

Three Theorems of optimization theory Fermat ’ s theorem (1629) Entire space, without constraints Lagrange multipliers rule (1788) Conditional optimization problem Kuhn-Tucker theorem (1951) Convex optimizaiton

To find the stationary points of functions It is necessary to solve a system of n equations with n unknown values.

Lagrange Multiplier Rules (1788)

Kuhn-Tucker Theorem (1951) Convex optimization Minimize a certain type of (convex) objective function under certain (convex) constraints of inequality type.

Remark

Neural Networks A learning machine: Nonlinearly mapped input vector x in feature space U Constructed a linear function into this space.

Neural Networks The Back-Propagation method The BP algorithm Neural Networks for the Regression estimation problem Remarks on the BP method.

The Back-Propagation method

The BP algorithm

For the regression estimation problem

Remark The empirical risk functional has many local minima The convergence of the gradient based method is rather slow. The sigmoid function has a scaling factor that affects the quality of the approximation.

Neural-networks are not well-controlled learning machines In many practical applications, however, demonstrates good results.