S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE.09.454/ECE.09.560 Fall 2010 Shreekanth Mandayam ECE Department Rowan University.

Slides:

Advertisements

Similar presentations

NEURAL NETWORKS Backpropagation Algorithm

Advertisements

Lecture 14 – Neural Networks

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks / Fall 2004 Shreekanth Mandayam ECE Department Rowan University.

Supervised learning 1.Early learning algorithms 2.First order gradient methods 3.Second order gradient methods.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks / Spring 2002 Shreekanth Mandayam Robi Polikar ECE Department.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2008 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2010 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ DIP/ECE Dept./Rowan University Digital Image Processing / Fall 2001 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2008 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ DIP/ECE Dept./Rowan University Digital Image Processing / Fall 2003 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ DIP/ECE Dept./Rowan University Digital Image Processing ECE /ECE Fall 2007 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2006 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2006 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ DIP/ECE Dept./Rowan University Digital Image Processing ECE /ECE Fall 2007 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ DIP/ECE Dept./Rowan University Digital Image Processing ECE /ECE Fall 2007 Shreekanth Mandayam ECE Department Rowan University.

Digital Image Processing / Fall 2003

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks / Spring 2002 Shreekanth Mandayam Robi Polikar ECE Department.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks / Fall 2004 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ EEMAG-1/ECE Dept./Rowan University Engineering Electromagnetics Fall 2004 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks / Fall 2004 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ DIP/ECE Dept./Rowan University Digital Image Processing ECE /ECE Fall 2007 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2006 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ EEMAG-1/ECE Dept./Rowan University Engineering Electromagnetics Fall 2004 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2010 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Shreekanth Mandayam ECE Department Rowan University Lecture.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2006 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ DIP/ECE Dept./Rowan University Digital Image Processing ECE /ECE Fall 2009 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks / Fall 2004 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2010 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks / Spring 2002 Shreekanth Mandayam Robi Polikar ECE Department.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2008 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ DIP/ECE Dept./Rowan University Digital Image Processing / Fall 2001 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ EEMAG-1/ECE Dept./Rowan University Engineering Electromagnetics Fall 2004 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ EEMAG-1/ECE Dept./Rowan University Engineering Electromagnetics Fall 2004 Shreekanth Mandayam ECE Department Rowan University.

LOGO Classification III Lecturer: Dr. Bo Yuan

S. Mandayam/ DIP/ECE Dept./Rowan University Digital Image Processing ECE /ECE Fall 2009 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2006 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2010 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ DIP/ECE Dept./Rowan University Digital Image Processing / Fall 2001 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ DIP/ECE Dept./Rowan University Digital Image Processing / Fall 2001 Shreekanth Mandayam ECE Department Rowan University.

S. Mandayam/ ANN/ECE Dept./Rowan University Smart Sensors / Spring 2004 Shreekanth Mandayam ECE Department Rowan University Artificial.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2010 Shreekanth Mandayam ECE Department Rowan University.

CS 484 – Artificial Intelligence

Neural Networks. Background - Neural Networks can be : Biological - Biological models Artificial - Artificial models - Desire to produce artificial systems.

Artificial Neural Networks

Appendix B: An Example of Back-propagation algorithm

Lecture 3 Introduction to Neural Networks and Fuzzy Logic President UniversityErwin SitompulNNFL 3/1 Dr.-Ing. Erwin Sitompul President University

Multi-Layer Perceptron

Non-Bayes classifiers. Linear discriminants, neural networks.

11 1 Backpropagation Multilayer Perceptron R – S 1 – S 2 – S 3 Network.

CSSE463: Image Recognition Day 14 Lab due Weds, 3:25. Lab due Weds, 3:25. My solutions assume that you don't threshold the shapes.ppt image. My solutions.

Back-Propagation Algorithm AN INTRODUCTION TO LEARNING INTERNAL REPRESENTATIONS BY ERROR PROPAGATION Presented by: Kunal Parmar UHID:

EEE502 Pattern Recognition

Neural Networks 2nd Edition Simon Haykin

Previous Lecture Perceptron W  t+1  W  t  t  d(t) - sign (w(t)  x)] x Adaline W  t+1  W  t  t  d(t) - f(w(t)  x)] f’ x Gradient.

S. Mandayam/ DIP/ECE Dept./Rowan University Digital Image Processing / Fall 2001 Shreekanth Mandayam ECE Department Rowan University.

Pattern Recognition Lecture 20: Neural Networks 3 Dr. Richard Spillman Pacific Lutheran University.

CSSE463: Image Recognition Day 14

Derivation of a Learning Rule for Perceptrons

CSSE463: Image Recognition Day 17

Backpropagation.

CSSE463: Image Recognition Day 17

CSSE463: Image Recognition Day 18

Lecture 04: Multilayer Perceptron

CSSE463: Image Recognition Day 17

Backpropagation.

CS621: Artificial Intelligence Lecture 22-23: Sigmoid neuron, Backpropagation (Lecture 20 and 21 taken by Anup on Graphical Models) Pushpak Bhattacharyya.

Artificial Neural Networks ECE /ECE Fall 2006

Artificial Neural Networks / Spring 2002

Presentation transcript:

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2010 Shreekanth Mandayam ECE Department Rowan University Lecture 5 October 11, 2010

S. Mandayam/ ANN/ECE Dept./Rowan UniversityPlan The “Optimal” Classifier Bayes Classifier Multilayer Perceptron Improvements to the backpropagation algorithm Input data processing Selection of training and test data - cross- validation Pre-processing: Feature Extraction Lab Project 2

S. Mandayam/ ANN/ECE Dept./Rowan University Recall: Backpropagation Forward Pass Fix w ji (n) Compute y j (n) Backward Pass Calculate  j (n) Update weights w ji (n+1) i j k Left Right i j k Left Right x y

S. Mandayam/ ANN/ECE Dept./Rowan University Bayes Classifier for a 2-Class Problem See Dr. Robi Polikar’s Pattern Recognition ECE /555

S. Mandayam/ ANN/ECE Dept./Rowan UniversityIllustration

S. Mandayam/ ANN/ECE Dept./Rowan University Bayes Classifier for a 2-Class Problem See Mike Muhlbaier’s MLP demo demos/NNdemo

S. Mandayam/ ANN/ECE Dept./Rowan University Backprop: “Improvements” pp Sequential vs. Batch update Maximizing information content Choice of activation function Target values Normalizing inputs Initialization Learning from “hints” Variation in learning rates

S. Mandayam/ ANN/ECE Dept./Rowan University Selection of Training and Test Data: Method of Cross-Validation Train Test Train TestTrain TestTrain TestTrain Trial 1 Trial 2 Trial 3 Trial 4 Vary network parameters until total mean squared error is minimum for all trials Find network with the least mean squared output error

S. Mandayam/ ANN/ECE Dept./Rowan University Feature Extraction Objective: Increase information content Decrease vector length Parametric invariance Invariance by structure Invariance by training Invariance by transformation

S. Mandayam/ ANN/ECE Dept./Rowan University Lab Project 2 /fall10/ann/lab2.htmlhttp://engineering.rowan.edu/~shreek /fall10/ann/lab2.html

S. Mandayam/ ANN/ECE Dept./Rowan UniversitySummary