1. S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE.09.454/ECE.09.560 Fall 2010 Shreekanth Mandayam ECE Department Rowan University http://engineering.rowan.edu/~shreek/fall10/ann/ Lecture 5 October 11, 2010

2. S. Mandayam/ ANN/ECE Dept./Rowan UniversityPlan The “Optimal” Classifier Bayes Classifier Multilayer Perceptron Improvements to the backpropagation algorithm Input data processing Selection of training and test data - cross- validation Pre-processing: Feature Extraction Lab Project 2

3. S. Mandayam/ ANN/ECE Dept./Rowan University Recall: Backpropagation Forward Pass Fix w ji (n) Compute y j (n) Backward Pass Calculate  j (n) Update weights w ji (n+1) i j k Left Right i j k Left Right x y

4. S. Mandayam/ ANN/ECE Dept./Rowan University Bayes Classifier for a 2-Class Problem See Dr. Robi Polikar’s Pattern Recognition ECE.09.455/555 http://users.rowan.edu/~polikar/CLASSES/ECE555/index_files/Page316.html

5. S. Mandayam/ ANN/ECE Dept./Rowan UniversityIllustration http://www.statsoft.com/textbook/naive-bayes-classifier/

6. S. Mandayam/ ANN/ECE Dept./Rowan University Bayes Classifier for a 2-Class Problem See Mike Muhlbaier’s MLP demo demos/NNdemo

7. S. Mandayam/ ANN/ECE Dept./Rowan University Backprop: “Improvements” pp. 144-150 Sequential vs. Batch update Maximizing information content Choice of activation function Target values Normalizing inputs Initialization Learning from “hints” Variation in learning rates

8. S. Mandayam/ ANN/ECE Dept./Rowan University Selection of Training and Test Data: Method of Cross-Validation Train Test Train TestTrain TestTrain TestTrain Trial 1 Trial 2 Trial 3 Trial 4 Vary network parameters until total mean squared error is minimum for all trials Find network with the least mean squared output error

9. S. Mandayam/ ANN/ECE Dept./Rowan University Feature Extraction Objective: Increase information content Decrease vector length Parametric invariance Invariance by structure Invariance by training Invariance by transformation

10. S. Mandayam/ ANN/ECE Dept./Rowan University Lab Project 2 http://engineering.rowan.edu/~shreek /fall10/ann/lab2.htmlhttp://engineering.rowan.edu/~shreek /fall10/ann/lab2.html

11. S. Mandayam/ ANN/ECE Dept./Rowan UniversitySummary