Artificial Neural Networks ECE.09.454/ECE.09.560 Fall 2006 Lecture 2 September 25, 2006 Shreekanth Mandayam ECE Department Rowan University http://engineering.rowan.edu/~shreek/fall06/ann/

Plan Recall: Neural Network Paradigm Recall: Perceptron Model Learning Processes Rules Paradigms Tasks Perceptron Training Algorithm Widrow-Hoff Rule (LMS Algorithm) Lab Project 1

Recall: Neural Network Paradigm Stage 1: Network Training Artificial Neural Network Indicate Desired Outputs Present Examples Determine Synaptic Weights “knowledge” Stage 2: Network Testing Artificial Neural Network Predicted Outputs New Data

Recall: ANN Model x Input Vector y Output Vector f Complex Nonlinear Artificial Neural Network y Output Vector f Complex Nonlinear Function “knowledge” f(x) = y

Recall: The Perceptron Model S j(.) wk1 wk2 wkm x1 x2 xm Inputs Synaptic weights Bias, bk Induced field, vk Output, yk uk Activation/ squashing function

“Learning” Mathematical Model of the Learning Process ANN [w]0 x y(0) Intitialize: Iteration (0) ANN [w]0 x y(0) [w] x y Iteration (1) [w]1 x y(1) desired o/p Iteration (n) [w]n x y(n) = d

Learning Rules Error Correction Learning Memory Based Learning Delta Rule or Widrow-Hoff Rule Memory Based Learning Nearest Neighbor Rule Hebbian Learning Competitive Learning Boltzman Learning

Error-Correction Learning wk1(n) Desired Output, dk (n) Activation/ squashing function x1 (n) Bias, bk wk2(n) x2 + S j(.) Output, yk (n) S Inputs Synaptic weights - Induced field, vk(n) wkm(n) Error Signal ek (n) xm

Learning Paradigms Supervised Unsupervised ANN Environment Teacher (Data) Teacher (Expert) S ANN error desired actual + -

Learning Paradigms Supervised Unsupervised ANN Delay Environment Cost (Data) Delay ANN Delayed Reinforcement Learning Cost Function

Learning Tasks Classification Pattern Association Pattern Recognition Function Approximation Filtering Classification x1 x2 1 2 DB x1 x2 1 2 DB

Perceptron Training Widrow-Hoff Rule (LMS Algorithm) y(n) = sgn [wT(n) x(n)] w(n+1) = w(n) + h[d(n) – y(n)]x(n) n = n+1 Matlab Demo

Lab Project 1 http://engineering.rowan.edu/~shreek/fall06/ann/lab1.html UCI Machine Learning Repository: http://www.ics.uci.edu/~mlearn/MLRepository.html Face Recognition: Generate images

Summary