1. Artificial Neural Networks ECE.09.454/ECE.09.560 Fall 2006
Lecture 2 September 25, 2006
Shreekanth Mandayam
ECE Department
Rowan University

2. Plan Recall: Neural Network Paradigm Recall: Perceptron Model
Learning Processes
Rules
Paradigms
Tasks
Perceptron Training Algorithm
Widrow-Hoff Rule (LMS Algorithm)
Lab Project 1

3. Recall: Neural Network Paradigm
Stage 1: Network Training
Artificial
Neural
Network
Indicate Desired Outputs
Present Examples
Determine
Synaptic
Weights
“knowledge”
Stage 2: Network Testing
Artificial
Neural
Network
Predicted Outputs
New Data

4. Recall: ANN Model x Input Vector y Output Vector f Complex Nonlinear
Artificial
Neural
Network y
Output
Vector f
Complex
Nonlinear
Function
“knowledge”
f(x) = y

5. Recall: The Perceptron ModelS
j(.)
wk1
wk2
wkm x1 x2 xm
Inputs
Synaptic
weights
Bias, bk
Induced field, vk
Output, yk uk
Activation/ squashing function

6. “Learning” Mathematical Model of the Learning Process ANN [w]0 x y(0)
Intitialize: Iteration (0)
ANN
[w]0 x
y(0)
[w] x y
Iteration (1)
[w]1 x
y(1)
desired
o/p
Iteration (n)
[w]n x
y(n) = d

7. Learning Rules Error Correction Learning Memory Based Learning
Delta Rule or Widrow-Hoff Rule
Memory Based Learning
Nearest Neighbor Rule
Hebbian Learning
Competitive Learning
Boltzman Learning

8. Error-Correction Learning
wk1(n)
Desired
Output,
dk (n)
Activation/ squashing function
x1 (n)
Bias, bk
wk2(n) x2 + S
j(.)
Output,
yk (n) S
Inputs
Synaptic
weights -
Induced field,
vk(n)
wkm(n)
Error
Signal
ek (n) xm

9. Learning Paradigms Supervised Unsupervised ANN Environment Teacher
(Data)
Teacher
(Expert) S
ANN
error
desired
actual + -

10. Learning Paradigms Supervised Unsupervised ANN Delay Environment Cost
(Data)
Delay
ANN
Delayed
Reinforcement
Learning
Cost
Function

11. Learning Tasks Classification
Pattern Association
Pattern Recognition
Function Approximation
Filtering
Classification x1 x2 1 2 DB x1 x2 1 2 DB

12. Perceptron Training Widrow-Hoff Rule (LMS Algorithm)
y(n) = sgn [wT(n) x(n)]
w(n+1) = w(n) + h[d(n) – y(n)]x(n)
n = n+1
Matlab Demo

13. Lab Project 1
UCI Machine Learning Repository:
Face Recognition: Generate images

14. Summary