1. S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks 0909.560.01/0909.454.01 Spring 2002 Shreekanth Mandayam Robi Polikar ECE Department Rowan University http://engineering.rowan.edu/~shreek/spring02/ann/ Lecture 3 February 7, 2002

2. S. Mandayam/ ANN/ECE Dept./Rowan UniversityPlan Multilayer Perceptron Architecture Signal Flow Learning rule - Backpropagation Matlab MLP Demo Continue Lab Project 1

3. S. Mandayam/ ANN/ECE Dept./Rowan University Multilayer Perceptron (MLP): Architecture 1 1 1         x1x1 x2x2 x3x3 y1y1 y2y2   w ij w jk w kl Input Layer Hidden Layers Output Layer Inputs Outputs

4. S. Mandayam/ ANN/ECE Dept./Rowan University MLP: Characteristics Neurons possess sigmoidal (logistic) activation functions Contains one or more “hidden layers” Trained using the “backpropagation” algorithm MLP with 1-hidden layer is a “universal approximator” 1 0 0.5 1  (t) t

5. S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Network Massively parallel distributed processor made up of simple processing units, which can store and retrieve experiential knowledge The network “learns” from the data presented to it The “knowledge” is stored in the interconnection weights Adapted from Haykin

6. S. Mandayam/ ANN/ECE Dept./Rowan University MLP: Signal Flow Function signal Error signal    Computations at each node, j Neuron output, y j Gradient vector, dE/dw ji Forward propagation Backward propagation

7. S. Mandayam/ ANN/ECE Dept./Rowan UniversityBackpropagation i j k Left Right Notation At a node j, y i (n) w ji (n) v j (n)  (.) y j (n) d j (n) e j (n)

8. S. Mandayam/ ANN/ECE Dept./Rowan University Backprop. (contd) i j k Left Right Notation If node j is a hidden node, y i (n) w ji (n) v j (n)  (.) y j (n) w kj (n) v k (n)  (.) y k (n) d k (n) e k (n)

9. S. Mandayam/ ANN/ECE Dept./Rowan University MLP Training Forward Pass Fix w ji (n) Compute y j (n) Backward Pass Calculate  j (n) Update weights w ji (n+1) i j k Left Right i j k Left Right x y

10. S. Mandayam/ ANN/ECE Dept./Rowan University MLP’s in Matlab http://engineering.rowan.edu/~shreek/spring02/ann/demos/mlp.m

11. S. Mandayam/ ANN/ECE Dept./Rowan UniversitySummary