1. S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE.09.454/ECE.09.560 Fall 2010 Shreekanth Mandayam ECE Department Rowan University http://engineering.rowan.edu/~shreek/fall10/ann/ Lecture 3 September 27, 2010

2. S. Mandayam/ ANN/ECE Dept./Rowan UniversityPlan Multilayer Perceptron Architecture Signal Flow Learning rule - Backpropagation Matlab MLP Demo Finish Lab Project 1 Start Lab Project 2

3. S. Mandayam/ ANN/ECE Dept./Rowan University Multilayer Perceptron (MLP): Architecture 1 1 1         x1x1 x2x2 x3x3 y1y1 y2y2   w ji w kj w lk Input Layer Hidden Layers Output Layer Inputs Outputs

4. S. Mandayam/ ANN/ECE Dept./Rowan University MLP: Characteristics Neurons possess sigmoidal (logistic) activation functions Contains one or more “hidden layers” Trained using the “backpropagation” algorithm MLP with 1-hidden layer is a “universal approximator” 1 0 0.5 1  (t) t

5. S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Network Massively parallel distributed processor made up of simple processing units, which can store and retrieve experiential knowledge The network “learns” from the data presented to it The “knowledge” is stored in the interconnection weights Adapted from Haykin

6. S. Mandayam/ ANN/ECE Dept./Rowan University MLP: Signal Flow Function signal Error signal    Computations at each node, j Neuron output, y j Gradient vector, dE/dw ji Forward propagation Backward propagation

7. S. Mandayam/ ANN/ECE Dept./Rowan UniversityBackpropagation i j k Left Right Notation At a node j, y i (n) w ji (n) v j (n)  (.) y j (n) d j (n) e j (n)

8. S. Mandayam/ ANN/ECE Dept./Rowan University Backprop. (contd) i j k Left Right Notation If node j is a hidden node, y i (n) w ji (n) v j (n)  (.) y j (n) w kj (n) v k (n)  (.) y k (n) d k (n) e k (n)

9. S. Mandayam/ ANN/ECE Dept./Rowan University MLP Training Forward Pass Fix w ji (n) Compute y j (n) Backward Pass Calculate  j (n) Update weights w ji (n+1) i j k Left Right i j k Left Right x y

10. S. Mandayam/ ANN/ECE Dept./Rowan University MLP’s in Matlab http://engineering.rowan.edu/~shreek/fall10/ann/demos/mlp.m

11. S. Mandayam/ ANN/ECE Dept./Rowan University Lab Projects 1 and 2 http://engineering.rowan.edu/~shreek /fall10/ann/lab1.htmlhttp://engineering.rowan.edu/~shreek /fall10/ann/lab1.html http://engineering.rowan.edu/~shreek /fall10/ann/lab2.htmlhttp://engineering.rowan.edu/~shreek /fall10/ann/lab2.html

12. S. Mandayam/ ANN/ECE Dept./Rowan UniversitySummary