1. S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE.09.454/ECE.09.560 Fall 2008 Shreekanth Mandayam ECE Department Rowan University http://engineering.rowan.edu/~shreek/fall08/ann/ Lecture 4 September 29, 2008

2. S. Mandayam/ ANN/ECE Dept./Rowan UniversityPlan Finish Lab Project 1 Multilayer Perceptron Architecture Signal Flow Learning rule - Backpropagation Matlab MLP Demo Begin Lab Project 2

3. S. Mandayam/ ANN/ECE Dept./Rowan University Lab Project 1 http://engineering.rowan.edu/~shreek /fall08/ann/lab1.htmlhttp://engineering.rowan.edu/~shreek /fall08/ann/lab1.html UCI Machine Learning Repository: http://www.ics.uci.edu/~mlearn/MLRepository.html

4. S. Mandayam/ ANN/ECE Dept./Rowan University Multilayer Perceptron (MLP): Architecture 1 1 1         x1x1 x2x2 x3x3 y1y1 y2y2   w ji w kj w lk Input Layer Hidden Layers Output Layer Inputs Outputs

5. S. Mandayam/ ANN/ECE Dept./Rowan University MLP: Characteristics Neurons possess sigmoidal (logistic) activation functions Contains one or more “hidden layers” Trained using the “backpropagation” algorithm MLP with 1-hidden layer is a “universal approximator” 1 0 0.5 1  (t) t

6. S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Network Massively parallel distributed processor made up of simple processing units, which can store and retrieve experiential knowledge The network “learns” from the data presented to it The “knowledge” is stored in the interconnection weights Adapted from Haykin

7. S. Mandayam/ ANN/ECE Dept./Rowan University MLP: Signal Flow Function signal Error signal    Computations at each node, j Neuron output, y j Gradient vector, dE/dw ji Forward propagation Backward propagation

8. S. Mandayam/ ANN/ECE Dept./Rowan UniversityBackpropagation i j k Left Right Notation At a node j, y i (n) w ji (n) v j (n)  (.) y j (n) d j (n) e j (n)

9. S. Mandayam/ ANN/ECE Dept./Rowan University Backprop. (contd) i j k Left Right Notation If node j is a hidden node, y i (n) w ji (n) v j (n)  (.) y j (n) w kj (n) v k (n)  (.) y k (n) d k (n) e k (n)

10. S. Mandayam/ ANN/ECE Dept./Rowan University MLP Training Forward Pass Fix w ji (n) Compute y j (n) Backward Pass Calculate  j (n) Update weights w ji (n+1) i j k Left Right i j k Left Right x y

11. S. Mandayam/ ANN/ECE Dept./Rowan University MLP’s in Matlab http://engineering.rowan.edu/~shreek/fall08/ann/demos/mlp.m

12. S. Mandayam/ ANN/ECE Dept./Rowan University Lab Project 2 http://engineering.rowan.edu/~shreek /fall08/ann/lab2.htmlhttp://engineering.rowan.edu/~shreek /fall08/ann/lab2.html UCI Machine Learning Repository: http://www.ics.uci.edu/~mlearn/MLRepository.html

13. S. Mandayam/ ANN/ECE Dept./Rowan UniversitySummary