Introduction to Neural Networks Introduction to Neural Networks Applied to OCR and Speech Recognition An actual neuron A crude model of a neuron Computational Neural Networks: A computational approach inspired by the architecture of the biological nervous system

Introduction to Neural Networks Cat Neural Probe to Study Response

Introduction to Neural Networks The Perceptron Model

Introduction to Neural Networks Example Weights: And & Or Problems

Introduction to Neural Networks Weight Adjustments

Introduction to Neural Networks 3-D and 2-D Plot of AND Table

Introduction to Neural Networks Training Procedure 1.First assign any values to w 1, w 2 and w 3 2.Using the current weight values w 1, w 2 and w 3 and the next training item inputs I 1 and I 2 compute the value: 3.If V 0 set computed output C to 1 else set to 0. 4.If the computed output C is not the same as the current training item output O, Adjust Weights. 5.Repeat steps 2-4. If you run out of training items, start with the first training item. Stop repeating if no weight changes through 1 complete training cycle).

Introduction to Neural Networks Gradient Descent Algorithm

Introduction to Neural Networks Linearly vs. Non-Linearly Separable

Introduction to Neural Networks The XOR Problem is Linearly Non- separable

Introduction to Neural Networks The Back Propagation Model

Introduction to Neural Networks Advantage of Backprop over Perceptron

Introduction to Neural Networks Backprop Learning Algorithm 1. Assign random values to all the weights 2. Choose a pattern from the training set (similar to perceptron). 3. Propagate the signal through to get final output (similar to perceptron). 4. Compute the error for the output layer (similar to the perceptron). 5. Compute the errors in the preceding layers by propagating the error backwards. 6. Change the weight between neuron A and each neuron B in another layer by an amount proportional to the observed output of B and the error of A. 7. Repeat step 2 for next training sample.

Introduction to Neural Networks Application: Needs Enough Training

Introduction to Neural Networks Application to Speech Processing

Introduction to Neural Networks Speech

Introduction to Neural Networks Appendix

Introduction to Neural Networks Error Propagation Algorithm If neuron A in one layer is connected to B,C, and D in the output layer, it is responsible for the errors observed in B,C, and D. Thus, the error in A is computable by summing up the errors in B, C, and D weighted by the connection strengths between A and each B, C and D