1. Neural Networks John Riebe and Adam Profitt

2. What is a neuron? Weights: Weights are scalars that multiply each input element Summer: The summer sums the input elements, P R, together with the bias Bias: A bias is a number that is added to the total from the summer Translation Function: A translation function is one of many specific functions used in neural networking. P R Elements of the input vector WWeights ∑Summer bBias nSum of all P elements and b ƒTranslation Function aOutput

3. Layers of the Neural Network There are only three different types of layers in a network: The Input Layer Moves the input vectors into each neuron of the first hidden layer The Hidden Layers Performs the bulk of the computations in most networks Hidden layers are not always required The Output Layer Each neuron in the output layer outputs it’s own result

4. Translation Functions

5. Types of Neural Networks Perceptrons: 1.Used to classify data. 2.Applies the hard-limit transfer function. 3.Usually does not have any hidden layers. Linear Filters 1.Used to solve linearly separable problems. 2.Applies the linear transfer function. Backpropagation 1.Generally has only one hidden layer. 2.Can solve any reasonable problem. 3.Hidden layers use sigmoid translations, outputs use the linear transfer function

6. Training Neurons Training a network sets the biases and weights in each neuron Training algorithms 1.Gives a network an input 2.Receives the output 3.Calculates error between output and target 4.Adjusts weights and biases 5.Goes back to step 1 Each time the algorithm goes through the steps is called an epoch. Most networks go through many epochs. To train a network you need: A network An input A target vector There are many different types of training algorithms. To name a few: Levenberg-Marquardt BFGS quasi-Newton Bayesian regularization One step secant Random order incremental

7. Matlab Application

8. Create a feed-forward network Syntax net = newff net = newff(PR,[S1 S2...S i ],{TF1 TF2...TF i }) Description net = newff creates a new network with a dialog box. newff(PR,[S1 S2...S i ],{TF1 TF2...TF i }) takes, PR - R x 2 matrix of min and max values for R input elements. S i - Size of ith layer, for Nl layers. TF i - Transfer function of ith layer, default = 'tansig'. The newff Function

9. The train Function Trains a neural network Syntax net = train(net,P,T) Description train trains a network. train(net,P,T) takes, net - Neural network object. P - Network inputs. T - Network targets, default = zeros.

10. The sim Function Syntax sim(net,P) Description sim simulates neural networks. sim(net,P) takes, net - Network. P - Network inputs. The sim function simulates a neural network. This function feeds the network the input, P, and displays the results.

11. Transfer Functions Revisited Transfer functions: Hard-Limit a = hardlim(n) Linear a = purelin(n) Log-Sigmoid a = logsig(n) Tan-Sigmoid a = tansig(n) Outputs either a 1 or a 0 Outputs the scaled and summed input Squeezes the input to between 0 and 1 Squeezes the input to between -1 and 1

12. The Baum-Haussler Rule The Baum-Haussler Rule is one of the most useful rules for neural networks. N hidden ≤ (N train E tolerance ) / (N pts + N outputs ) This rule helps you determine the maximum number of neurons you will need for your network to function properly. This is NOT a law: it will not work in all situations. Sometimes you just have to use another method.

13. Bibliography Demuth, Howard and Mark Beale. “Neural Network Toolbox User’s Guide.“ 1992-2003 URL: http://www.mathworks.com/access/helpdesk/help/toolbox/nnet/nnet.shtml