Artificial Neural Networks An Introduction

What is a Neural Network? A human Brain A porpoise brain The brain in a living creature A computer program Simulates (at a very rudimentary level) a biological brain Limited connections

Artificial Neural Networks Artificial neural networks are information technology inspired by studies of the brain and nervous system ANNs are used to simulate the massively parallel processes that are effectively used in the brain for learning, and storing information and knowledge

Biological Neuron Dendrites Axon Soma Membrane Synapse Neurotransmitter Spikes

Simple Neuron Configuration Summation (weighted) Transfer Output (Y) InputsWeights W1W1 W2W2 W3W3 W4W4 X1X1 X2X2 X3X3 X4X4

Threshold Logic Units Outputs are 0 or 1 If the activation (accumulated weighted input) is larger than threshold the unit generates a signal

Sigmoidal Transfer function Outputs are in the range from 0 to 1 y=1/(1+exp(-a)) Is differentiable

Neural Network Architecture In feedforward NN, neurons are grouped into layers The neurons on each layer are the same type There are different types of layers Input layer: receive input from external sources Output layer: communicate to user Hidden layer(s): neurons communicate only with other layers

Sample Network Configuration Input layer Hidden layer Output layer

Some Characteristics of ANN Tolerance to noise; Reliability; Two layer networks are restricted to linearly separable problems; Additional layers can solve more complicated problems; “ Black Box ”. Why? Non-linearity; Logic hidden in weights; Universal approximators.

Learning Methods Supervised Error Backpropagation Counter-Propagation Unsupervised Hebb ’ s rule Competitive Learning Reinforcement

Error Backpropagation Algorithm Generalized Delta Rule; Allowed training multi-layer ANN; Revived interest in ANN; Error terms are propagated back through the network; The weight coefficients are updated iteratively;

Error Backpropagation Algorithm: Drawbacks Local Minima; Biologically implausible; Possibility of “ network paralysis ” ; Slowness; Oscillations.

Problems solved by ANN Classification Cluster Analysis Approximation Forecasting Association Data compression

Benefits of ANN Parallelism Learning Generalization NN can learn the characteristics of a general category of objects on specific examples from that category Robustness (reliability) Tolerance to noise Performance does not degrade appreciably if some of its neurons or interconnections are lost (Distributed memory)

Limitations of ANN Two-layer NN limited to linearly separable problems Local minima & oscillations Number of hidden layers/units hard to determine Lack of transparency (perspicuity)

Sample of Applications Business Credit scoring Bankruptcy prediction Bond rating Security trading Technological processes Robotics Consumer electronics