Outcomes  Look at the theory of self-organisation.  Other self-organising networks  Look at examples of neural network applications

Four requirements for SOM Weights in neuron must represent a class of pattern  one neuron, one class

Four requirements for SOM Inputs pattern presented to all neurons and each produces an output.  Output: measure of the match between input pattern and pattern stored by neuron.

Four requirements A competitive learning strategy selects neuron with largest response.

Four requirements A method of reinforcing the largest response.

Architecture  The Kohonen network (named after Teuvo Kohonen from Finland) is a self-organising network  Neurons are usually arranged on a 2- dimensional grid  Inputs are sent to all neurons  There are no connections between neurons

Architecture Kohonen network X

Theory  For a neuron output (j) is a weighted some:  Where x is the input, w is the weights, net is the output of the neuron

Four requirement-Kohonen networks  True  Euclidean distance and weighted sum  Winner takes all  Learning rule of Kohonen learning

Output value  The output of each neuron is the weighted sum  There is no threshold or bias  Input values and weights are normalized

“Winner takes all”  Initially the weights in each neuron are random  Input values are sent to all the neurons  The outputs of each neuron are compared  The “winner” is the neuron with the largest output value

Training  Having found the winner, the weights of the winning neuron are adjusted  Weights of neurons in a surrounding neighbourhood are also adjusted

Neighbourhood X Kohonen network neighbourhood

Training  As training progresses the neighbourhood gets smaller  Weights are adjusted according to the following formula:

Weight adjustment  The learning coefficient (alpha) starts with a value of 1 and gradually reduces to 0  This has the effect of making big changes to the weights initially, but no changes at the end  The weights are adjusted so that they more closely resemble the input patterns

Example  A Kohonen network receives the input pattern  Two neurons in the network have weights and -0.6 –  Which neuron will have its weights adjusted and what will the new values of the weights be if the learning coefficient is 0.4?

Answer

Summary  The Kohonen network is self-organising  It uses unsupervised training  All the neurons are connected to the input  A winner takes all mechanism determines which neuron gets its weights adjusted  Neurons in a neighbourhood also get adjusted

Demonstration  A demonstration of a Kohonen network learning has been taken from the following websites:  

Applications of Neural Networks

Example Applications  Analysis of data  Classifying in EEG  Pattern recognition in ECG  EMG disease detection.

Gueli N et al (2005) The influence of lifestyle on cardiovascular risk factors analysis using a neural network Archives of Gerontology and Geriatrics –172  To produce a model of risk facts in heart disease.  MLP used  The accuracy was relatively good for chlorestremia and triglyceremdia:  Training phase around 99%  Testing phase around 93%  Not so good for HDL

Subasi A (in press) Automatic recognition of alertness level from EEG by using neural network and wavelet coefficients Expert Systems with Applications xx (2004) 1–11  Electroencephalography (EEG)  Recordings of electrical activity from the brain.  Classifying operation  Awake  Drowsy  Sleep

 MLP   Hidden layer – log-tanh function  Output layer – log-sigmoid function  Input is normalise to be within the range 0 to 1.

 Accuracy  95%+/-3% alert  93%+/-4% drowsy  92+/-5% sleep  Feature were extracted and form the input to the network, from wavelets.

Karsten Sternickel (2002) Automatic pattern recognition in ECG time series Computer Methods and Programs in Biomedicine –115  ECG – electrocardiographs – electrical signals from the heart.  Wavelets again.  Classification of patterns  Patterns were spotted

Abel et al (1996) Neural network analysis of the EMG interference pattern Med. Eng. Phys. Vol. 18, No. 1. pp. 12-l 7  EMG – Electromyography – muscle activity.  Interference patterns are signals produce from various parts of a muscle- hard to see features.  Applied neural network to EMG interference patterns.

 Classifying  Nerve disease  Muscle disease  Controls  Applied various different ways of presenting the pattern to the ANN.  Good for less serve cases, serve cases can often be see by the clinician.

Example Applications  Wave prediction  Controlling a vehicle  Condition monitoring

Wave prediction  Raoa S, Mandal S(2005) Hindcasting of storm waves using neural networks Ocean Engineering 32 (2005) 667–684  MLP used to predict storm waves.  2:2:2 network  Good correlation between ANN model and another model

van de Ven P, Flanagan C, Toal D (in press) Neural network control of underwater vehicles Engineering Applications of Artificial Intelligence  Semiautomous vehicle  Control using ANN  ANN replaces a mathematical model of the system.

Silva et al (2000) THE ADAPTABILITY OF A TOOL WEAR MONITORING SYSTEM UNDER CHANGING CUTTING CONDITIONS Mechanical Systems and Signal Processing (2000) 14(2),  Modelling tool wear  Combines ANN with other AI (Expert systems)  Self organising Maps (SOM) and ART2 investigated  SOM better for extracting the required information.

Examples to try yourself  A.1 Number recognition (ONR)  neuralNetworks neuralNetworks  Details: mple_ocr.asp mple_ocr.asp

 B.1 Kohonen Self Organising Example 1  neuralNetworks neuralNetworks  B.2 Kohonen 3D travelling salesman problem  regensburg.de/~saj39122/jfroehl/diplom/e- index.html regensburg.de/~saj39122/jfroehl/diplom/e- index.html