Download presentation
Presentation is loading. Please wait.
Published byKai Manninen Modified over 5 years ago
1
Self Organizing Maps A major principle of organization is the topographic map, i.e. groups of adjacent neurons process information from neighboring parts of the sensory systems. Topographic maps can be distorted in the sense that the amount of neurons involved is more re- lated to the importance of the task performed, than to the size of the region of the body surface that provides the input signals. Biological inspiration Brain makes maps of sensory input 29-Apr-19 Rudolf Mak TU/e Computer Science
2
Brain Maps A part of the brain that contains many topographic
maps is the cerebral cortex. Some of these are: Visual cortex Various maps, such as retinotopic map Somatosensory cortex Somatotopic map Auditory cortex Tonotopic map Sensors: pick up energy from environment) Taste (chemical) Smell (chemical) Vision (photoelectric) Hearing (mechanical, vibrations) Touch (mechanical, thermal) Skin (2D surface) cortex (2D surface) (first approximation) 29-Apr-19 Rudolf Mak TU/e Computer Science
3
Somatotopic Map The somatosensory cortex processes the
information of the sen- sory neurons that lie below the skin. Note that both the skin and the somatosensory cortex can be seen as two-dimensional spaces Note the large part devoted to lips and hands Cross section view 29-Apr-19 Rudolf Mak TU/e Computer Science
4
Somatosensory Man Picture of the male body with the body
parts scaled accor- ding to the area de- voted to these parts in the somatosenso- ry cortex Man toolmaker?! 29-Apr-19 Rudolf Mak TU/e Computer Science
5
Unsupervised Selforganizing Learning
The neurons are arranged in some grid of fixed topology The winning neuron is the neuron with its weight vector nearest to the supplied input vector In principle all neurons are allowed to change their weight The amount of change of a neuron, however, depends on the distance (in the grid) of that neuron to the winning neuron. Larger distance implies smaller change. Map requires neurons to be positioned in a grid Should be a notion of neighborhood proximity 29-Apr-19 Rudolf Mak TU/e Computer Science
6
Grid Topologies The following topologies are frequently used
One-dimensional grids Line Ring Two-dimensional grids Square grid Torus Hexagonal grid If additional knowledge of the input space is avail- able more sophisticated topologies can be used. 29-Apr-19 Rudolf Mak TU/e Computer Science
7
Neighborhoods & box distance
Square and hexagonal grid with neighborhoods based on box distance Grid-lines are not shown 29-Apr-19 Rudolf Mak TU/e Computer Science
8
Manhattan or Link Distance
4 3 2 1 Distance to the cen- tral cell measured in number of links 29-Apr-19 Rudolf Mak TU/e Computer Science
9
Euclidean Distance 2 1 29-Apr-19 Rudolf Mak TU/e Computer Science
10
Topologically Correct Maps
The aim of unsupervised self-organizing learning is to construct a topologically correct map of the input space. For any two neurons i and j in the grid, let d(i,j) be their fixed distance in the grid. A mapping is called topological correct when 29-Apr-19 Rudolf Mak TU/e Computer Science
11
Neighborhood Functions
The allowed weight change of neuron j when i is the winning neuron is given by the neighbor- hood function h(i, j). Common choices are: (Winner takes it all) Beta large h(i,j) ~ 1 large neighborhood Beta small, small neighborhood Simple competitive learning is a special case 29-Apr-19 Rudolf Mak TU/e Computer Science
12
Unsupervised Self-organizing Learning (incremental version)
29-Apr-19 Rudolf Mak TU/e Computer Science
13
Unsupervised Self-organizing Learning (batch version)
29-Apr-19 Rudolf Mak TU/e Computer Science
14
Error Function For a network with weight matrix W and training set we define the error function E(W) by Let , then 29-Apr-19 Rudolf Mak TU/e Computer Science
15
Gradients of the Error functions
Because It follows that the gradient of the error is given by 29-Apr-19 Rudolf Mak TU/e Computer Science
16
Tuning the Learning Process
The learning process usually consists of two phases: A phase in which the weight vectors reorder and become disentangled. In this phase neigh-borhoods (b) must be large. A phase in which the weight vectors are fine- tuned to the part of the training set for which they are the respective winners. In this phase the neighborhoods (b) must be small to avoid interference from other neurons. 29-Apr-19 Rudolf Mak TU/e Computer Science
17
Phonotopic Map Input vectors are 15 dimensional speech samples from the Finnish language Each vector component represents the average output power over 10ms interval in a certain range of the spectrum (200 Hz – 6400 Hz) Neurons are organized in a 8x12 hexagonal grid After formation of the map, the individual neurons were calibrated to represent phonemes The resulting map is called the phonetic typewriter 29-Apr-19 Rudolf Mak TU/e Computer Science
18
Phonetic Typewriter The phonetic typewriter is constructed by Tuevo Kohonen, see e.g. his book “Self-Organizing Maps”, Springer, 1995. 29-Apr-19 Rudolf Mak TU/e Computer Science
19
Travelling Salesman Problem
TSP is one of the notorious difficult (NP-Complete) combinatorial optimization problems. The so-called elastic net method can be used to (approximately) solve the Euclidean version of this problem (Durbin and Willshaw). To that end one uses a SOM in which the neurons are arranged in a one-dimensional cycle. 29-Apr-19 Rudolf Mak TU/e Computer Science
20
Space-filling curves More space filling curves 29-Apr-19
Rudolf Mak TU/e Computer Science
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.