Presentation is loading. Please wait.

Presentation is loading. Please wait.

Kohonen Self-organizing Feature Maps

Published byΒαριησού Μιαούλης Modified over 5 years ago

Similar presentations

Presentation on theme: "Kohonen Self-organizing Feature Maps"— Presentation transcript:

1 Kohonen Self-organizing Feature Maps
Order out of randomness

2 Copyright Gene A. Taglairini, PhD
Kohonen Network Model Neurons Inputs 11/26/2018 Copyright Gene A. Taglairini, PhD

3 Copyright Gene A. Taglairini, PhD
Network Features Input nodes are connected to every neuron The “winner” neuron is the one whose weights are most “similar” to the input Neurons participate in a “winner-take-all” behavior The winner output is set to 1 and all others to 0 Only weights to the winner and its neighbors are adapted 11/26/2018 Copyright Gene A. Taglairini, PhD

4 Copyright Gene A. Taglairini, PhD
Neighborhoods Symmetric with maximum value weight changes applied near the center Excitatory near the center, surrounded by an inhibitory band, and may be enclosed by an excitatory influence rapidly decreasing to zero 11/26/2018 Copyright Gene A. Taglairini, PhD

5 Network Equations: Similarity
Similarity is measured using the Euclidean distance from an input pattern vector inputp The vector of weights w[i] represents all the weights to neuron i; hence, w[i][j] is the weight that joins input j to neuron i The winner has the smallest value of s 11/26/2018 Copyright Gene A. Taglairini, PhD

6 Network Equations: Weight Adaptation
Suppose the winning neuron has index a For an input j and neuron i, the weight change Dw[i][j] and the new value of the weight w[i][j] are: Dw[i][j] = h (input[j] – w[i][j] ) NbdWt(a, i) w[i][j] = w[i][j] + Dw[i][j] Where h is the learning rate and NbdWt(a, i) is the neighborhood weighting function 11/26/2018 Copyright Gene A. Taglairini, PhD

7 Network Equations: Neighborhood Weighting Function
The neighborhood weighting function may take forms such as: Where s is a scalar that sets the dilation of the weighting function 11/26/2018 Copyright Gene A. Taglairini, PhD

8 On-center Off-surround Neighborhood
A field that: Reinforces stimuli “near” the center Attenuates effects in a region about the center Abates rapidly outside the attenuating region Contributes to noise control and localizes representation Complementary to off-center on-surround 11/26/2018 Copyright Gene A. Taglairini, PhD

9 On-center Off-Surround Neighborhood
11/26/2018 Copyright Gene A. Taglairini, PhD

10 Mexican Hat Neighborhood
11/26/2018 Copyright Gene A. Taglairini, PhD

11 Basic Training Algorithm
Initialize weights from input nodes to neurons Choose a neighborhood function While (input patterns mismatch weights) Find a winner Adapt weights in the vicinity of the winner Develop an interpretation for the encoding—identify which neurons encode what patterns 11/26/2018 Copyright Gene A. Taglairini, PhD

12 Copyright Gene A. Taglairini, PhD
Applications Speech encoding for a phonetic typewriter Geometric pattern coding 11/26/2018 Copyright Gene A. Taglairini, PhD

Download ppt "Kohonen Self-organizing Feature Maps"

Similar presentations

© Negnevitsky, Pearson Education, 2005 1 Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.

© Negnevitsky, Pearson Education, Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.
Memristor in Learning Neural Networks

Memristor in Learning Neural Networks
2806 Neural Computation Self-Organizing Maps Lecture 9 2005 Ari Visa.

2806 Neural Computation Self-Organizing Maps Lecture Ari Visa.
Un Supervised Learning & Self Organizing Maps. Un Supervised Competitive Learning In Hebbian networks, all neurons can fire at the same time Competitive.

Un Supervised Learning & Self Organizing Maps. Un Supervised Competitive Learning In Hebbian networks, all neurons can fire at the same time Competitive.
Neural Networks Chapter 9 Joost N. Kok Universiteit Leiden.

Neural Networks Chapter 9 Joost N. Kok Universiteit Leiden.
Unsupervised learning. Summary from last week We explained what local minima are, and described ways of escaping them. We investigated how the backpropagation.

Unsupervised learning. Summary from last week We explained what local minima are, and described ways of escaping them. We investigated how the backpropagation.
Self Organization: Competitive Learning

Self Organization: Competitive Learning
5/16/2015Intelligent Systems and Soft Computing1 Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.

5/16/2015Intelligent Systems and Soft Computing1 Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.
Kohonen Self Organising Maps Michael J. Watts

Kohonen Self Organising Maps Michael J. Watts
Unsupervised Learning with Artificial Neural Networks The ANN is given a set of patterns, P, from space, S, but little/no information about their classification,

Unsupervised Learning with Artificial Neural Networks The ANN is given a set of patterns, P, from space, S, but little/no information about their classification,
Artificial neural networks:

Artificial neural networks:
Competitive learning College voor cursus Connectionistische modellen M Meeter.

Competitive learning College voor cursus Connectionistische modellen M Meeter.
Unsupervised Networks Closely related to clustering Do not require target outputs for each input vector in the training data Inputs are connected to a.

Unsupervised Networks Closely related to clustering Do not require target outputs for each input vector in the training data Inputs are connected to a.
X0 xn w0 wn o Threshold units SOM.

X0 xn w0 wn o Threshold units SOM.
Self Organizing Maps. This presentation is based on: SOM’s are invented by Teuvo Kohonen. They represent multidimensional.

Self Organizing Maps. This presentation is based on: SOM’s are invented by Teuvo Kohonen. They represent multidimensional.
Slides are based on Negnevitsky, Pearson Education, 2005 1 Lecture 8 Artificial neural networks: Unsupervised learning n Introduction n Hebbian learning.

Slides are based on Negnevitsky, Pearson Education, Lecture 8 Artificial neural networks: Unsupervised learning n Introduction n Hebbian learning.
November 9, 2010Neural Networks Lecture 16: Counterpropagation 1 Unsupervised Learning So far, we have only looked at supervised learning, in which an.

November 9, 2010Neural Networks Lecture 16: Counterpropagation 1 Unsupervised Learning So far, we have only looked at supervised learning, in which an.
November 2, 2010Neural Networks Lecture 14: Radial Basis Functions 1 Cascade Correlation Weights to each new hidden node are trained to maximize the covariance.

November 2, 2010Neural Networks Lecture 14: Radial Basis Functions 1 Cascade Correlation Weights to each new hidden node are trained to maximize the covariance.
KNN, LVQ, SOM. Instance Based Learning K-Nearest Neighbor Algorithm (LVQ) Learning Vector Quantization (SOM) Self Organizing Maps.

KNN, LVQ, SOM. Instance Based Learning K-Nearest Neighbor Algorithm (LVQ) Learning Vector Quantization (SOM) Self Organizing Maps.
Neural Networks based on Competition

Neural Networks based on Competition

Similar presentations

Ads by Google