Presentation is loading. Please wait.

Presentation is loading. Please wait.

November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 1 Self-Organizing Maps (Kohonen Maps) In the BPN, we used supervised.

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 1 Self-Organizing Maps (Kohonen Maps) In the BPN, we used supervised."— Presentation transcript:

1 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 1 Self-Organizing Maps (Kohonen Maps) In the BPN, we used supervised learning. This is not biologically plausible: In a biological system, there is no external “teacher” who manipulates the network’s weights from outside the network. Biologically more adequate: unsupervised learning. We will study Self-Organizing Maps (SOMs) as examples for unsupervised learning (Kohonen, 1980).

2 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 2 Self-Organizing Maps (Kohonen Maps) In the human cortex, multi-dimensional sensory input spaces (e.g., visual input, tactile input) are represented by two-dimensional maps. The projection from sensory inputs onto such maps is topology conserving. This means that neighboring areas in these maps represent neighboring areas in the sensory input space. For example, neighboring areas in the sensory cortex are responsible for the arm and hand regions.

3 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 3 Self-Organizing Maps (Kohonen Maps) Such topology-conserving mapping can be achieved by SOMs: Two layers: input layer and output (map) layer Two layers: input layer and output (map) layer Input and output layers are completely connected. Input and output layers are completely connected. Output neurons are interconnected within a defined neighborhood. Output neurons are interconnected within a defined neighborhood. A topology (neighborhood relation) is defined on the output layer. A topology (neighborhood relation) is defined on the output layer.

4 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 4 Self-Organizing Maps (Kohonen Maps) BPN structure: input vector x I1I1I1I1 output vector o I2I2I2I2 InInInIn O1O1O1O1 O2O2O2O2 O3O3O3O3 OmOmOmOm … …

5 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 5 Self-Organizing Maps (Kohonen Maps) Common output-layer structures: One-dimensional (completely interconnected for determining “winner” unit) Two-dimensional (connections omitted, only neighborhood relations shown [green]) i i Neighborhood of neuron i

6 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 6 Self-Organizing Maps (Kohonen Maps) A neighborhood function  (i, k) indicates how closely neurons i and k in the output layer are connected to each other. Usually, a Gaussian function on the distance between the two neurons in the layer is used:  position of i position of k

7 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 7 Unsupervised Learning in SOMs For n-dimensional input space and m output neurons: (1) Choose random weight vector w i for neuron i, i = 1,..., m (2) Choose random input x (3) Determine winner neuron k: ||w k – x|| = min i ||w i – x|| (Euclidean distance) (4) Update all weight vectors of all neurons i in the neighborhood of neuron k: w i := w i +  ·  (i, k)·(x – w i ) (w i is shifted towards x) (5) If convergence criterion met, STOP. Otherwise, narrow neighborhood function  and learning parameter  and go to (2).

8 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 8 Unsupervised Learning in SOMs Example I: Learning a one-dimensional representation of a two-dimensional (triangular) input space: 0 25000 20 100100010000

9 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 9 Unsupervised Learning in SOMs Example II: Learning a two-dimensional representation of a two-dimensional (square) input space:

10 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 10 Unsupervised Learning in SOMs Example III: Learning a two- dimensional mapping of texture images

11 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 11 The Hopfield Network The Hopfield model is a single-layered recurrent network. It is usually initialized with appropriate weights instead of being trained. The network structure looks as follows: X1X1X1X1 X2X2X2X2 XNXNXNXN …

12 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 12 The Hopfield Network We will focus on the discrete Hopfield model, because its mathematical description is more straightforward. In the discrete model, the output of each neuron is either 1 or –1. In its simplest form, the output function is the sign function, which yields 1 for arguments  0 and –1 otherwise.

13 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 13 The Hopfield Network We can set the weights in such a way that the network learns a set of different inputs, for example, images. The network associates input patterns with themselves, which means that in each iteration, the activation pattern will be drawn towards one of those patterns. After converging, the network will most likely present one of the patterns that it was initialized with. Therefore, Hopfield networks can be used to restore incomplete or noisy input patterns.

14 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 14 The Hopfield Network Example: Image reconstruction (Ritter, Schulten, Martinetz 1990) A 20  20 discrete Hopfield network was trained with 20 input patterns, including the one shown in the left figure and 19 random patterns as the one on the right.

15 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 15 The Hopfield Network After providing only one fourth of the “face” image as initial input, the network is able to perfectly reconstruct that image within only two iterations.

16 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 16 The Hopfield Network Adding noise by changing each pixel with a probability p = 0.3 does not impair the network’s performance. After two steps the image is perfectly reconstructed.

17 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 17 The Hopfield Network However, for noise created by p = 0.4, the network is unable the original image. Instead, it converges against one of the 19 random patterns.

18 November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 18 The Hopfield Network The Hopfield model constitutes an interesting neural approach to identifying partially occluded objects and objects in noisy images. These are among the toughest problems in computer vision. Notice, however, that Hopfield networks require the input patterns to always be in exactly the same position, otherwise they will fail to recognize them.

Download ppt "November 24, 2009Introduction to Cognitive Science Lecture 21: Self-Organizing Maps 1 Self-Organizing Maps (Kohonen Maps) In the BPN, we used supervised."

Similar presentations

© Negnevitsky, Pearson Education, 2005 1 Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.

© Negnevitsky, Pearson Education, Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.
Memristor in Learning Neural Networks

Memristor in Learning Neural Networks
2806 Neural Computation Self-Organizing Maps Lecture 9 2005 Ari Visa.

2806 Neural Computation Self-Organizing Maps Lecture Ari Visa.
Organizing a spectral image database by using Self-Organizing Maps Research Seminar 7.10.2005 Oili Kohonen.

Organizing a spectral image database by using Self-Organizing Maps Research Seminar Oili Kohonen.
November 18, 2010Neural Networks Lecture 18: Applications of SOMs 1 Assignment #3 Question 2 Regarding your cascade correlation projects, here are a few.

November 18, 2010Neural Networks Lecture 18: Applications of SOMs 1 Assignment #3 Question 2 Regarding your cascade correlation projects, here are a few.
Un Supervised Learning & Self Organizing Maps. Un Supervised Competitive Learning In Hebbian networks, all neurons can fire at the same time Competitive.

Un Supervised Learning & Self Organizing Maps. Un Supervised Competitive Learning In Hebbian networks, all neurons can fire at the same time Competitive.
Neural Networks Chapter 9 Joost N. Kok Universiteit Leiden.

Neural Networks Chapter 9 Joost N. Kok Universiteit Leiden.
Unsupervised learning. Summary from last week We explained what local minima are, and described ways of escaping them. We investigated how the backpropagation.

Unsupervised learning. Summary from last week We explained what local minima are, and described ways of escaping them. We investigated how the backpropagation.
Self Organization: Competitive Learning

Self Organization: Competitive Learning
5/16/2015Intelligent Systems and Soft Computing1 Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.

5/16/2015Intelligent Systems and Soft Computing1 Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.
Kohonen Self Organising Maps Michael J. Watts

Kohonen Self Organising Maps Michael J. Watts
Artificial neural networks:

Artificial neural networks:
Adaptive Resonance Theory (ART) networks perform completely unsupervised learning. Their competitive learning algorithm is similar to the first (unsupervised)

Adaptive Resonance Theory (ART) networks perform completely unsupervised learning. Their competitive learning algorithm is similar to the first (unsupervised)
Unsupervised Networks Closely related to clustering Do not require target outputs for each input vector in the training data Inputs are connected to a.

Unsupervised Networks Closely related to clustering Do not require target outputs for each input vector in the training data Inputs are connected to a.
Self-Organizing Map (SOM). Unsupervised neural networks, equivalent to clustering. Two layers – input and output – The input layer represents the input.

Self-Organizing Map (SOM). Unsupervised neural networks, equivalent to clustering. Two layers – input and output – The input layer represents the input.
Machine Learning: Connectionist McCulloch-Pitts Neuron Perceptrons Multilayer Networks Support Vector Machines Feedback Networks Hopfield Networks.

Machine Learning: Connectionist McCulloch-Pitts Neuron Perceptrons Multilayer Networks Support Vector Machines Feedback Networks Hopfield Networks.
X0 xn w0 wn o Threshold units SOM.

X0 xn w0 wn o Threshold units SOM.
Slides are based on Negnevitsky, Pearson Education, 2005 1 Lecture 8 Artificial neural networks: Unsupervised learning n Introduction n Hebbian learning.

Slides are based on Negnevitsky, Pearson Education, Lecture 8 Artificial neural networks: Unsupervised learning n Introduction n Hebbian learning.
Self-Organizing Hierarchical Neural Network

Self-Organizing Hierarchical Neural Network
November 19, 2009Introduction to Cognitive Science Lecture 20: Artificial Neural Networks I 1 Artificial Neural Network (ANN) Paradigms Overview: The Backpropagation.

November 19, 2009Introduction to Cognitive Science Lecture 20: Artificial Neural Networks I 1 Artificial Neural Network (ANN) Paradigms Overview: The Backpropagation.

Similar presentations

Ads by Google