Presentation is loading. Please wait.

Presentation is loading. Please wait.

November 18, 2010Neural Networks Lecture 18: Applications of SOMs 1 Assignment #3 Question 2 Regarding your cascade correlation projects, here are a few.

Published byReynard Warren Modified over 9 years ago

Similar presentations

Presentation on theme: "November 18, 2010Neural Networks Lecture 18: Applications of SOMs 1 Assignment #3 Question 2 Regarding your cascade correlation projects, here are a few."— Presentation transcript:

1 November 18, 2010Neural Networks Lecture 18: Applications of SOMs 1 Assignment #3 Question 2 Regarding your cascade correlation projects, here are a few tips to make your life easier. First of all, the book suggests that after adding a new hidden-layer unit and training its weights, in the output layer we only need to train the weights of the newly added connections (or – your instructor’s idea - just use linear regression to determine them). While that is a very efficient solution, the original paper on cascade correlation suggests to always retrain all output layer weights after adding a hidden- layer unit.

2 November 18, 2010Neural Networks Lecture 18: Applications of SOMs 2 Assignment #3 Question 2 This will require more training, but it may find a better (=lower error) overall solution for the weight vectors. Furthermore, it will be easier for you to use the same training procedure over and over again instead of writing a single-weight updating function or a linear regression function. For this output weight training, you can simply use your backpropagation algorithm and remove the hidden-layer training. The cascade correlation authors suggest Quickprop for speedup, but Rprop also works.

3 November 18, 2010Neural Networks Lecture 18: Applications of SOMs 3 Assignment #3 Question 2 In order to train the weights of a new hidden-layer unit, you need to know the current error for each output neuron and each exemplar. You can compute these values once and store them in an array. After creating a new hidden unit with random weights and before training it, determine the current sign S k of the covariance between the unit’s output and the error in output unit k (do not update S k during training, it can lead to convergence problems).

4 November 18, 2010Neural Networks Lecture 18: Applications of SOMs 4 Assignment #3 Question 2 For the hidden-layer training, you can also use Quickprop or Rprop. Once a new hidden-layer unit has been installed and trained, its weights and thus its output for a given network input will never change. Therefore, you can store the outputs of all hidden units in arrays and use these stored data for the remainder of the network buildup/training. No optimizations are required for this question (sorry, no prizes here), but it is interesting to try it anyway.

5 November 18, 2010Neural Networks Lecture 18: Applications of SOMs 5 Self-Organizing Maps (Kohonen Maps) Network structure: input vector x x1x1x1x1 output vector o x2x2x2x2 xnxnxnxn O1O1O1O1 O2O2O2O2 O3O3O3O3 OmOmOmOm … …

6 November 18, 2010Neural Networks Lecture 18: Applications of SOMs 6 Self-Organizing Maps (Kohonen Maps)

7 November 18, 2010Neural Networks Lecture 18: Applications of SOMs 7 Unsupervised Learning in SOMs In the textbook, a different kind of neighborhood function is used. Instead of having a smooth, continuous function  (i, k) to indicate connection strength, a neighborhood boundary is defined. All neurons within the neighborhood of the winner unit adapt their weights to the current input by exactly the same proportion . The size of the neighborhood is decreased over time.

8 November 18, 2010Neural Networks Lecture 18: Applications of SOMs 8 Unsupervised Learning in SOMs N.hood for 0  t < 10 N.hood for 10  t < 20 N.hood for 20  t < 30 N.hood for 30  t < 40 N.hood for t > 39

9 November 18, 2010Neural Networks Lecture 18: Applications of SOMs 9 Unsupervised Learning in SOMs Example I: Learning a one-dimensional representation of a two-dimensional (triangular) input space: 0 25000 20 100100010000

10 November 18, 2010Neural Networks Lecture 18: Applications of SOMs 10 Unsupervised Learning in SOMs Example II: Learning a two-dimensional representation of a two-dimensional (square) input space:

11 November 18, 2010Neural Networks Lecture 18: Applications of SOMs 11 Unsupervised Learning in SOMs Example III: Learning a two- dimensional mapping of texture images

12 November 18, 2010Neural Networks Lecture 18: Applications of SOMs 12 Unsupervised Learning in SOMs Examples IV and V: Learning two-dimensional mappings of RGB colors and NFL images: http://www.shy.am/2005/12/kohonen-self-organizing- map-demos/ Example VI: Interactive SOM learning of two- and three- dimensional shapes: http://www.cs.umb.edu/~marc/cs672/wsom.exe

13 November 18, 2010Neural Networks Lecture 18: Applications of SOMs 13 Unsupervised Learning in SOMs Example VII: A Self-organizing Semantic Map for Information Retrieval (Xia Lin, Dagobert Soergel, Gary Marchionini) http://www.cs.umb.edu/~marc/cs672/lin1991.pdf

Download ppt "November 18, 2010Neural Networks Lecture 18: Applications of SOMs 1 Assignment #3 Question 2 Regarding your cascade correlation projects, here are a few."

Similar presentations

Multi-Layer Perceptron (MLP)

Multi-Layer Perceptron (MLP)
Beyond Linear Separability

Beyond Linear Separability
1 Image Classification MSc Image Processing Assignment March 2003.

1 Image Classification MSc Image Processing Assignment March 2003.
Unsupervised learning. Summary from last week We explained what local minima are, and described ways of escaping them. We investigated how the backpropagation.

Unsupervised learning. Summary from last week We explained what local minima are, and described ways of escaping them. We investigated how the backpropagation.
Kohonen Self Organising Maps Michael J. Watts

Kohonen Self Organising Maps Michael J. Watts
Adaptive Resonance Theory (ART) networks perform completely unsupervised learning. Their competitive learning algorithm is similar to the first (unsupervised)

Adaptive Resonance Theory (ART) networks perform completely unsupervised learning. Their competitive learning algorithm is similar to the first (unsupervised)
Unsupervised Networks Closely related to clustering Do not require target outputs for each input vector in the training data Inputs are connected to a.

Unsupervised Networks Closely related to clustering Do not require target outputs for each input vector in the training data Inputs are connected to a.
Self-Organizing Map (SOM). Unsupervised neural networks, equivalent to clustering. Two layers – input and output – The input layer represents the input.

Self-Organizing Map (SOM). Unsupervised neural networks, equivalent to clustering. Two layers – input and output – The input layer represents the input.
Machine Learning: Connectionist McCulloch-Pitts Neuron Perceptrons Multilayer Networks Support Vector Machines Feedback Networks Hopfield Networks.

Machine Learning: Connectionist McCulloch-Pitts Neuron Perceptrons Multilayer Networks Support Vector Machines Feedback Networks Hopfield Networks.
X0 xn w0 wn o Threshold units SOM.

X0 xn w0 wn o Threshold units SOM.
Simple Neural Nets For Pattern Classification

Simple Neural Nets For Pattern Classification
Radial Basis Functions

Radial Basis Functions
November 19, 2009Introduction to Cognitive Science Lecture 20: Artificial Neural Networks I 1 Artificial Neural Network (ANN) Paradigms Overview: The Backpropagation.

November 19, 2009Introduction to Cognitive Science Lecture 20: Artificial Neural Networks I 1 Artificial Neural Network (ANN) Paradigms Overview: The Backpropagation.
1 Chapter 11 Neural Networks. 2 Chapter 11 Contents (1) l Biological Neurons l Artificial Neurons l Perceptrons l Multilayer Neural Networks l Backpropagation.

1 Chapter 11 Neural Networks. 2 Chapter 11 Contents (1) l Biological Neurons l Artificial Neurons l Perceptrons l Multilayer Neural Networks l Backpropagation.
September 30, 2010Neural Networks Lecture 8: Backpropagation Learning 1 Sigmoidal Neurons In backpropagation networks, we typically choose  = 1 and 

September 30, 2010Neural Networks Lecture 8: Backpropagation Learning 1 Sigmoidal Neurons In backpropagation networks, we typically choose  = 1 and 
November 9, 2010Neural Networks Lecture 16: Counterpropagation 1 Unsupervised Learning So far, we have only looked at supervised learning, in which an.

November 9, 2010Neural Networks Lecture 16: Counterpropagation 1 Unsupervised Learning So far, we have only looked at supervised learning, in which an.
September 16, 2010Neural Networks Lecture 4: Models of Neurons and Neural Networks 1 Capabilities of Threshold Neurons By choosing appropriate weights.

September 16, 2010Neural Networks Lecture 4: Models of Neurons and Neural Networks 1 Capabilities of Threshold Neurons By choosing appropriate weights.
November 2, 2010Neural Networks Lecture 14: Radial Basis Functions 1 Cascade Correlation Weights to each new hidden node are trained to maximize the covariance.

November 2, 2010Neural Networks Lecture 14: Radial Basis Functions 1 Cascade Correlation Weights to each new hidden node are trained to maximize the covariance.
Neural Networks Chapter 8. 8.1 Feed-Forward Neural Networks.

Neural Networks Chapter Feed-Forward Neural Networks.
November 30, 2010Neural Networks Lecture 20: Interpolative Associative Memory 1 Associative Networks Associative networks are able to store a set of patterns.

November 30, 2010Neural Networks Lecture 20: Interpolative Associative Memory 1 Associative Networks Associative networks are able to store a set of patterns.

Similar presentations

Ads by Google