1. Pattern Association

2. Introduction
Pattern association involves associating a new pattern with a stored pattern.
Is a “simplified” model of human memory.
Types of associative memory:
Heteroassociative memory
Autoassociative memory
Hopfield Net
Bidirectional Associative Memory (BAM)

3. Introduction These are usually single-layer networks.
The neural network is firstly trained to store a set of patterns in the form s : t
s represents the input vector and t the corresponding output vector.
The neural network is then tested on a set of data to test its “memory” by using it to identify patterns containing incorrect or missing information.

4. Introduction Associative memory can be feedforward or recurrent.
Autoassociative memory cannot hold an infinite number of patterns. Factors that affect this:
Complexity of each pattern
Similarity of input patterns

5. Heteroassociative Memory Architecture

6. Heteroassociative Memory
The inputs and output vectors s and t are different.
The Hebb rule is used as a learning algorithm or calculate the weight matrix by summing the outer products of each input-output pair.
The heteroassociative application algorithm is used to test the algorithm.

7. The Hebb Algorithm
Initialize weights to zero, wij =0, where i = 1, …, n and j = 1, …, m.
For each training case s:t repeat:
xi = si , where i=1,...,n
yi = tj, where j = 1, .., m
Adjust weights wij(new) = wij(old) + xiyj, where i = 1, .., n and j = 1, .., m

8. Exercise
Train a heteroassociative neural network to store the following input and output vectors:
Test the neural network using all input data and the following input vector:

9. Autoassociative Memory
The inputs and output vectors s and t are the same.
The Hebb rule is used as a learning algorithm or calculate the weight matrix by summing the outer products of each input-output pair.
The autoassociative application algorithm is used to test the algorithm.

10. Autoassociative Memory Architecture

11. Exercise
Store the pattern in an autoassociative neural network.
Test the neural network on the following input:

12. The Hopfield Neural Network
Is a recurrent associative memory neural network.
Application algorithm
Exercise: Store the pattern [ ] using a Hopfield neural network. Test the neural network to see whether it is able to correctly identify an input vector with two mistakes in it: [ ]. Note θi=0, for i=1,..,4

13. Bidirectional Associative Memory (BAM)
Consists of two layers, x and y.
Signals are sent back and forth between both layers until an equilibrium is reached.
An equilibrium is reached if the x and y vectors no longer change.
Given an x vector the BAM is able to produce the y vector and vice versa.
Application algorithm

14. BAM Exercise
Store the vectors representing the following patterns using a BAM:
[ ] with the output vector [1 -1]
[ ] with the output vector [-1 1]
θi=0, θj=0 for i = 1,..3 and j=1..2