Pattern Association

Introduction Pattern association involves associating a new pattern with a stored pattern. Is a “simplified” model of human memory. Types of associative memory: Heteroassociative memory Autoassociative memory Hopfield Net Bidirectional Associative Memory (BAM)

Introduction These are usually single-layer networks. The neural network is firstly trained to store a set of patterns in the form s : t s represents the input vector and t the corresponding output vector. The neural network is then tested on a set of data to test its “memory” by using it to identify patterns containing incorrect or missing information.

Introduction Associative memory can be feedforward or recurrent. Autoassociative memory cannot hold an infinite number of patterns. Factors that affect this: Complexity of each pattern Similarity of input patterns

Heteroassociative Memory Architecture

Heteroassociative Memory The inputs and output vectors s and t are different. The Hebb rule is used as a learning algorithm or calculate the weight matrix by summing the outer products of each input-output pair. The heteroassociative application algorithm is used to test the algorithm.

The Hebb Algorithm Initialize weights to zero, wij =0, where i = 1, …, n and j = 1, …, m. For each training case s:t repeat: xi = si , where i=1,...,n yi = tj, where j = 1, .., m Adjust weights wij(new) = wij(old) + xiyj, where i = 1, .., n and j = 1, .., m

Exercise Train a heteroassociative neural network to store the following input and output vectors: 1 -1 -1 -1 1 -1 1 1 -1 -1 1 -1 -1 -1 -1 1 -1 1 -1 -1 1 1 -1 1 Test the neural network using all input data and the following input vector: 0 1 0 -1

Autoassociative Memory The inputs and output vectors s and t are the same. The Hebb rule is used as a learning algorithm or calculate the weight matrix by summing the outer products of each input-output pair. The autoassociative application algorithm is used to test the algorithm.

Autoassociative Memory Architecture

Exercise Store the pattern 1 1 1 -1 in an autoassociative neural network. Test the neural network on the following input: 1 1 1 -1 -1 1 1 -1 1 -1 1 -1 1 1 -1 -1 1 1 1 1 0 0 1 -1 0 1 0 -1 0 1 1 0

The Hopfield Neural Network Is a recurrent associative memory neural network. Application algorithm Exercise: Store the pattern [1 1 1 0] using a Hopfield neural network. Test the neural network to see whether it is able to correctly identify an input vector with two mistakes in it: [0 1 1 0]. Note θi=0, for i=1,..,4

Bidirectional Associative Memory (BAM) Consists of two layers, x and y. Signals are sent back and forth between both layers until an equilibrium is reached. An equilibrium is reached if the x and y vectors no longer change. Given an x vector the BAM is able to produce the y vector and vice versa. Application algorithm

BAM Exercise Store the vectors representing the following patterns using a BAM: [ 1 -1 1] with the output vector [1 -1] [-1 1 -1] with the output vector [-1 1] θi=0, θj=0 for i = 1,..3 and j=1..2