Neural Networks

Associative Model of Memory Learning is the process of forming associations between related patterns. Human memory connects items (ideas, sensations, etc.) that are similar, that are contrary, that occur in close proximity, or that occur in close succession (Kohonen, 1987). We cannot remember an event before it happens. Therefore an event happens, some change takes place in our brains so that subsequently we can remember the event. So memory is inherently bound up in the learning process

Associative Model of Memory In a neurobiological context, memory refers to the relatively enduring neural alterations induced by the interactions of an organism with its environment (Tayler, 1986). Without such a change, there can be no memory. Furthermore, for the memory to be useful, it must be accessible to the nervous system so as to influence future behavior. When a particular activity pattern is learned, it is stored in the brain, from which it can be recalled later when required.

Short- and Long-Term Memories Memory may be divided into “short-term” and “long-term” memory, depending on the retention time (Arbib, 1989). Short-term memory refers to a compilation of knowledge representing the “current” state of the environment. Any discrepancies between knowledge stored in short-term memory and a “new” state are used to update the short-term memory. Long-term memory, on the other hand, refers to knowledge stored for a long time or permanently.

Fundamental Property of Associative Memory A fundamental property of the associative memory is that “it maps an output pattern of neural activity onto an input pattern of neural activity”. In particular, during the learning phase, a “key pattern” is presented as stimulus, and the memory transforms it into a “memorized” or “stored pattern”. The storage takes place through specific changes in the synaptic weights of the memory. During the retrieval or recall phase, the memory is presented with a stimulus that is a noisy version or incomplete description of a key pattern originally associated with a stored pattern. Despite imperfections in the stimulus, the associative memory has the capability to recall the stored pattern correctly.

Some Characteristics of the Associative Memory The memory is distributed. Both the stimulus (key) pattern and the response (stored) pattern of an associative memory consist of data vectors. Information is stored in memory by setting up a spatial pattern of neural activities across a large number of neurons. Information contained in a stimulus not only determines its storage location in memory but also an address for its retrieval. Despite the fact that the neurons do not represent reliable and low-noise computing cells, the memory exhibits a high degree of resistance to noise and damage of a diffusive kind. There may be interactions between individual patterns stored. (Otherwise, the memory would have to be exceptionally large for it to accommodate the storage of a large number of patterns in perfect isolation from each other.) There is therefore, the distinct possibility of the memory making errors during the recall process.

Auto- Versus Hetero-associative Memory There are two types of association: Auto-association: A key vector (pattern) is associated with itself in memory. This is most useful for pattern completion where a partial pattern (a pair of eyes) or a noisy pattern (a blurred image) is associated with its complete and accurate representation (the whole face). The input and output signal (data) spaces have the same dimensionality. Hetero-association: A vector is associated with another vector which may have different dimensionality. We may still hope that a noisy or partial input vector will retrieve the complete output vector.

Linear Versus Non-linear Associative Memory An associative memory may also be classified as linear or non-linear, depending upon the model adopted for its neurons. Let the data vectors a and b denote the stimulus (input) and the response (output) of an associative memory, respectively. Linear Associative Memory: Input-output relationship is: b = M a where M is called the “memory matrix”. Nonlinear Associative Memory: Here the input-output relationship is of the form: b = j( M; a ) a where in general, j(. ; .) is a nonlinear function of the memory matrix and the input vector.

Block Diagram of Associative Memory Memory Matrix M Response b Stimulus a

A Simple Network for Holding Associative Memory Inputs Weights Output Neurons Input Outputs