Biologically Inspired Robotics Group,EPFL Associative memory using coupled non-linear oscillators Semester project Final Presentation Vlad TRIFA

Project summary Litterature review Implementation of an associative memory using coupled oscillators and analysis of performance/drawbacks. Mixture with the BIRG model Generalization to complex signals Better control on the capacity Final discussion about relevant issues concerning the performances of both models Conclusion

Associative Memory Animal and human memory works by association. Able to retrieve a stored pattern upon presentation of a partial and noisy representation of an input signal. Many models developed since early 80’s Concepts taken from statisic mechanics and hebbian learning rule turned neural networks into dynamic systems. Useful into understanding dynamics of networks (emergence) but… Lack of biologically plausible mechanisms (coupling, binary,…) Low capacity and performance (Global coupling: N 2 parameters)

Oscillators Oscillating systems are very common in nature and possess very intersting properties. Synchronization Energy efficient mechanism for temporal correlation Many brain processes rely on interaction of oscillators CPG Olfactory and visual cortex Temporal correlation hypothesis and binding problem Information can be stored as phase relationships patterns, where coupled oscillators converge.

Analyzed model Can be found in [Borisyuk, 2001]. Oscillators described by phase, amplitude, and frequency.

The model

Dynamics

Model performance The capacity of this model is not easy to derive, due to the random phase shifts, and to the dynamics of the nonlinear term: We do not know what percentage of overlapping is possible, as memorized patterns can be “overwritten“. Implies that error increases as memory is filled. Robustness due to distributed memorization. But, loss of groups influence strongly the retrieval error.

Discussion The model is interesting as it is based on oscillating systems, thus can be easily implementable on many oscillating systems (PLL, etc…). A very nice methodology that is embedded in the system is proposed in order to decide where to store an input signal is proposed. Random phase shifts ensure some robustness to the system, but too big influence on the performance.

Drawbacks The all-to-all coupling into groups is not efficient computationally and it uses too many oscillators. Due to the explicit input signal embedded in the equations, we can only learn sine functions. The input dimension is annoying. Complexity is increased with no performance increase. The time is reset after each stimulus. We need to present the input in-phase with the oscillators. We cannot learn sequences.

Improvements We want to be able to learn complex signals. Starting from the model in [Righetti et al, 2005], we want to extend the model to form a network.

Discussion Simpler model, more computationally efficient. We gained a much better control on the amount of oscillators to dedicate for a frequency component. We are able to memorize complex signals in a robust and fault tolerant manner, under some constraints. but… Unfortunately, the capacity depends on the complexity of the signals to store. We lost the selection of the storage sites based on phase relationships we had with the previous model.

Future work We need to find a mechanism (embedded in the dynamics) that can select where each component should be stored depending on the signal. It would be very intersting to create links between different clusters activated by the same signal, similar to associative connections forming according to the correlation of neural activity between assemblies, enhancing robustness if attenuated components. Reduce parameters, so we need only to select the amount of oscillators allocated per component.

Conclusion This work should be considered as an attempt to provide insights on how it is possible to store information encoded as a complex signals in a reliable manner, simply by using oscillating systems with local interactions. Our approach is interesting as it uses some concepts that are common in biological neuronal networks such as Oscillating components with local interactions No global external process to supervise the learning procedure

Thank you! References : [Borisyuk, 2001] [Righetti et al., 2005] [Singer, 1995]