1. Corso su Sistemi complessi:
Neural Networks: il modello di Hopfield
Aprile 2009

2. Goal: understand one of most relevant cognitive tasks :
Associative memory
Hopfield model performs elementary tasks (such as pattern recognition and noise reduction): examples of a general paradigm called Computing with Attractors: Items of the memory are represented by stationary firing patterns across the network.
Warning: a big gap between formal model level of description in these associative
memory and the complexity of cortical network dynamics

3. Historical Overview McCulloch-Pitts neuron model 1943
Hebb learning rule
Hodgkin & Huxley neuron model 1952
Hopfield model of associative memory 1982
.. among others

4. Associative Memory: Hopfield model
Item memorized

5. Associative memory: Hopfield Model
how can we Store a set of p patterns
in such a way that when presented with a new pattern , the network responds by producing whichever one of the stored patterns most closely resembles ?

6. Associative Memory: Hopfield model
Initial Intermediate Final
State State State

7. Hopfield Model: How can we store one pattern?
In the model
Hebb Rule
Check that Stability Condition is satisfied :
Moreover, if a number (less than half) of the bits of the starting pattern Si are wrong, they will be overwhelmed in the sum by the majority that are right, and therefore sgn(hi)=xi
Torino Aprile 2009

8. Long-term SYNAPTIC PLASTICITY
Plasticity (activity-dependent changes in synaptic strenght) is widely believed to be the cellular basis of learning and memory.
Donald Hebb 1949
When an axon of cell A is near enough to excite cell B or repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.

9. Hopfield Model: store p patterns?
In the Model
Hebb Rule
Check if Stability condition is satisfied :

10. Hopfield Model: Let’s examine the Stability of pattern n ?
Cross-talk term
Storage capacity
p / N < ac =0.138

11. Hopfield model: the energy function
Energy function of the Hopfield model:
The energy function always decreases (or remains constant) as the system evolves according to its dynamical rule
Attractors (memorized patterns) are at the local minima of the energy surface.
Basins of attraction - catchment areas around each minimum x2
Energy landscape x1

12. Hopfield model: attractors are minima of the energy function
For small enough p, the stored patterns xm are attractors of the dynamics – i.e. local minima of the energy function-
But these are not the only attractors
Additional spurious minima:
mixture states (such as )
Load parameter a= p/N
a<aC
a>aC

13. Phase diagram of Hopfield Model