1. Memory and Learning: Their Nature and Organization in the Brain
James L. McClelland
Stanford University

2. Some features of human learning and memory
Over a lifetime we can learn so much
Names of thousands of people, things
The layout of familiar neighborhoods
Academic domains including math, history, biology…
Many skills such as driving, typing, reading, computer programming, playing chess, …
And yet our memory is far from perfect
We forget
We falsely recollect
We blend information from different experiences

3. Results of Successive Reconstructions

4. Constructive Memory Memory and the Paleontologist metaphor
Fragments stitched together with the aid of plaster, glue … prior knowledge, beliefs, and desires.
Fragments may come from one or many dinosaurs… not necessarily of the same species!
From metaphor to mechanism:
What do we know about learning and memory in the brain that can help explain how we can learn so much, and yet also have the limitations that we do?

5. How Does the Brain Learn?
By adjustments of the strengths of connections among neurons that occur in the course of neural activity
There are about 100 billion neurons in the brain, with up to 100 thousand synapses each, providing an tremendous substrate for learning and memory

6. What is a Memory?
The trace left in the memory system by an experience?
A representation brought back to mind of a prior event or experience?
Note that in some theories, these things are assumed to be one and the same.
Not so in a neural network!
Here, the knowledge is in your connections

7. In a neural network…
The trace left by an experience is a pattern of adjustments to connections among units participating in processing it.
The representation brought back to mind is a pattern of activation which may be similar to that produced by the experience, constructed with the participation of the affected connections.
Such connections are generally assumed also to be affected by many other leaning episodes, so the process of ‘reinstatement’ is always subject to influence from traces of other experiences.

8. Computational Models of Learning in Neural Networks
We don’t really know how the brain succeeds in achieving remarkable feats of learning
But we are getting better and better at training artificial neural networks
We generally use an error-correcting learning algorithm, driven by mismatch between actual and desired output otherwise known as gradient descent
With these models we have captured quite a lot of what we know about the time course and outcome of human learning, so I assume the brain has figured out how to achieve the same result, even though it remains unclear how it actually does it

9. A pattern associator model

10. A Deep Neural Network

11. A Deep Convolutional Neural Network

12. Combining a CNN with an RNN to describe images

13. Auto-associator network

14. A DQN that learns to play games
Model learns to predict the reward value of each possible action given
The visual input based on trial and error learning.
A model combining supervised learning with this approach beats a human expert at Go.

15. Properties of Neural Network Models
They can learn many input-output mappings in the same set of connection weights
Repetition leads to greater strength
Noise gets averaged out over repetitions – the central tendency emerges
The learn to generalize so they can respond to novel inputs
Similar inputs produce similar outputs
That’s good much of the time
But can result in interference when similar inputs must be mapped to different outputs

16. A Neural Network Based Approach to the Neural Basis of Memory
Complementary and Cooperating Brain Systems
Memory performance depends on multiple contributing brain systems.
Each system is a neural network, but may be parameterized differently, to perform complementary functions
Contributions of components to overall task performance depend on their neuro-mechanistic properties.
Components work together so that overall performance may be better than the sum of the independent contributions of the parts.

17. The Complementary Learning Systems Theory (McClelland, McNaughton & O’Reilly, 1995)
Key findings from effects of brain damage
The basic theory
Neuroscience data consistent with the account
Why there should be complementary systems

18. Bi-lateral destruction of hippocampus and related areas produces:
- Profound deficit in forming new arbitrary associations and new episodic memories.
- Preserved general intelligence, knowledge and acquired skills.
- Preserved learning of new skills and item-specific priming.
- Loss of recently learned material w/ preservation of prior knowledge, acquired skills, and remote memory.

19. The Theory: Processing and Learning in Neocortex
An input and a response to it result in activation distributed across many areas in the neocortex.
Small connection weight changes occur as a result, producing
Item-specific effects
Gradual skill acquisition
These small changes are not sufficient to support rapid acquisition of arbitrary new associations.

20. Complementary Learning System in the Hippocampus
Bi-directional connections produce a reduced description of the cortical pattern in the hippocampus.
Large connection weight changes bind bits of reduced description together
Cued recall depends on pattern completion within the hippocampal network
Consolidation occurs through repeated reactivation, leading to cumulation of small changes in cortex.
hippocampus

21. Supporting Neurophysiological Evidence
The necessary pathways exist.
Anatomy and physiology of the hippocampus support its role in fast learning.
Reactivation of hippocampal representations during sleep.

24. Different Learning and Coding Characteristics of Hippocampus and Neocortex
Hippocampus learns quickly to allow one- (or a few)-trial learning of particulars of individual items and events.
Cortex learns slowly to allow sensitivity to overall statistical structure of experience.
Hippocampus uses sparse conjunctive representations to maintain the distinctness of specific items and events to minimize interference.
Cortex uses representations that start out highly overlapping and differentiate gradually to allow:
Generalization by default
Differentiation where necessary

25. Recording was made while animal traversed an eight-arm radial maze.
Examples of neurons found in entorhinal cortex and hippocampal area CA3, consistent with the idea that the hippocampus but not cortex uses sparse conjunctive coding
Recording was made while animal traversed an eight-arm radial maze.

26. Why Are There Complementary Learning Systems?
Discovery of structure requires gradual interleaved learning with dense (overlapping) patterns of activation.
Models based on this idea have led to successful accounts of many aspects of conceptual development and disintegration of conceptual knowledge in neurodegenerative disease (R&M’04).
Rapid learning of new information in such systems leads to catastrophic interference.
Structured knowledge gradually built up is rapidly destroyed.

27. Keil, 1979

28. The Model of Rumelhart (1990)

29. The Training Data:
All propositions true of items at the bottom level of the tree, e.g.:
Robin can {grow, move, fly}

30. Target output for ‘robin can’ input

33. Early
Later
Later Still
E x p e r i e
n c e

34. Emergence of Cognitive Abilities through Interleaved Learning
Distributed representations that capture semantic knowledge and many other cognitive abilities emerge through a gradual interleaved learning process
The progression of learning and the representations formed capture many aspects of cognitive development, and the outcome exhibits the acquired cognitive abilities and skills exhibited by human adults

35. What happens in this system if we try to learn something new?
Such as a Penguin

36. Learning Something New
Used network already trained with eight items and their properties.
Added one new input unit fully connected to the representation layer
Trained the network with the following pairs of items:
penguin-isa living thing-animal-bird
penguin-can grow-move-swim

37. Rapid Learning Leads to Catastrophic Interference

38. How the Hippocampus Solves the Catastrophic Interference Problem
Rapid learning in the hippocampus allows the system to learn new things, while leaving the knowledge in the cortex intact.
If experiences with new item are occasionally replayed to the cortex, interleaved with ongoing experiences with other things, it can be gradually integrated into neocortex without producing catastrophic interference.

39. Avoiding Catastrophic Interference with Interleaved Learning

40. Overview What is “a memory”?
The essence of the connectionist/PDP perspective
Contrasting systems-level approaches to the neural basis of memory
The complementary learning systems approach
McClelland, McNaughton, and O’Reilly, 1995
How the complementary learning systems work together to create ‘episodic’ and ‘semantic’ memory.

41. Effect of Prior Association on Paired-Associate Learning in Control and Amnesic Populations
Base rates

42. Kwok & McClelland Model of Semantic and Episodic Memory
Model includes slow learning cortical system and a fast-learning hippocampal system.
Cortex contains units representing both content and context of an experience.
Semantic memory is gradually built up through repeated presentations of the same content in different contexts.
Formation of new episodic memory depends on hippocampus and the relevant cortical areas, including context.
Loss of hippocampus would prevent initial rapid binding of content and context.
Episodic memories benefit from prior cortical learning when they involve meaningful materials.
Hippocampus
Relation
Cue
Context
Target
Neo-Cortex

43. Simulation Results From KM Model
Base rates in model

44. Summary
Your memories are in your connections; memories are constructed using these traces (and those of other experiences) to constrain the construction process.
Memory performance involves cooperation among brain systems:
Cortical regions that gradually learn to represent content and context
Medial temporal regions that can learn conjunctive associations of cortical patterns rapidly
Working together, these systems allow us to
learn new things rapidly without catastrophic interference
Rely simultaneously on prior knowledge and new learning