LEARNING & MEMORY Jaan Aru jaan.aru@gmail.com
Deep neural nets
Learning through weight changes
Feedback connections!
But the biggest difference?
The most famous patient: H.M.
Patient H.M. Hippocampus Medial temporal lobe
Types of memory
MEMENTO
Patient H.M. demonstrates that ... 1) medial temporal lobe is important for memory 2) but it is not necessary for all memory! 3) there is a difference between short-term and long term memory 4) there is a difference between declarative and procedural memory 5) medial temporal lobe is necessary for long term declarative memory
Taxonomy of long-term memory
complementary learning systems (CLS) intelligent agents must possess two learning systems neocortex and hippocampus (MTL) NC gradually acquires structured knowledge HC quickly learns the specifics of individual experiences.
Structured Knowledge Representation System in Neocortex hippocampus
Instance-Based Representation in the Hippocampal System pattern separation: DG is crucial in selecting a distinct neural activity pattern in CA3 for each experience pattern completion : in CA3 reactivation of part of the pattern that was activated during storage can reactivate the rest of the pattern
Open question What constitutes an „individual experience“? = when does hippocampus do its dance? Every moment something changes .. New experience whenever a big change in high level semantic features
Replay of Hippocampal Memories
Replay of Hippocampal Memories Circumventing the Statistics of the Environment All the experiences don’t have the same priority! 1. Experience can have different lengths 2. replay is biased towards rewarding events Novel, significant reweighting experiences shapes neocortical learning For the cortex it is the same
One shot learning?
Can we apply it in AI?? DeepMind: ‘experience replay’ interleaved with ongoing game-play biasing replay towards significant events Prioritized experience replay Its success supports the role of the hippocampus in reweighting experiences
external memory
differentiable neural computer
Taxonomy of long-term memory
ENCODING CONSOLIDATION RETRIEVAL
But what are these stored features? What gets actually stored? ENCODING But what are these stored features? What gets actually stored?
One image, hundred ways to encode it
How you encode it determines how you remember
For remembering, try to go mentally back to the situation where you ENCODING CONSOLIDATION RETRIEVAL For remembering, try to go mentally back to the situation where you learned it
Encoding what? What you encode determines what you remember! What you encode depends on your mood, attention, other thoughts and associations – it is highly variable Adult humans mostly do not encode sensory features, but rather semantic concepts Transfer learning is hard with sensory features, but easy with semantic concepts
Transfer learning Transfer learning is as good as your features Transfer learning is hard with sensory features, but easy with semantic features As two similar things might be very different in sensory space Whereas semantic features are multisensory / amodal – different sensory features can be combined
Summary ANNs learn structured knowledge Learning single experiences is essential Based on hippocampus Prioritized replay to cortex Applying in AI has been a success (DeepMind) One image, hundred ways to encode it Semantic encoding enables transfer