1. Neuromodulation: acetylcholine and memory consolidation Michael E. Hasselmo Trends in Cognitive Sciences – Vol. 3, No. 9, September 1999

2. Why „two-stage model“ of long-term memory storage? - traditional concept of „long-term“ memory can be divided into long – term episodic and semantic memory  not sensitive to hipp. lesions 1,2  suggestion, that these properties of memory might result from a two- stage process for long-term memory formation 4-7 intermediate – term episodic memory  sensitive to lesions of hippocampus and

3. Two-stage model of long-term memory formation (Hasselmo). rapid encoding information from environment intermediate-term episodic representation (excitatory connections between CA3 pyramids are strengthened that are activated by the event 4,7-9 ) { active waking EEG: theta re-activation of the stored representations including spread of activity across the strengthened excitatory connections 6,10 further strengthening of CA3-CA3 and CA3-CA1 connections 4,9 slower strengthening of the excitatory associative synaptic connections between neurons representing these features in neocortex activation of neurons representing the event in EC 5 and association neocortex 11 { quiet waking, SWS EEG: SPWs  Formation of links between these neurons that could be described as semantic or long-term episodic memories in neocortex 4,7,8

4. - Ribot‘s law: the most recently encoded episodic memories are the most sensitive to hippocampal damage (process: „temporally graded amnesia“ 3 ) evidence:  in subjects with damage to hippocampal subregions the temporal extent of retrograde amnesia depends on number of regions involved 2  subjects having been treated with electroconvulsive therapy 1  monkeys trained on object recognition prior to a hippocampal lesion, impairment was - greater, when lesion followed training within 0 – 25 days - minor in monkeys with lesion 75 days after training 12  in rats having been trained to associate a particular testing location (=context) with a tone-shock pairing, the ability to associate was - greater if hippocampal lesions are performed 0 – 2 days after tone-shock pairing - reduced if lesions were applied 5 – 10 days later 13 (location [=context] learning depends on hippocampus; tone-shock pairing depends on amygdala) Neuropsychological evidence for the „two-stage model“(1)  neuropsychological data supporting the „two-stage model“

5. Neuropsychological evidence for the „two-stage model“(2) criticism on concept of hippocampal activity to guide the formation of neocortical associations 14,15 :  human data: hippocampal damage caused a very long-term retrograde amnesia 15 (but: these lesions included subiculum and EC whereas very short-tem retrograde amnesia occurs when damage is restricted to specific hippocampal subfields 2 )  thus, consolidation does not neccessarily result in formation of links in association neocortex, but could instead result in strengthening of representations within hippocampus itself or within the EC  „multiple-trace“ hypothesis 15 possible also to be valid! (= reactivation of memories and formation of additional representations within the hippocampal formation; in this scenario memories are reactivated in CA3 and guide strengthening of associations in CA1 or the EC 4 )  but: also the process of forming multiple traces requires two stages with different dynamics: first for encoding of new sensory information second for reactivation of an old memory to form additional traces  animal studies: extensive lesions: flat gradient of retrograde amnesia partial lesions: temporally graded retrograde amnesia 14  whole process takes place within the hippocampus

6. Computational evidence for the „two-stage model - diverse computational models describing the interaction between hippocampus and neocortex when reactivating representations for consolidation 7,8,16-18 - key feature of these models:  initial storage of an association in an „associative matrix“ (= component of hippocampus) which requires capacity for rapid synaptic modification in the hippocampus (LTP as a physiological correlate? 9 )  then, stored representations from this matrix are repetitively retrieved to activate units in association neocortex (this repetitive reactivation results in gradual strengthening of the connections between neurons in the associative neocortex) = slow process compared to synaptic modification within the hippocampus (experimentally supported by observation that LTP in rat association neocortex is enhanced when stimulation is spaced and repeated over a series of days 20 )

7. Computational evidence for the „two-stage model (2) - why two stages? 1. discussed in terms of modification of existing semantic representations in the neocortex 3 example: network established that codes a range of items of semantic knowledge here: „birds can fly“ („robins are birds that fly… sparrows are birds that fly…“) if such a network were sequentially trained with a new item inconsistent with previous input:  this new information could interfere with the previous learned knowledge and could lead to the belief that all birds swim but: if the new information is stored as an episodic memory in hippocampal formation and later inter- leaved with other examples during training of neocortical representations during consolidation  interleaved learning of multiple different examples is necessary for the formation of efficient representation in a number of different models (e.g. 21,22 ) 2. in models focussing on formation of representations in multi-layer hierachical networks 23,24 alternate phases of dominant feedforward versus dominant feedback connection have been used  examples of computational models of memory consolidation favouring two-stage architecture  new information can be incorporated as an exception in semantic networks „penguins are birds; penguins swim but do not fly“ „robins are birds that fly… sparrows are birds that fly… penguins are birds that swim… eagles are birds that fly“

8. Electrophysiological evidence for the „two-stage model“  hippocampal networks appear to reactivate external stimuli they have responded to during active waking also during quiet waking and slow-wave sleep  experiments demonstrating direction of flow of activity in the hippocampal formation: (experimental evidence: - place cells that fired during waking have an enhanced probability to fire again in a subsequent sleep episode 25 - pairs of place cells that code adjacent positions during active waking show more correlated firing during subsequent SWS 6,10,11 ) - during active waking (theta present): extensive neuronal activity in layer II of the EC (= layer providing input to the hipocampus) 5 - during quiet waking and slow-wave sleep, SPWs selectively activate hippocampal target structures in the EC („deep“ layers V and VI) but not superficial layers 5 - also demonstration of correlation between sharp wave-associating ripples an sleep spindles of prefrontal cortex 26  suggests that hippocampus could be inducing co-activation of neurons in neocortical regions, which could form new crossmodal associations 27

9. Neuromodulation during sleep-wake cycle  the two stage model requires different dynamic states during each stage  during active waking: predominant influence of EC on CA3 Feedback connections arising from CA3 are functional but do not dominate over the feedforward connection  during quiet waking and slow-wave sleep: predominant influence of CA3 on CA1 and EC  how are these different dynamic states modulated?  Hypothesis: modulation mediated by acetylcholine, which shows parallel fluctuations during different stages of waking and sleep

10. Fig. 3. Histogram bars represent levels of neuromodulators (shown at left) in cortical structures relative to levels measured during waking (here normalized to be the same for all neuromodulators). Acetylcholine (Ach) levels are based on microdialysis measurements in hippocampus[32 and 33]. Levels of acetylcholine do not increase as much in the cortex during REM sleep [33]. For norepinephrine (NE), level of unit activity in the locus coeruleus is shown [75 and 85]. Serotonin (5HT) levels are based on microdialysis in the frontal cortex [77].3233 758577 frontal cortex locus coeruleus hippocampus Changes in cortical neuromodulation during waking, slow-wave sleep and REM sleep

11. Neuromodulation during sleep-wake cycle - active waking (AW) - influence of hippoc. on EC is weak during AW (as shown by - low spiking activity in deep layers of EC 5 - small amplitude of EC field potentials arising from hippoc. connections 36,37 ) - defined by behaviour (exploration, sniffing, extensive rearing-up) and EEG (large-amplitude theta 31 in hippoc. and EC) - microdialysis measurements: high levels of ACh release 32,33 - effects of ACh could provide a specific mechanism for weakening the hippocampal feedback during AW (experimental evidence from brain slice or intact animal: suppressive effect of ACh at excit. glut. synapt. transmission at CA3  EC feedback connections 24,38-45 but weaker effects on feedforward connections to hippocampus 42-44,46,47  Fig. 4) - specific effects of ACh (brain slices): - suppresses transmission at excitatory recurrent collaterals in CA3 44,48 - suppresses transmission at Schaffer collaterals 38,39,43,49,50 - suppresses transmission at CA1  Subiculum connection 45

12. Neuromodulation during sleep-wake cycle - active waking (AW) 2 - effects of ACh (in vivo data) - CA3  CA1 connection is weakened (reduced EPSPs) at - local ACh-infusion 40-42 - stimulation of cholinergic innervation from the medial septum 54 - sensory stimulation 46 (effect blocked by atropine 42 )  Taken together, ACh-mediated suppression of excitatory feedback would act to reduce the influence of hippocampus on EC and other cortical areas during active waking But: no total suppression of feedback from hippocampus  Sufficient feedback for retrieval of relevant stored information ACh does not suppress most of feedforward connections to hippocampus - PP input to CA1 much weaker suppressed than CA3  CA1 connection within the same slice 43 - direct EC  CA3 input also less suppressed 48

13. Neuromodulation during sleep-wake cycle - active waking (AW) 3 - effects of ACh on mossy fiber  CA3 synapse - cholinergic suppression also present 57, but weaker than associational inputs to CA3 radiatum at comparable doses 43 - no cholinergic suppression in the outer molecular layer of dentate gyrus 47,58,59, (PP input from lateral EC) - cholinergic suppression does appear - in the middle molecular layer 47 (input from medial EC) - in the inner molecular layer 59 (commissural input from contralatral dentate gyrus) - cholinergic influence on PP input to dentate gyrus is weak (experimentally supported by fact that angular bundle-evoked EPSPs in the dentate gyrus are larger during high ACh-levels of active waking than during low levels of SWS) 36,60 - medial septum-stimulation leads to increases in population spike activity in the dentate gyrus while having no systematic effect on EPSPs 61,62 (possible explanation for this enhanced spiking: GABAergic input from MS; absence of an effect on EPSPs suggests that feedforward input is not suppressed) - postsynaptic effects of ACh (which would enhance the response of neurons to feedforward input 64 ): - depolarization of pyramidal cells 65,66 - suppression of spike frequency adaptation 67,68 - specific cholinergic effects on dentate gyrus:

14. Neuromodulation during sleep-wake cycle - active waking (AW) 4 functional purpose of suppressing the feedback to EC during active waking? - suppression is not total, as recently stored memories from hippocampus are still accessible for retrieval - but: hazard that the strength of connections neccessary for the strong transmission of stored memories back to CA1 and EC would dominate over afferent input - thus: partial cholinergic suppression of excitatory feedback allows cued retrieval without hallucinatory retrieval  this could distort the initial perception of sensory information, causing interference during learning in temporal structures and – if retrieval activity is sufficiently strong – would cause hallucinations (as mediated by the influence of cholinergic antagonists at high doses)

15. Behavioural effects of cholinergic antagonists - during active waking, high levels of ACh could result in a predominant feedforward flow of information, allowing normal interaction with the environment - what happens, when – during waking – reduction of the cholinergic effects is induced by administration of a muscarinic receptor antagonist, e.g. scopolamine? - prediction 1: enhancement of feedback effects in the cortex and this will interfere with the feedforward sensory input to hippocampus  low doses of scop. impair encoding of new information in human subjects but do not effect (even slightly enhance!) retrieval a,b  higher doses would cause stronger feedback effects, which might be the reason for the described hallucinogenic effects of muscarinic antagonists c-e - prediction 2: when cholinergic modulation is blocked after learning of paired associates, this should interfere with learning of subsequent overlapping paired associates  preliminary supporting data (then still unpublished): - learning of related paired associates (A-B, lotion-bottle) - then: injection of scopolamine - new paired associates, either/or not overlapping (A-C, lotion-oil vs. D-E, kitchen-spoon) - then test: greater impairment after learning of overlapping paired associates

16. Neuromodulation during sleep-wake cycle - quiet waking - defined by behaviour (grooming, eating 5,30,33 ) and EEG (Sharp waves in hippocampus 5,37 ) - this decrease would release glutamatergic synapses from cholinergic suppression (experimental evidence: evoked synaptic potentials are much larger in CA1 or in EC during quiet waking than during active waking 60 or when a rat is not being presented with sensory stimulation 46 ) - stronger feedback from CA3 to EC would then allow SPWs to spread from CA3  CA1  EC 4,5,37 - possibly drop of cholinergic inhibition also explains generation of SPWs and - microdialysis measurements: decrease of levels of ACh (~60% of those during active waking 33 ) consistent: finding that „retrohippocampal“ structures (deep layers of EC) show much stronger spiking activity during SPWs of quiet waking than during theta activity of active waking 5

17. Neuromodulation during sleep-wake cycle - slow-wave sleep (1) - defined by EEG pattern (delta waves in neocortex) - microdialysis measurements: decrease of levels of ACh (~33% of those during active waking 32,33 ) - this drop in ACh levels would further release glutamatergic synapses from cholinergic suppression  even stronger excitatory feedback than during quiet waking (= very large increase in the effect of excitatory recurrent connections within CA3 and excitatory feedback connections from CA3 to CA1 and EC) - this drop could therefore underly the increase of SPW activity during SWS 4,37 - the relese of suppression of excitatory transmission could contribute to the greater tendency of cells to fire together during SWS if they fired in the previous waking period 6,10,11 - loss of cholinergic suppression should also enhance the spread of activity in response to stimulation – shown for CA1 and EC (  increased magnitude of evoked synaptic potentials during SWS 36,30 )

18. Neuromodulation during sleep-wake cycle - slow-wave sleep (2) - what functional role could this enhancement of excitatory feedback have?  provide the appropriate dynamics for the formation of additional traces within CA3 and CA1, and could allow the hippocampus to „train“ the EC or association neocortex on the basis of previously encoded associations  not all information for episodic memories is stored in the hippocampus but CA3 region provides a mechanism for linking together disparate information from multiple regions of association neocortex;  during waking, these links can be used for cued retrieval of recently stored information by neocortex, but they should not be able to strongly drive neocortical activity to the level of distorting sensory inputs (hazard of interference leading to hallucinations!)  during SWS, the associative links formed in the hippocampus need to be reactivated and need to influence other brain areas in a strong enough manner to drive the slower modifications of neocortical synapses  neurons coding an association in the hippocampus must be able to drive cells in EC and neocortex without any assistance of sensory input – reduction of cholinergic suppression might provide the opportunity for this strong feedback influence; additionally: physiological activity during SWS has been proposed to be appropriate for modification of synaptic components 70

19. Neuromodulation during sleep-wake cycle - slow-wave sleep (3) - evidence for the importance of SWS for episodic memories:  subjects are better at retrieval of word lists if they learn the list before falling asleep and are tested on retrieval when awakened in the middle of the night, than if they learn the list after some hours of sleep and are tested in the morning 71  if semantic memory is constantly being reshaped during the consolidation process in SWS, then sustained disruption of SWS should impair semantic memory  this suggests greater importance of SWS for memory consolidation because most SWS occurs in the early part of the night  certain types of epileptic syndromes display constant seizure activity during SWS 72,73 which can cause severe impairments of memory function (also delusion, loss of language function) 73

20. Neuromodulation during sleep-wake cycle - REM sleep (1) - defined by EEG pattern (similar to that observed during waking) and rapid eye movements, muscular atonia, muscle twitches 31, ponto-geniculo-occipital waves 33,74 and theta oscillations in the hippocampus - microdialysis measurements: ACh levels in the hippocampus increase to levels above those seen during active waking 32,33  levels of ACh resemble those seen during waking - contrasting: levels of other neuromodulators: - noradrenergic neurons projecting to neocortex decrease in activity to low levels during SWS and show no activity during REM sleep 75,76 - serotonergic neurons projecting to neocortex also decrease activity during REM (recordings from raphe nuclei 75 and microdialysis of serotonin in frontal cortex 77 )  Function of this change in neuromodulatory state? - norepinephrine suppresses feedback excitatory synaptic transmission in the piriform cortex, with a much weaker effect on feedforward transmission 78 - norepinephrine does not suppress feedback within hippocampal formation 80  Hypothesis: - during waking: high levels of ACh and norepinephrine shut down recurrent connections in neocortex - during REM sleep: high levels of ACh in hippocampus might suppress feedback there, the lower levels of ACh and very low levels of norepinephrine in cortex might allow spread of activity in neocortex without strong influence from hippocampus

21. Neuromodulation during sleep-wake cycle - REM sleep (2) Theoretical framework:  waking involves initial encoding of episodic representations in the hippocampus  SWS would allow transmission of episodic representations from hippocampus to EC and association cortex  REM would allow neocortical structures to undergo a re-analysis, in which this episodic information would be re-interpreted in relation to previous semantic representations  REM could represent the development of new feedforward representations for behavior (prediction: REM would therefore also be important for procedural tasks  experiment: evidence, that performance in procedural memory task seems to depend on quantity of REM sleep 81-84 )  process of altering neocortical representations during REM sleep could utilize the information clamped on the neocortex by hippocampal feedback during SWS  indeed: perceptual learning task performance correlated most strongly with SWS during first quarter of the night and REM sleep during the last quarter of the night 83,84

22. Fig. 1.. (A) During active waking, information coded by neocortical structures flows through the entorhinal cortex and dentate gyrus (DG) into hippocampal region CA3 (connections less sensitive to modulation by ACh; thick arrows). Here, synaptic modification forms an intermediate-term representation, binding together different elements of an episodic memory. Connections suppressed by ACh modulation (thin arrows) to region CA1, entorhinal cortex and association cortex are strong enough to mediate immediate retrieval, but do not overwhelm the feedforward connectivity. (B) During quiet waking or slow-wave sleep, memories are reactivated in region CA3 during EEG phenomena termed sharp waves. These waves of activity flow back through region CA1 to entorhinal cortex[5] and neocortex [26]. This will enable the slow consolidation (formation of separate traces) of long-term episodic memory in hippocampal region CA1, entorhinal cortex and association neocortex, and might underlie modification of semantic memory within circuits of association neocortex.526 Two-stage model of long-term memory formation

23. Fig. 1.. (A) During active waking, information coded by neocortical structures flows through the entorhinal cortex and dentate gyrus (DG) into hippocampal region CA3 (connections less sensitive to modulation by ACh; thick arrows). Here, synaptic modification forms an intermediate-term representation, binding together different elements of an episodic memory. Connections suppressed by ACh modulation (thin arrows) to region CA1, entorhinal cortex and association cortex are strong enough to mediate immediate retrieval, but do not overwhelm the feedforward connectivity. (B) During quiet waking or slow-wave sleep, memories are reactivated in region CA3 during EEG phenomena termed sharp waves. These waves of activity flow back through region CA1 to entorhinal cortex[5] and neocortex [26]. This will enable the slow consolidation (formation of separate traces) of long-term episodic memory in hippocampal region CA1, entorhinal cortex and association neocortex, and might underlie modification of semantic memory within circuits of association neocortex.526 Two-stage model of long-term memory formation