1. BURSTING Hasaeam Cho Bio-NanoStructure Lab of Prof. MC Choi.

2. I will summary the paper …

3. Contents Introduction Bursting midbrain DA neurons – beyond RPE signal Primate studies Rodent studies Beyond phenomenology – how bursts and pauses are generated in DA neurons Afferent inputs controlling bursts in DA neurons Afferent inputs controlling pauses Intrinsic conductances in DA neurons as gates for burst and pause control In vitro dynamic clamp approaches to channel function in DA bursting Conclusions References

4. STEP 1 INTRODUCTION

5. DA midbrain neurons Dopamine (DA) midbrain neurons project to several striatal and cortical target areas Are essentially involved in important brain functions such as Action selection Motor performance Motivation Reward-based learning Working memory Cognition

6. Reward Prediction Error RPE The reward prediction error (RPE) The difference between expected and actually delivered rewards Prediction Error = actual reward – expected reward

7. RPE The reward prediction error (RPE) The difference between expected and actually delivered rewards It’s Positive when the reward > expectation Positive RPE is expressed as a phasic increase of firing above the tonic background rate. It’s Negative when the reward < expectation or not delivered at all (reward omission) Negative RPE is expressed as a transient reduction of firing frequency below background rate or even by a period of complete electrical silence (a pause)

8. RPE Midbrain DA neurons Short (<500 ms) bursts of high- frequency In vivo, occur time-locked (>50-300 ms after) to either unexpected reward delivery or, after learning, sensory cues that predict upcoming reward delivery within the next few seconds. Quantitative analysis revealed the cue-related intra-burst firing frequency was associated with both the expected reward amplitude and expected probability of delivery, and the intra-burst frequency after reward delivery, RPE. Schultz, 2007

9. Reinforcement-learning theory The reinforcement -learning theory Both negative and positive RPEs act as teaching signals. Most likely by changing synaptic weights of glutamatergic cortico-striatal synapses on the most prominent target neurons of midbrain DA neurons Via altering the occupancy and signaling of postsynaptic D1- and D2-type receptors.

10. Excitation, Inhibition, Disinhibition Excitation Excitatory postsynaptic potential (EPSP) Causes depolarization Inhibition Inhibitory postsynaptic potential (IPSP) Causes hyperpolarization Disinhibition A temporary loss of inhibition

11. STEP 2 BURSTING MIDBRAIN DA NEURONS – BEYOND RPE SIGNAL Burst firing of DA neurons under diverse behavioral contexts in awake animals.

12. Primate studies Burst firing was induced by additional cues. As aversive, a blow of cold air to the eyes that triggers a protective blink response In many previous studies have characterized dopamine neurons as a functionally homogeneous population. However, the largest population 40% of DA neurons did not show phasic response to ACS (ACS = air-puff-predictive conditioned stimulus). But, displayed typical responses to reward-predicting cues. An even smaller 10% responses to unexpected airpuffs dorsolateral DA subpopulation showed large and relatively short (100 ms) phasic burst.

13. Primate studies Matsumoto and Hikosaka, 2009

14. Primate studies  The midbrain DA population might be indeed relatively uniform in their responses to unexpected reward and reward-predicting cues,  But displays a topographically organized diversity in response to other salient events. which might not directly instruct RPE-learning but initiate. E.g. orienting responses.

15. Primate studies The recent wave of studies in awake primates have significantly widened the functional context for burst firing among different types of DA neurons as well as within the burst firing itself. However, have remained descriptive and phenomenological Because pharmacological or even optogenetic tools have not yet been used.

16. Rodent studies SFB% : the percentage of spikes within a spike train that were fired within bursts spikes fired in bursts The degree of “burstiness” of DA neuron The start (ISI 160 ms) of burst firing, suggested by Grace and Bunney. With this criterion, bursting became quantifiable. Therefore, alternative burst detection methods have been introduced in recent years relate burst firing and pauses to the stochastic properties of the spike train are independent of the absolute firing frequency. Also, DA recordings in awake rodents, either freely moving or head fixed are increased.

17. Rodent studies In summary, Cue- and reward-associated burst firing is present in some, but never all in recent awake rodent studies even when ruled out by cell type-specific optogenetic tagging. Thus, functional diversity among DA VTA neurons appears to be the norm. Burst firing have also been recorded in behavioral settings not directly related to reward-driven classic or operant conditioning paradigms. Recordings in awake rodents identified a diverse phenomenology of burst firing associated with salient sensory cues, delivery of reward and the control of action sequences. Ventral tegmental area; VTA, substantia nigra (pars compacta); SN(C)

18. STEP 3 BEYOND PHENOMENOLOGY – HOW BURSTS AND PAUSES ARE GENERATED IN DA NEURONS Burst firing of DA neurons by synaptic excitation and GABAergic disinhibition.

19. Beyond phenomenology The behavioral importance of bursts and pauses Can only be appreciated from studies done in awake behaving animals Most underlying mechanisms arise from studies in anesthetized rodents or in vitro brain slice studies. DA neurons exhibit spontaneous bursts, even in anesthetized animal. Much of studies of these spontaneous bursts. Mechanisms in anesthetized preparations and in awake behaving animals No guarantee that they are identical. Some evidence that they might be very similar. Spontaneous bursts are indistinguishable from reward related bursts in duration, firing frequency, and other structural features. The firing rates Of single spike (non-bursting), about 0-10 Hz. During bursts in vivo, rates up to 50 Hz.

20. Afferent inputs controlling bursts in DA neurons Excitatory synaptic input in vivo is necessary for bursting in DA neurons. The synaptic inputs to midbrain DA neurons Penduculopontine nucleus (PPN)/Lateral dorsal tegmentum (LDT) In vivo disinhibition of PPN increased the burst firing of putative DA neurons in the SN and VTA by about 50%. PPN inhibition reduced burst firing by about 50% Subthalamic nucleus (STN) GABA-mediated disinhibition of the STN leads to increased burst firing in a subpopulation of DA neurons in an NMDA-sensitive manner. In contrast to excitatory input, it has been estimated that 70% of all afferents to DA neurons are inhibitory.

21. Zweifel et al., 2009

22. Afferent inputs controlling pauses in vivo, IPSPs generated by GABAergic synaptic inputs are the obvious candidate afferents. IPSPs are capable of delaying action potentials in a time-dependent way. The powerful somatic oscillatory currents drive the single-spike firing.  The cell will fire again unless there is strong inhibition of long duration.  Pauses are only possible with a synchronized increase in inhibition from significant GABAergic input or a decrease in tonic excitatory input.

23. Afferent inputs controlling pauses Cohen et al., 2012

24. STEP 4 INTRINSIC CONDUCTANCES IN DA NEURONS AS GATES FOR BURST AND PAUSE CONTROL Control of burst firing by distinct potassium channels in DA neurons.

25. Gates for burst and pause control Two complementary approaches used to study functional contribution of postsynaptic channels to DA bursting. 1.In vivo extracellular approach is to pharmacologically or molecularly modulate channel activity and then analyze the related changes in burst firing parameters. 2.In vitro intracellular brain slice approach provides a high level of experimental control is to induce bursts in DA neurons by stimulation of afferents and then assess underlying channel mechanisms.

26. Gates for burst and pause control ATP-sensitive potassium (K-ATP) channels Necessary for bursting in vivo for a medial subpopulation of SNC neurons. In vitro, selective opening of K-ATP channels in the presence of tonic NMDA receptor stimulation is sufficient to switch medial SNC neurons to a burst- firing mode. Enable NMDA-mediated bursting of medial DA SN neuron in vitro and in vivo.

27. Schiemann et al., 2012

29. Gates for burst and pause control The bio-physical mechanisms is not yet clear how K-ATP channel opening enables burst firing which in vivo upstream mechanism controls K-ATP channel open probabilities in DA neurons Behavioral data suggest that K-ATP channel-mediated bursting in medial DA SN neurons is important for explorative behavior.

30. STEP 5 IN VITRO DYNAMIC CLAMP APPROACHES TO CHANNEL FUNCTION IN DA BURSTING Realistic burst firing dissected by dynamical clamp techniques.

31. Dynamic Clamp technique A method that uses computer simulations to introduce virtual conductances into real neurons. representing a hybrid between computational models and biological neurons. can be used to study specific parameters of NMDA, AMPA, GABA, and other channel conductances in shaping bursts and pauses in DA neurons.

32. In vitro dynamic clamp approaches to channel function

33. Deister et al., 2009

34. In vitro dynamic clamp approaches to channel function Lobb et al., 2010

35. In vitro dynamic clamp approaches to channel function Dynamic clamp experiments show that Realistic burst firings the NMDA receptor conductance is capable of following the activity of DA neurons at bursting rates. Intrinsic cellular dynamics are an important part of bursting in the dopaminergic neuron. Future studies More complete burst mechanism by incorporating synaptic kinetics with the ion channel kinetics how specific excitatory inputs may differentially affect the firing pattern of DA neurons. more data regarding dendritic ion channel distributions.

36. In vivo study

37. STEP 6 CONCLUSION

38. Highlights STEP 2 Burst firing of DA neurons under diverse behavioral contexts in awake animals. STEP 3 Burst firing of DA neurons by synaptic excitation and GABAergic disinhibition. STEP 4 Control of burst firing by distinct potassium channels in DA neurons. STEP 5 Realistic burst firing dissected by dynamic clamp techniques.

39. CONCLUSION Transient changes of firing in midbrain DA neurons have received great attention. Recent work broadened the behavioral framework of these changes both in non-human primates and rodents. However, up to now most mechanistic studies are currently limited to in vitro preparations. Future in vivo studies are likely to advance mechanistic understanding. With the advent of optogenetic tagging of distinct DA neurons in the midbrain and the potential to selectively drive synaptic inputs, However, intracellular in vivo recordings of DA neurons are needed to improve our mechanistic understanding of burst and pause firing of defined DA neurons in awake behaving animals.

40. REFERENCES Figure 1 Generating bursts (and pauses) in the dopamine midbrain neurons, C. A. Paladini and J. Roeper, Neuroscience (2014) Step 1 Introduction - Figure 1, 2, 3, 4, 5, 6, 7 http://www.medizinische-fakultaet-hd.uni-heidelberg.de/Simon.102039.0.html PPTs of Brain Science Fundamental Figure 2 of Schultz, 2007a Step 2 - Figure 7, 8 Figure 1, 2 of Matsumoto and Hikosaka, 2009 Step 3 – Figure 9 Figure 3 of Cohen et al., 2012 Step 4 – Figure 10, 11 Figure 2, 3 of Schiemann et al., 2012 Figure 12 http://blog.gametize.com/2014/03/7-smart-ways-to-improve-learning-with-gamification-part- 2/boring/http://blog.gametize.com/2014/03/7-smart-ways-to-improve-learning-with-gamification-part- 2/boring/ Step 5 The dynamic clamp – Figure 13, 14, 15, 16 http://rtxi.org/docs/tutorials/2014/12/05/dynamic-clamp/ Figure 1 of Dynamic clamp with StdpC software, Ildiko Kemenes et al., 2011, Nature Protocols Figure 6 of Diester et al., 2009 Figure 1 of Lobb et al., 2010

41. TANK YOU!