Volume 95, Issue 6, Pages 1395-1405.e3 (September 2017) Dopamine Neurons Respond to Errors in the Prediction of Sensory Features of Expected Rewards  Yuji K. Takahashi, Hannah M. Batchelor, Bing Liu, Akash Khanna, Marisela Morales, Geoffrey Schoenbaum  Neuron  Volume 95, Issue 6, Pages 1395-1405.e3 (September 2017) DOI: 10.1016/j.neuron.2017.08.025 Copyright © 2017 Terms and Conditions

Figure 1 Identification and Waveform Features of Putative Dopamine Neurons (A) Result of cluster analysis based on the half-time of the spike duration and the ratio comparing the amplitude of the first positive and negative waveform segments ((n – p)/(n + p)). Data on the left show VTA neurons (n = 473) from the experimental group, plotted as reward-responsive (filled black circles) and nonresponsive dopamine neurons (filled gray circles), and neurons that classified with other clusters, no clusters, or more than one cluster (open circles). Data on the right show VTA neurons (n = 556) from rats that received AAV1-Flex-taCasp3-TEVp infusions, plotted as dopamine (filled red circles) or non-dopamine neurons (open red circles) based on whether they were assigned to the dopamine cluster from the experimental data. Drawings on right show electrode tracks in experimental (n = 8, gray) and Casp3 groups (n = 5, red) and representative coronal brain slices showing unilateral loss of TH+ neurons in VTA in a Casp3-infused rat (left hemisphere; average loss versus intact side, 76.5%; range 65%–85% over all 5 rats). (B) Bar graph indicating average amplitude ratio of putative dopamine neurons, non-dopamine neurons recorded from experimental rats, and VTA neurons recorded from Casp3-infused rats. (C) Bar graph indicating average half-duration of putative dopamine neurons, non-dopamine neurons recorded from experimental rats, and VTA neurons recorded in Casp3-infused rats. Error bars, SEM. The average amplitude ratio and spike duration of the neurons recorded in the Casp3 rats did not differ from neurons in non-dopamine clusters recorded in the experimental rats (amplitude ratio, t test, t946 = 0.99, p = 0.32; spike duration, t test, t946 = 1.40, p = 0.16). Indeed, only six of the neurons recorded in these rats (filled red circles in A) classified as the putative dopamine neurons (1% versus 17%, X2 = 87.7, p < 0.0000001). This amounted to an average of 0.06 putative dopamine neurons per session in the Casp3 group versus 0.79 neurons per session observed in the experimental rats (t test, p < 0.01). The prevalence of the non-dopamine neurons actually increased from 3.84 neurons per session in the experimental rats to 5.44 neurons per session in the Casp3 group (t test, p < 0.01), perhaps due to network effects of the loss of the TH+ neurons. Neuron 2017 95, 1395-1405.e3DOI: (10.1016/j.neuron.2017.08.025) Copyright © 2017 Terms and Conditions

Figure 2 Task Design and Behavior (A) Picture of apparatus used in the task, showing the odor port (∼2.5 cm diameter) and two fluid wells. (B) Line deflections indicate the time course of stimuli (odors and reward) presented to the animal on each trial. Dashed lines show when a reward was omitted, and solid lines show when reward was delivered. At the start of each recording session, one well was randomly designated to deliver the big reward, which consisted of three drops of flavored milk (chocolate or vanilla). One drop of the other flavored milk was delivered in the other well (block 1). In the second and fourth blocks, number of drops delivered in the two wells was switched without changing the flavors (value shift). In the third and fifth blocks, the flavors delivered in the two wells were switched without changing the number of drops (identity shift). (C) Chocolate- and vanilla-flavored milk was equally preferred in 2 min consumption tests conducted at the end of some sessions. Gray lines indicate data from individual rats. (D and E) Choice rates in last 15 trials before and first 40 trials after a switch in reward number (D) or flavor (E). y axis indicates percent choice of side designated as big reward after block switch. Inset bar graphs show average choice rates in the last 15 before and first 40 trials after the switch. (F) Reaction times on the last ten forced-choice trials in response to big and small amounts of each flavor. (G) Percentage correct on the last ten forced-choice trials in response to big and small amounts of each flavor. (H) Number of licks in 500 ms after first drop of reward on the last 10 trials in response to big and small amounts of each flavor. B, big; S, small. Error bars, SEM. Neuron 2017 95, 1395-1405.e3DOI: (10.1016/j.neuron.2017.08.025) Copyright © 2017 Terms and Conditions

Figure 3 Changes in Reward-Evoked Activity of Reward-Responsive Dopamine Neurons (n = 60) to Changes in Reward Number (A and B) Average firing on first five (red) and last five (blue) trials after a shift in reward number. (A) shows firing to the three drops of the big reward when the small reward had been expected, and (B) shows firing to the single drop of the small reward when the big reward had been expected. Big-B, three drops of reward B; small-A, one drop of reward A. (C) Distributions of difference scores comparing firing to first (left), second (middle), and third drops (right) of the big reward in the first five versus last five trials in a number shift block. (D) Distributions of difference scores comparing firing to the single drop of the small reward (left), and omissions of second (middle) and third drops (right) of the big reward in the first five versus last five trials in a number shift block. Difference scores were computed from the average firing rate of each neuron. The numbers in each panel indicate results of Wilcoxon signed-rank test (p) and the average difference score (u). (E) Changes in average firing before and after reward number shift. Light gray, black, and dark gray solid lines indicate firing at the time of the first, second, and third drop of reward on big trials. Light gray, black, and dark gray dashed lines indicate firing at the time of the small reward, and omissions of the second and third drops thereafter on small trials. Error bars, SEM. (F) Correlation between difference scores representing changes in firing to delivery and omission of the second drop of the big reward. Neuron 2017 95, 1395-1405.e3DOI: (10.1016/j.neuron.2017.08.025) Copyright © 2017 Terms and Conditions

Figure 4 Changes in Reward-Evoked Activity of Reward-Responsive Dopamine Neurons (n = 48) to Changes in Reward Identity (A and B) Average firing on last five trials before (green) and first five trials after (red) a shift in reward identity for the big (A) and small (B) reward. Big-A, three drops of reward A; Big-B, three drops of reward B; small-A, one drop of reward A; small-B, one drop of reward B. (C) Distributions of difference scores comparing firing to first (left), second (middle), and third drops (right) of the big reward in the last five versus first five trials before and after identity shift. (D) Distributions of difference scores comparing firing to the single drop of the small reward (left), and omissions of second (middle) and third drops (right) of the big reward in the last five versus first five small trials before and after an identity shift. Difference scores were computed from the average firing rate of each neuron. The numbers in each panel indicate results of Wilcoxon signed-rank test (p) and the average difference score (u). (E) Changes in average firing before and after reward identity shift. Black line indicates average firing at the time of the first and second drops of the big reward and the small reward. Gray dashed line indicates average firing 0.5 and 1.0 s after small reward. Error bars, SEM. (F) Correlation between changes in firing to shifts in reward identity, shown here, and changes in firing to delivery (blue dots) or omission (red dots) of the second drop of the big reward, shown in Figure 3. Neuron 2017 95, 1395-1405.e3DOI: (10.1016/j.neuron.2017.08.025) Copyright © 2017 Terms and Conditions

Figure 5 Schematic Illustrating Different Ways that Error Signals Might Appear in Response to Changes in Reward Number and Identity in the Behavioral Design Reproduced from Figure 2, Depending on What Information Dopaminergic Errors Reflect (A) If dopamine neuron firing reflects errors in cached value only, then the conventional prediction would be increased firing only to the second drop of the big reward and decreased firing only on omission of this second drop of reward, since these are the only places in the design where cached-value predictions are clearly violated. (B) If dopamine neuron firing reflects errors in cached value plus a novelty bonus or salience, then the prediction is for the same cached-value errors shown in (A) on number shifts, plus increased firing to each drop of reward after an identity shift, assuming the unexpected flavor of each drop is salient or novel. (C) If dopamine neuron firing reflects errors in cached value, based on both the odor cues and also the sensory features of the first drop of each reward, then the prediction is for the same cached-value errors shown in (A) for number shifts, plus a mixture of increased and decreased firing to identity shifts dependent on cached value accrued by the first drop’s flavor in the prior block. For example, when chocolate was the small reward previously and becomes the large reward, one would expect the first drop to evoke decreased firing because it would be less valuable than expected (since its flavor predicts no more drops), followed by increased firing to subsequent drops because they would be unexpected. This again assumes the rat is using the flavor of the first drop to make predictions about subsequent drops. Note this does not apply if the two wells deliver similar amounts of different reward, and as illustrated in Figure S2, dopamine neurons also exhibited errors in response to identity shifts under these conditions. (D) Finally, if dopamine neuron firing reflects errors in the prediction of sensory information or features, either instead of or in addition to cached-value errors, then the predictions are for increased firing to unexpected events generally and decreased firing to their omission. See text for full description (green, positive errors; red, negative errors; red boxes highlight only place of divergence of predictions between C and D). Neuron 2017 95, 1395-1405.e3DOI: (10.1016/j.neuron.2017.08.025) Copyright © 2017 Terms and Conditions