Prefrontal Cortex and Reward Chai Bishop 5/3

I.Pathological Gambling is Linked to Reduced Activation of the Mesolimbic Reward System Jan Reuter, thomas Raedler, Michael Rose, Iver hand, Jan Glascher, christian Buchel Pathological gambling: -defined as gambling that interferes with interpersonal relationships and negatively affects financial and socioeconomic status -defined as gambling that interferes with interpersonal relationships and negatively affects financial and socioeconomic status -a common disorder with a prevalence of 1.6% in adults -a common disorder with a prevalence of 1.6% in adults

Premise of study: – Linkage between Drug Addiction and Pathological Gambling -symptoms -symptoms -mesolimbic reward system -mesolimbic reward system -drug self administration experiments -drug self administration experiments *By analogy to drug addiction, it has been speculated that pathological gambling might also be related to a deficiency of the mesolimbic dopaminergic reward system

Methods: -Using fMRI imaging and a guessing task known to robustly activate the ventral striatum (Knutson, Delgado) - the experimenters studied 12 pathological gamblers (assessed with a questionnaire) and 12 closely matched healthy controls

Results: Fig.1 -there was significantly greater activity during winning than during losing in the ventral striatum which served as verification that the task robustly activated the ventral striatum -fewer voxels (volume pixel, smallest distinguishable boxed shaped part of a 3D image) were activated in pathological gamblers than in controls -only the controls showed additional activation in the ventromedial and ventrolateral prefrontal cortex (VMPFC) (fig. 1c)

Fig.2 -direct comparison of both groups showed significantly lower activation of the right ventral striatum in pathological gamblers than in controls (fig 2b) -pathological gamblers showed significantly weaker activation in the VMPFC (fig 2c)

Additional data: -Regression analysis to correlate signal changes in the ventral striatum with the severity of gambling in each patient showed that the severity of gambling in pathological gamblers showed a significant negative correlation w/ the response in the right ventral striatum (r = -0.53) and the response in the VMPFC (r = -.53) (fig 2a,d) -To ensure this result was not related to depression or smoking habits in some pathological gamblers 2 additional analyses were performed: 1. A categorical analysis comparing non-depressed pathological gamblers to a smaller control group perfectly matched for smoking that showed the same pattern 2. A correlation analysis w/o the depressed patients that confirmed the initial data and actually showed an even stronger relationship between gambling severity and hypoactivation of the right ventral striatum (r = -0.88) and VMPFC (r = -0.67)

Conclusion: - through studying pathological gamblers and controls during a guessing game using fMRI, the experimenters observed a reduction of ventral striatal and ventromedial prefrontal activation in the pathological gamblers that was negatively correlated with gambling severely linking hypoactivation of these areas to disease severity which may represent the neural basis for impaired impulse control in pathological gamblers. - amongst pathological gamblers natural enforcers are not strong enough for dopamine to reach and maintain a homeostatic baseline level.

II. Prediction Error for Free Monetary Reward in the Human Prefrontal Cortex N. Ramnani, R. Elliot, B.S Athwal, and R.E. Passingham Premise of the study: -“The execution of goal-directed behavior is always followed by monitoring for the successful achievement of the goal. For this process to work effectively, a representation of the expected goal must be compared with the outcome.” -“The execution of goal-directed behavior is always followed by monitoring for the successful achievement of the goal. For this process to work effectively, a representation of the expected goal must be compared with the outcome.” -Midbrain dopamine systems of the primate brain send projections to the basal ganglia and widespread regions of the frontal lobes (Ghashghaei and Barbas). These routes are important for conveying reward-related information to frontostriatal circuitry involved in cognitive processing-the firing characteristics of dopamine neurons are determined by the ability of animals to predict rewards in advance of their occurrence and whether the predictions about outcomes are violated or verified.

Primary Aim : -to localize regions of the human brain that were responsive to violations of expectations related to freely delivered monetary rewards Methods: -event related fMRI in 6 human subjects to examine activity related to the failure of expected rewards and the occurrence of unexpected rewards, where outcomes were not contingent on any behavior -event related fMRI in 6 human subjects to examine activity related to the failure of expected rewards and the occurrence of unexpected rewards, where outcomes were not contingent on any behavior --An earlier study Ramnani and Miall (2003) used the same exact conditioned stimuli and reported that when conditioned stimuli predict reward activity is found in the ventral pallidum –on the basis of that study Ramnani et al. used a region of interest approach to test for the possibility that one could predict activity in the same region in this study for conditioned stimuli that predict rewards compared w/conditioned stimuli that predict nonrewards, even if the outcomes are as predicted

Results: Fig 2. -prediction errors evoke activity in 2 separate frontotemporal networks, the anterior prefrontal cortex (a medial part of the orbitofrontal cortex) and in lateral parts of the dorsal prefrontal cortex (Table 2, fig. 2B), -unexpected rewards evoked phasic event related decreases in the anterior prefrontal cortex and parahippocampal gyrus in most subjects but increases in the inferior frontal sulcus and cerebellar cortex

Fig 3. -This study found high activations within its region of interest, the ventral striatum which supports their conclusion that conditioned stimuli that predict monetary reward cause activity changes in the basal ganglia.

Conclusion: - Projections between orbital and polar areas of the prefrontal cortex and areas of the temporal lobes found in primates (Barbas et al.) represent important connections that enable visual information from the temporal lobe ‘object’ processing pathways to reach the prefrontal cortex, nearly all fibers that connect the temporal pole with the prefrontal cortex pass through the uncinate fascicle (Ungerleider) -The ability to form specific object-rewards associations was severely impaired after uncinate fascicle lesions (Parker and Gaffan) -This study has demonstrated that violations of predictions made on the basis of such associations specifically cause activity changes in frontotemporal circuitry. - Due to the experimental design Ramnani et al. were able to separately determine activity changes related to unexpected rewards and unexpected failures of reward in reversal trials, changes were found in the anterior prefrontal cortex for both types of prediction error but the medial part of the orbitofrontal cortex shows activity changes that are specific to the presentation of unexpected reward -The results are consistent with the view that changes in activity in frontotemporal circuits do not simply represent the processing of predictive stimuli and rewards but actively encode the associations between them *Lesions may therefore impair learning because frontotemporal circuitry is required for the processing of error feedback

III. Separate Neural systems Value Immediate Delayed Monetary Rewards Samuel M. McClure, David I. Laibson, George Loewenstein, Jonathan D. Cohen Premise: -Human decision makers are apparently torn between impulsiveness and patience -consumers behave impatiently today but prefer/plan to act patiently in the future Ex. “Someone offered the choice between $10 today and $11 tomorrow might be tempted to choose the immediate option. However, if asked today to choose between $10 in a year and $11 in a year and a day, the same person is likely to prefer the slightly delayed but larger amount.” Ex. “Someone offered the choice between $10 today and $11 tomorrow might be tempted to choose the immediate option. However, if asked today to choose between $10 in a year and $11 in a year and a day, the same person is likely to prefer the slightly delayed but larger amount.”Hypothesis: -The discrepancy between short-run and long –run preferences reflects the differential activation of distinguishable neural systems. - β, short run impatience is driven by the limbic system, which responds preferentially to immediate rewards and is less sensitive to the value of future rewards - δ, long run patience is mediated by the lateral prefrontal cortex and associated structures, which are able to evaluate tradeoffs between abstract rewards, including rewards in the more distant future *Based on a time discounting function referred to as beta-delta preference (Pollack)

Methods: participants made a series of binary choices between smaller/earlier and larger/later monetary amounts while their brains were scanned using fMRI specific amounts ranging from $5-$40 and times of availability ranging from the day of the experiment to 6 weeks later

Results: Fig. 1 -brain areas disproportionately activated by choices involving an immediate outcome ( β areas) include the ventral striatum, medial orbitoprefrontal cortex, and medial prefrontal cortex. As predicted these are the classic limbic structures and closely related paralimbic cortical projections

Fig.2 -voxels that loaded on “all decisions” variable are δ areas, these were not activated by all decision epochs and were not preferentially activated by experimental choices that included an option for a reward today, areas of the LPFC as predicted

Conclusions: -When humans are offered the choice between rewards available at different points in time, the relative values of the options are discounted according to their expected delays until delivery -2 separate systems are involved in such decisions: 1. Parts of the limbic system (associated with the midbrain dopamine system including paralimbic cortex) are preferentially activated by decisions involving immediately available rewards, β 2. Regions of the lateral prefrontal cortex and posterior parietal cortex are engaged uniformly by intertemporal choices irrespective of delay, δ -this analysis shows that the degree of engagement of the these areas is consistent with a key role in future planning

Overall Conclusions on PFC and Reward PFC plays an important role in I. Gambling: negative correlation between PFC activation and addiction severity, implicating that it is not a choice II. Prediction Error: predictions based on monetary reward irrespective of behavior, implicates PFC in stimuli association processes III. Decisions based on monetary reward: irrespective of delay, future planning skills