1. Wallis, JD Helen Wills Neuroscience Institute UC, Berkeley
Neuronal Mechanisms in Prefrontal Cortex Underlying Adaptive Choice Behavior
Wallis, JD
Helen Wills Neuroscience Institute
UC, Berkeley

2. Background
Role of prefrontal cortex (PFC) in reward-guided choice behavior
2 Questions:
Does PFC encode reward or behavioral sequelae of reward?
Is encoding specific to reward outcome or reflective of abstract value signal?

3. Background
Orbitofrontal cortex (OFC) is a key region in choice behavior
Has functions in
emotions and reward
Thought to regulate planning
behavior associated with
sensitivity to reward and
punishment
Paul Wicks’ brain

4. OFC
Damage to the OFC leaves cognitive abilities intact, but impairs ability to make decisions
The cautionary tale of Elliot
OFC neurons encode expected rewards
Neurons show response to delivery of juice rewards predicted by a visual stimulus
Useful for decision making

5. DLPFC
Reward encoding neurons are also found in the dorsolateral PFC (DLPFC)
Neurons showed a difference in firing rate depending on large/small expected reward
However, are these neurons encoding the value of the reward or a behavioral correlate of the reward (tensed muscles, heightened accuracy)?

6. Experiment 1 Comparison of reward encoding in the DLPFC and the OFC
2 monkeys choose between pictures associated with small/large fruit juice rewards

7. Experiment 1

8. Methods
Each picture associated with delivery of a certain amount of juice
Subjects learn to maximize reward
Reward-picture contingencies reversed after 27 out of 30 successes
Most important neuronal activity after 2nd picture appears

9. Methods
Activity was recorded simultaneously from multiple electrodes in DLPFC and OFC
167 DLPFC neurons
134 OFC neurons

10. OFC Results Spike density histograms from 2 representative OFC neurons
Activity recorded during predictive cue
One neuron showed higher firing rate when the monkey expected 4 drops of juice
Another encoded the reward in parametric fashion
Firing rate not affected by saccade

11. OFC Results
Figure 2

12. DLPFC Results
DLPFC neurons show responses related to both reward and saccade
One neuron discriminated between different amounts of juice only during right saccade
Another encoded reward in parametric fashion (positive?) with a greater increase during left saccade

13. DLPFC Results
Figure 3

14. Statistics
2-way ANOVA on mean firing rate with factors of Reward and Saccade
OFC:
28% significant main effect of Reward
19% significant interaction with Saccade
DLPFC
13% significant main effect of Reward
43% Reward-Saccade interaction

15. Statistics

16. Statistics Some neurons in both areas had similar properties:
27% parametric increase with reward size
15% parametric decrease with reward size
59% encode specific reward

17. ROC Analysis
Sliding receiver operating characteristic (ROC) analysis of the selectivity time-course revealed differences in encoding of reward between OFC and DLPFC
Probability that an independent observer could predict reward based on firing rate
Starting from 500ms prior to 2nd picture an ROC curve was generated from 10ms increments

18. ROC Analysis
Latency at which selectivity appeared was computed as the point at which the curve exceeded 0.6
No difference between OFC (mean 426ms) and DLPFC (mean 467ms)
(t-test = 1, d.f. = 110, P > 0.1)

19. ROC Analysis Selectivity rose more rapidly and peaked earlier in OFC
80 ms earlier on average
Short latency indicates OFC is encoding reward’s value rather than correlated behavioral/cognitive processes

20. ROC Analysis

21. Summary
Neurons sensitive to expected reward are found in both the OFC and DLPFC
OFC neurons encoded only reward while DLPFC neurons encoded reward and saccade
OFC neurons encoded reward earlier than DLPFC

22. Summary
Therefore OFC is first prefrontal region receiving reward information
From basolateral amygdala
Encodes immediate reward –Winstanley, CA et al
From gustatory and olfactory cortices

23. Baxter, MG and Murray, EA The amygdala and reward

24. Summary
DLPFC is where reward value converges with subjects actions: reward choice

25. Experiment 2
OFC response indicates to the motor system which action leads to largest reward
However, decision making needs to be more complex
Reward value is determined by
(Reward – Cost) x P of success
To what extent does OFC encode these variables?

26. Experiment 2
OFC may integrate all variables relevant to decision making to derive an abstract value signal
Neuronal currency
Another study tested whether PFC neurons were capable to responding to multiple parameters

27. Methods
Monkeys were trained to choose between pictures associated with particular rewards
Recordings from OFC, MPFC, and lateral PFC taken simultaneously

28. Methods 3 variables were manipulated:
Probability: Some pictures predict fixed amount of juice on only a certain proportion of trials
Reward: Some pictures were associated with varying amounts of juice
Effort: Monkey had to earn fixed amount of juice by pressing a lever multiple times

29. Results
1/3 of PFC neurons responded parametrically to just 1 parameter
Found in all 3 areas
Some neurons responded to a combination of parameters
Progressive increase from LPFC to OFC to MPFC
16% -> 27% -> 48%

30. Summary
OFC and MPFC combine multiple variables in order to make a decision by deriving abstract value signals
Too difficult to make direct comparisons between all possible choices
Each choice can be valued against a common reference scale (currency)
Example: how many bananas is your car worth?

31. Summary Abstract value signals allow flexibility and novelty
Simplifies the task of the motor system
Allows instantaneous choice
Patients with OFC and MPFC damage show unusual patterns of decision making
A > B , B > C but A < C

32. Conclusions
Damage to the OFC impairs decision making while leaving other cognitive abilities intact
OFC is implicated in reward info processing
Must differentiate between reward and behavioral sequelae of reward
Short latency of neuronal reward-related responses indicates encoding of reward’s value in OFC

33. Conclusions
In contrast, DLPFC encodes reward info in relation to behavioral responses
PFC neurons also encode other variables related to decision making, including probability of success and effort required
OFC and MPFC neurons are responsible for integrating these variables to derive an abstract value signal