Interaction of Sensory and Value Information in Decision-Making Institute for Theoretical Physics and Mathematics Tehran January 16, 2006

Alan Rorie

low level sensory analyzers SENSORY INPUT low level sensory analyzers REWARD HISTORY representation of stimulus/ action value DECISION MECHANISMS Psychologists and economists, however, have long understood that to produce adaptive behavior decision mechanisms must analyze sensory information in the context of the value or utility of available stimuli and actions. The brain for its part must represent the value of competing stimuli and actions and update this representation based upon the outcome of ongoing behavior. Recently we have become interested in studying the neural basis of this representation. However, we first needed a reliable means of quantifying the relative value that an animal ascribes to competing actions. To address this problem we have employed a behavioral phenomenon called matching behavior that I will now briefly describe to you motor output structures ADAPTIVE BEHAVIOR

low level sensory analyzers SENSORY INPUT low level sensory analyzers representation of stimulus/ action value DECISION MECHANISMS Psychologists and economists, however, have long understood that to produce adaptive behavior decision mechanisms must analyze sensory information in the context of the value or utility of available stimuli and actions. The brain for its part must represent the value of competing stimuli and actions and update this representation based upon the outcome of ongoing behavior. Recently we have become interested in studying the neural basis of this representation. However, we first needed a reliable means of quantifying the relative value that an animal ascribes to competing actions. To address this problem we have employed a behavioral phenomenon called matching behavior that I will now briefly describe to you motor output structures REWARD HISTORY ADAPTIVE BEHAVIOR

Motion discrimination task with multiple reward conditions. Monkey must discriminate the direction of the motion. Differs from the matching task because target values are fixed Variable coherences span psychophysical threshold, creating a range of difficulties Imagien this situation … Creates conflict between sensory and reward information Only correct choices are rewarded

“Absolute” and “relative” reward magnitude Differ in absolute reward magnitude Differ in relative reward magnitude

Effect of absolute and relative reward magnitude on behavior Absolute magnitude No effect on choice Relative magnitude Biases choices T1 T2 n=51 T1 T2 T1 T2 T1 = 12 T2 = -14 AVE T1=16 T2= -13 RELATIVE MAGNATUDE ADDITIVE Lack of interaction menas that he is always combinin g the motions and reward information Beta ONE T1 T2 T1 T2

We know from behavior: We ask: Absolute magnitude does not influence choices Relative magnitude influences choices Motion coherence influences choices We ask: Whether, and how, absolute magnitude, relative magnitude, and motion coherence are represented in LIP as the decision unfolds in time?

Area LIP in the Macaque Brain http://www.loni.ucla.edu/data/monkey Sensory-based decisions (Shadlen & Newsome, ‘96, ‘01) Value-based decisions (Sugrue, Corrado & Newsome, ‘04)

LIP neurons are spatially selective LIP RESPONSE FIELD GO!

LIP neurons are spatially selective LIP RESPONSE FIELD GO!

Imagien this situation …

Representation of Absolute Reward Magnitude in LIP

Representation of Absolute Reward Magnitude in LIP

Representation of Absolute Reward Magnitude in LIP Chose In

Representation of Absolute Reward Magnitude in LIP Chose In Chose Out

Representation of Relative Reward Magnitude in LIP

Representation of Relative Reward Magnitude in LIP

Representation of Relative Reward Magnitude in LIP Chose In

Representation of Relative Reward Magnitude in LIP Chose In Chose Out

Representation of Relative Reward Magnitude in LIP Chose In Chose Out

Summary of population activity Absolute Relative Choice Relative Absolute Absolute Choice How can we quantify these dynamics?

Modeling Dynamics

Modeling Dynamics

Modeling Dynamics

Modeling Dynamics

Modeling Dynamics

Modeling Dynamics

Modeling Dynamics

Modeling Dynamics

Conclusions: behavior Absolute reward magnitude does not affect choice Relative reward magnitude biases choice Motion coherence biases choice The biasing effects of relative magnitude and coherence are additive: reward information does not change psychophysical sensitivity to motion coherence (or vice versa).

Conclusions: physiology The representation of sensory and reward information is dynamic; the profile changes dramatically during the course of a trial. The critical decision variables—relative reward magnitude and motion coherence—are present in LIP at the precise time when the decision is being formed. Absolute reward magnitude is represented in LIP even though it does not influence choice behavior. Most single LIP neurons show effects of multiple variables; the representation is multiplexed.

Future Directions: Origins of sensory and reward signals How are sensory and reward signals cast into a common additive currency for guiding decisions? Why is the profile of signals in LIP changing so dramatically throughout the trial? What does this imply for the computational strategy embodied in cortical circuitry?

Indeed there are now no logical (and I believe no insurmountable technical) barriers to the direct study of the entire chain of neural events that lead from the initial central representation of sensory stimuli…to the detection and discrimination processes themselves, and to the formation of general commands for behavioral responses and detailed instructions for their motor execution. V . B . Mountcastle, Handbook of Physiology, 1985

The Optimal Bias 87% -> 88% OPT= 5.9% Gets 2%fewer rewards then if he had done nothing at all

55% 78% 47%