Does the brain compute confidence estimates about decisions?

EXPERIMENT SETTING AND BEHAVIORAL DATA -Two choice odor mixture categorization - Randomized delays: between entering the odor port and odor delivery; and between entering the choice port and reward delivery (for correct choices) - Reward contingencies deterministic -Decision uncertainty varies with stimulus difficulty due to imperfect perception of stimuli and knowledge of category boundary

NEURAL DATA - Single neuron activity in the orbitofrontal cortex (OFC) was recorded during the delay period between choice and reward - Firing rates of many OFC neurons during this period were modulated by stimulus difficulty: A large fraction of OFC neurons fired more intensely for more difficult stimuli; smaller fraction showed opposite tuning. This is consistent with previous findings that OFC neurons activity correlates with the expected values of reward.

MORE NEURAL DATA Is the firing rate modulated by anything else except stimulus difficulty? Many neurons show different firing rates for correct and error choices, and this difference is observed before the outcome delivery: Again, a large fraction of neurons fired more for incorrect trials; smaller fraction had opposite tuning. Notice that the difference in firing rates is larger for easier stimuli which paradoxical if the firing rate is driven by stimulus difficulty!

SUMMARIZING THE NEURAL DATA Explanation?

DYNAMIC LEARNING ? Maybe reward predictions are based on recent reinforcement history? In this scenario, outcome selectivity would arise because the present trial’s expected outcome is correlated with recent trials’ outcomes. Linear regression: firing rate depends linearly on stimulus, choice, and history of previous outcomes. It turns out that although some OFC neurons do carry information about past trials, including the history of recent outcomes does not improve the model fit significantly. Dynamic learning? No…

INTERPRETATION OF NEGATIVE OUTCOME SELECTIVITY: ERROR ? Maybe the negative outcome selective population of neurons signals error rather than uncertainty? Negative outcome selectivity might arise if, after executing a choice, extra sensory or memory information enters the decision-making circuits and causes realization that an error occurred (even before obtaining feedback). This is tested by separating the highest firing rate trials. It turns out that they are associated not with errors but rather with near-chance performance. Also, this does not explain the V-shaped curves. Signaling error? No…

OUTCOME SELECTIVITY ANALYSIS ? OFC is known to signal outcome expectations. Outcome prediction might arise from a combination of stimulus and side selectivity. If firing rates are driven by stimulus difficulty and choice side: Dashed arrows signify the distance between error and correct choices of a given difficulty. The direction of solid arrows signifies whether error or correct choices have higher rates for a given stimulus. This is not what we see in the data… Outcome selectivity? No…

CONFIDENCE MODEL? The probability of a correct trial outcome could be estimated based on a subjective measure of confidence about the decision. Model: measure of confidence = comparison of the perceived stimulus value and the recalled category boundary. Note that the model (and the subject) only has access to a stimulus sample and not the stimulus type. Errors only occur where distributions overlap, which is smaller than the entire range; so the maximal distance between samples will be smaller for error trials, thus the uncertainty will be higher. For easy stimuli errors are rare because the overlap is small, and the samples in this case will be close (high uncertainty).

CONFIDENCE MODEL CONTINUED Dashed arrows signify the distance between error and correct choices of a given difficulty. The direction of solid arrows signifies whether error or correct choices have higher rates for a given stimulus.

CONFIDENCE MODEL CONTINUED Model Data

ALTERNATIVE WAY TO DEFINE CONFIDENCE: ‘RACE’ MODEL Decision confidence can be calculated from the difference between two decision variables at the time a decision is reached.

CAN CONFIDENCE GUIDE ADAPTIVE BEHAVIOR ? Confidence estimates derived solely from the decision variables in the current trial can provide good estimates of the expected decision outcome across trials. This indicates that confidence estimates are readily available in the rodent brain. Can rats use this information behaviorally? REINITIATION EXPERIMENT

CONCLUSIONS - Firing rates of many neurons in the OFC match closely to the predictions of confidence model. - These firing rates cannot be readily explained by alternative mechanisms. - Thus, confidence estimates are likely to be readily available even in the rodent brain. When a decision is made, the brain not only makes a choice but also generates an evaluation of uncertainty of that decision. - Confidence estimates can drive adaptive behavior. - Computation of subjective confidence may be a core component in decision-making.

FURTHER QUESTIONS - Two different classes of decision model yielded similar results. Other methods for estimating confidence? - OFC has been suggested to generate reward prediction and to signal outcome risk. The data is consistent with both functions. Further experiments will be needed to distinguish between these two alternatives. -Do OFC neurons drive the reinitiation behavior directly, or through other functions (controlling exploration, focusing attention etc.)?