1. Uncertainty, Neuromodulation and Attention Angela Yu, and Peter Dayan

2. Addressed Issues How to handle uncertainty in changing contexts How many types of uncertainty do we have and what are their neural substrates Which kind of inference is possible and how predictions can be made Is there an experiment for a generalized task to examine the interplay of those different kinds of uncertainties

3. Two Classical Attentional Paradigms Probabilistic cueing: A cue explicitly predicts the location of a subsequent target with a certain probability (termed cue validity). In this task, subjects process the target stimuli more rapidly and accurately on correctly cued trials (valid cue) than on incorrectly cued trials (invalid cue) Observation: VE varies inversely with the level of ACh. (expected uncertainty) Attention-shifting tasks: Rats undergo an UNEXPECTED shift from spatial to visual cues that indicate which route they must take in order to proceed from one end of the maze to the other Observation: Boosting NE with the drug idazoxan in this task accelerates the detection of the cue shift and learning of new cues. (unexpected uncertainty)

4. The Experiment: A generalized task 5- Arrows Trial: target after cue Subject: report target Block1: T-1 trials, blue is relevant, prediction probability:  Block2: from trial T on, blue not relevant any more, for instance red with new 

5. The Subjects’ Task The subjects’ implicit probabilistic task on each trial is to predict the likelihood of the target appearing on left or right given the set of cue stimuli on that trial. This requires to infer the identity (color) of the currently relevant arrow and estimate its validity. In turn, they must accurately detect the infrequent and unsignaled switches in the cue identity (and the context)

6. Mathematical Analysis

7. The Ideal Learner Algorithm During the experiment the subject must decide how to allocate attention to the various cues in order to predict the target stimulus in the current trial, as a function of the probable current context, which depends on the whole history of observations. When the presumed cue appears to predict the target location incorrectly on a particular trial, it is necessary to distinguish between the possibility of a oneoff invalid trial and a contextual change. Bayes Rule

8. The Ideal Learner Algorithm Z t is the normalizing constant for the distribution Iterative method for computing the joint posterior Integration is expensive

9. Approximate Algorithm NE reports the estimated lack of confidence to the particular color that is currently believed to be relevant. ACh reports the estimated invalidity of the color that is assumed to be relevant A trial perceived to be valid always Increases confidence in the current context, as well as estimated cue validity. But when a trial is invalid, subjects have to decide between maintaining the context with increased invalidity or discarding it altogether.

10. Approximate Algorithm Keep a few parameters in mind, the most likely context, the currently pertaining cue validity, the confidence associated with the model, and the number of total trials for the current context. If valid trial, then apply reinforcement, otherwise check whether a contextual change has taken place.

11. Approximate Algorithm In case of valid trail:

12. Approximate Algorithm In case of invalid trial: A contextual change should have taken place iff: 

13. Simulation Results