Analysis Assumptions -x m - m + c 0.6 -x 0.5 c m - m + 0.4 0.3 0.2 0.1 -10 -8 -6 -4 -2 2 4 6 8 10 Decision variable x varies as a function of t. Choice is made at some time t = signal lag + rt. At the time the choice is made: For a single difficulty level, two distributions, with means +m, -m, and equal sd s set to 1. Choose high reward if decision variable x > -Xc For three difficulty levels, fixed s = 1, means mi (i=1,2,3), assume same Xc for all difficulty levels. Xc can be regarded as a positive increment to the state of the decision variable; high reward is chosen if x > 0 in this case.

Only one diff level Three diff levels Subject’s sensitivity, as defined in theory of signal detectability When response signal delay varies For each subject, fit with function from UM’01

Subject Sensitivity

Optimal “bias” Xc/s based on observed sensitivity data. Observed “bias”, treated as positive offset favoring response associated with high reward. 1.5 -Xc/s 1 Optimal “bias” Xc/s based on observed sensitivity data. 0.5 -10 -8 -6 -4 -2 2 4 6 8 10

Some possible models OU process (l < 0, n0 = 0) following F&H, with reward bias effect implemented as: An alteration in initial condition, subject to decay Optimal time-varying decision boundary outside of the OU process An input ‘current’ starting at presentation of reward signal Noise from reward onset Noise from stimulus onset A constant offset or criterion shift unaffected by time

1. Reward as a change in initial condition, subject to decay Note: Effect of the bias decays away for lambda<0. There is a dip at At t=0, p=1. Feng & Holmes notes

2. Time-varying optimal bias (Outside of OU process) Note: Effect of the bias persists. There is a dip at At t=0, p=1. The smaller the stimulus effect, the larger the bias. The harder the stimulus condition, the later the dip.

3.1. Reward acts as input “current”, stays on from reward signal to end of trial, noise starts at reward onset Reward signal comes t seconds before stimulus Note: Effect of the bias persists There is no dip. At t=0, p<1. They forgot the 2 here. Thoeritically, the dip should happen at 1/lambda* log ( (ac-bk)/(ack^2-bk^2) ), where k=exp(lambda*tau). The t calculated is negative. Feng & Holmes notes 10

3.2. Same as 3.1 but variability is introduced only at stimulus onset Note: Effect of the bias persists There is dip at At t=0, p=1 since all accumulators have no variance.

4. Reward as a constant offset Note: Equivalent to 3.2 for large lt There is a dip at At t=0, p=1

Some possible models OU models (l < 0, n0 = 0) following F&H, with reward bias effect implemented as: An alteration in initial condition, subject to decay Optimal time-varying decision boundary outside of the OU process An input ‘current’ starting at presentation of reward signal Noise from reward onset Noise from stimulus onset A constant offset or criterion shift unaffected by time While none fit perfectly, starting point variability (n0 > 0) would potentially improve 3.2 and 4.

Jay’s favorite mechanistic story (draws from Simen’s model) Participant learns to inject waves of activation that prime response accumulators; waves peak just after stimulus onset and have a residual. Wave is higher for hi rwd response. Stimulus activation accumulates as in LCAM. Response signal initiates added drive to both accumulators equally. First accumulator to fixed threshold initiates the response.