The fussy brain: What makes one option more attractive than another? Steve Fleming and Louise Whiteley

Value Some decisions are about information gathering, where what matters is being accurate. Many everyday decisions are about what is valuable to us now, and in the future… OR ?

Predicting the future

What is a decision? Decision Short- vs Long-term gain Context Risk Information gathering Prior Beliefs Value

Bentham and probability Jeremy Bentham believed that using “felicific calculus” it was possible to work out the best action to take “Nature has placed mankind under the guidance of two sovereign masters; pain and pleasure. It is for them alone to point out what we ought to do, as well as to determine what we shall do” Jeremy Bentham,

Darling’s investment – predicting the future The value of the share can rise or fall… Down 5% Up 10% p = 0.2 p = 0.8

Darling’s investment – predicting the future Down 5% Up 10% p = 0.2 p = 0.8 Expected value of share = weight each outcome by its probability, then add them all up

Darling’s investment – predicting the future Down 5% Up 10% p = 0.2 p = 0.8 Expected value of share = weight each outcome by its probability, then add them all up EV(share) =  outcomes p(outcome) x r(outcome) = (0.2 x 10) + (0.8 x -5) = -2

Darling’s investment – discounting the future Down 20% Up 25% p = 0.2 p = 0.8 Time Value Down 20% Up 15% p = 0.2 p = 0.8 Time Value Share 1 Share 2 In six months… In six weeks… OR

Darling’s investment – discounting the future EV(share) =  outcomes p(outcome) x r(outcome) EV(share) =  outcomes λ x p(outcome) x r(outcome) Time Value Measuring impulsivity… Time Value

How are these values learnt?

What is a decision? Decision Short- vs Long-term gain Context Risk Information gathering Prior Beliefs Value + Learning

Investigating value in the brain 1.Find neurons that signal our preferences 2.Work out how these neurons learn from experience to predict future values 3.See how these neurons are affected by probability 4.See how these neurons are affected by when you get the reward

1. Find neurons that signal our preferences OFC – Orbitofrontal cortex MT LIP OFC

Neurons representing value of choice… vs. We want to know if OFC neurons can keep track of different preferences V(pineapple) V(orange)

orange pineapple during instruction just before reward Preferences in the OFC Different groups of neurons within OFC are associated with different types of reward (e.g. orange vs. pineapple) OFC neurons also know how much reward is on offer - e.g. six apples vs. one piece of cake Padoa-Schioppa & Assad (2006)

2. Work out how neurons predict future values Learn from the past! New value = prediction + new information = difference between prediction and what happened… So: New value = prediction + α(outcome – prediction)

How does the brain predict future values? (outcome – prediction) Schultz et al. (1997) Science Reward unpredicted, reward occurs Reward predicted, reward occurs Reward predicted, reward absent

Changing our predictions with new information Time Read label It’s corked Taste…

New information in the brain… Seymour et al. (2004) Basal ganglia

3. See how these neurons are affected by probability p = 0.8p = 0.2

LIP Platt & Glimcher (1997) How does the brain respond to probability?

Knutson et al. (2005) OFCBasal ganglia EV =  outcomes p(outcome) x r(outcome) How does the brain respond to probability?

Would you like a) £900 now or b) £1000 in one month’s time? 4. See how these neurons are affected by when you get a reward

Short- and long-term gain in the brain Kable & Glimcher (2007) OFC

Brain data help us refine our theory OFC Kable & Glimcher (2007) Two theories: –a) brain region knows about “absolute” value, communicates it to somewhere else which knows about how far away it is in time –b) discounting the future is inherent to our value system

What is involved in making a decision? What happens when things get more complicated…? Decision Short- vs Long-term gain Context Risk Information gathering Prior Beliefs Value

Many decision systems in parallel We’ve been focusing on how the brain learns values from experience, building up habits that can be used again

Many decision systems in parallel Sometimes, we can’t learn habits, and need to look ahead in a more sophisticated way…

Complicated or one-off decisions…

Many decision systems in parallel And sometimes we don’t need to bother - we have innate values attached to things like food and shelter

Bentham again… “the game of push pin is of equal value with poetry” vs. J.S. Mill… “it is better to be … Socrates dissatisfied than a fool satisfied” Complicated value

Many decision systems in parallel In the next talk we hear more about these three systems, about how the brain chooses which system to use, and how this can lead us astray…

Any questions...?

What is a decision? Decision Short- vs Long-term gain Context Risk Information gathering Prior Beliefs Value

Neurons representing value of choice… Padoa-Schioppa & Assad (2006)