1. The fussy brain: What makes one option more attractive than another? Steve Fleming and Louise Whiteley

2. Value Some decisions are about information gathering, where what matters is being accurate. Many everyday decisions are about what is valuable to us now, and in the future… OR ?

3. Predicting the future

4. What is a decision? Decision Short- vs Long-term gain Context Risk Information gathering Prior Beliefs Value

5. Bentham and probability Jeremy Bentham believed that using “felicific calculus” it was possible to work out the best action to take “Nature has placed mankind under the guidance of two sovereign masters; pain and pleasure. It is for them alone to point out what we ought to do, as well as to determine what we shall do” Jeremy Bentham, 1748-1832

6. Darling’s investment – predicting the future The value of the share can rise or fall… Down 5% Up 10% p = 0.2 p = 0.8

7. Darling’s investment – predicting the future Down 5% Up 10% p = 0.2 p = 0.8 Expected value of share = weight each outcome by its probability, then add them all up

8. Darling’s investment – predicting the future Down 5% Up 10% p = 0.2 p = 0.8 Expected value of share = weight each outcome by its probability, then add them all up EV(share) =  outcomes p(outcome) x r(outcome) = (0.2 x 10) + (0.8 x -5) = -2

9. Darling’s investment – discounting the future Down 20% Up 25% p = 0.2 p = 0.8 Time Value Down 20% Up 15% p = 0.2 p = 0.8 Time Value Share 1 Share 2 In six months… In six weeks… OR

10. Darling’s investment – discounting the future EV(share) =  outcomes p(outcome) x r(outcome) EV(share) =  outcomes λ x p(outcome) x r(outcome) Time Value Measuring impulsivity… Time Value

11. How are these values learnt?

12. What is a decision? Decision Short- vs Long-term gain Context Risk Information gathering Prior Beliefs Value + Learning

13. Investigating value in the brain 1.Find neurons that signal our preferences 2.Work out how these neurons learn from experience to predict future values 3.See how these neurons are affected by probability 4.See how these neurons are affected by when you get the reward

14. 1. Find neurons that signal our preferences OFC – Orbitofrontal cortex MT LIP OFC

15. Neurons representing value of choice… vs. We want to know if OFC neurons can keep track of different preferences V(pineapple) V(orange)

16. orange pineapple during instruction just before reward Preferences in the OFC Different groups of neurons within OFC are associated with different types of reward (e.g. orange vs. pineapple) OFC neurons also know how much reward is on offer - e.g. six apples vs. one piece of cake Padoa-Schioppa & Assad (2006)

17. 2. Work out how neurons predict future values Learn from the past! New value = prediction + new information = difference between prediction and what happened… So: New value = prediction + α(outcome – prediction)

18. How does the brain predict future values? (outcome – prediction) Schultz et al. (1997) Science Reward unpredicted, reward occurs Reward predicted, reward occurs Reward predicted, reward absent

19. Changing our predictions with new information Time Read label It’s corked Taste…

20. New information in the brain… Seymour et al. (2004) Basal ganglia

21. 3. See how these neurons are affected by probability p = 0.8p = 0.2

22. LIP Platt & Glimcher (1997) How does the brain respond to probability?

23. Knutson et al. (2005) OFCBasal ganglia EV =  outcomes p(outcome) x r(outcome) How does the brain respond to probability?

24. Would you like a) £900 now or b) £1000 in one month’s time? 4. See how these neurons are affected by when you get a reward

25. Short- and long-term gain in the brain Kable & Glimcher (2007) OFC

26. Brain data help us refine our theory OFC Kable & Glimcher (2007) Two theories: –a) brain region knows about “absolute” value, communicates it to somewhere else which knows about how far away it is in time –b) discounting the future is inherent to our value system

27. What is involved in making a decision? What happens when things get more complicated…? Decision Short- vs Long-term gain Context Risk Information gathering Prior Beliefs Value

28. Many decision systems in parallel We’ve been focusing on how the brain learns values from experience, building up habits that can be used again

29. Many decision systems in parallel Sometimes, we can’t learn habits, and need to look ahead in a more sophisticated way…

30. Complicated or one-off decisions…

31. Many decision systems in parallel And sometimes we don’t need to bother - we have innate values attached to things like food and shelter

32. Bentham again… “the game of push pin is of equal value with poetry” vs. J.S. Mill… “it is better to be … Socrates dissatisfied than a fool satisfied” Complicated value

33. Many decision systems in parallel In the next talk we hear more about these three systems, about how the brain chooses which system to use, and how this can lead us astray…

34. Any questions...?

35. What is a decision? Decision Short- vs Long-term gain Context Risk Information gathering Prior Beliefs Value

36. Neurons representing value of choice… Padoa-Schioppa & Assad (2006)