1. Alan Pickering Department of Psychology a.pickering@gold.ac.uk
Striatal Dopamine (DA) and Learning: Do Category Learning (CL) data constrain computational models?
Alan Pickering
Department of Psychology

2. Overview Classic CL findings and questions
DA, the striatum and learning
Generate simple hypothesis about CL deficits in Parkinson’s Disease
Generate simple biologically-constrained neural net to test hypothesis
Simulate CL data on 2 types of matched CL tasks
Conclusions – why model fails

3. Classic Findings and Questions
Parkinson’s Disease (PD) patients are impaired at CL tasks.
Why?
-What psychological processes are
impaired?
-What brain regions and neuro-
transmitters are involved?

4. Category Learning in Parkinson’s Disease
Weather task: Knowlton et al, 1996

5. Category Learning in Parkinson’s Disease
Main Findings: Knowlton et al, 1996

6. Key Facts PD involves prominent damage to the striatum
CL may (sometimes) involve procedural/habit learning
Striatal structures are part of cortico-striato-pallido-thalamic loops possibly implicated in procedural learning
The striatum is strongly innervated by ascending DA projections

7. Simple Interpretation
CL deficits in PD may arise because of damage to …
loss of ascending DA signals
which compromise the functioning of (parts of) …
the striatum

8. Three Learning Processes Which Might Be DA-Related
Appetitive reinforcement and motivation
DA cell firing increses/decreases provide a positive/negative reinforcement signal which is required for synaptic strengthening/ weakening
“3-factor learning rule”
(e.g., Wickens; Brown et al etc.)

9. Corticostriatal (Medium Spiny Cell) Synapse

10. DA Receptors in Striatum
After Schultz, 1998
DA receptors: Unfilled rectangles
GLU receptors: Filled rectangles

11. DA-Related Processes (cont)
Reward Prediction Error
Midbrain DA neurons increase firing in response to unexpected rewards and decrease firing to nonoccurrence of expected rewards
Firing change= reward prediction error
Schultz, Suri, Dickinson, Dayan etc.

12. DA Cell Recordings: Evidence For Reward Prediction Error
CUE
REWARD

13. DA-Related Processes (cont)
Modulation of Neural Signals
Floresco et al (2001): “DA receptor activity serves to strengthen salient inputs while inhibiting weaker ones”
Also: Nicola & Malenka; J.D.Cohen; Ashby & Cassale; Salum et al; Nakahara; Schultz

14. Evidence For Modulation
Nicola & Malenka, 1997
Recorded effect of DA on response of striatal cells to strong and weak inputs
Strong
Weak

15. Linking 3-Factor Learning & Reward Prediction Error
Striatal Cell
Cue
Reward prediction
Reward predictionerror
Reward
DA Cell
Excitatory
Inhibitory
Reinforcement

16. Simple Working Hypothesis
CL is impaired in PD patients (and other DA-compromised groups) due to “reduced DA function” in striatum (tail of caudate)
The loss of ascending DA input reduces the reinforcing function of the reward prediction error signal innervating the striatum

17. Modelling Biologically-constrained neural net
Data to be simulated taken from Ashby et al (2003)
Data from young and old controls (YC, OC) and PD patients
Study used matched CL tasks: rule-based (RB) and Information Integration (II)
Ashby and colleagues believe these tasks are handled by distinct CL systems

18. Ashby et al: II Task 3 of the 4 dimensions determine categories
Not readily verbalisable
Cat A
Cat B

19. Ashby et al: RB Task
1 dimension (background colour) determines category
Readily verbalisable rule
Cat A
Cat B

20. Ashby et al: Results
Proportion failing to learning to criterion in 200 trials
II Task
RB Task

21. RB Task: Results
Trials to criterion for learners
II Task
RB Task

22. Modelling
Constrained by input and output connections of striatum (caudate)
Learning rule based on known
3-factor form of synaptic plasticity in striatum
Learning rule consistent with reward prediction error properties of DA neurons

23. Connections of Striatum

24. Schematic Model
Reward prediction

25. Model Learning Rule wJK = kR*E*ykout*xJout wJK = kN*E*ykout*xJout
Reward prediction error, E
When reward present, E>0
wJK = kR*E*ykout*xJout
When reward absent, E<0
wJK = kN*E*ykout*xJout

26. Modelling of Reduced DA Function
Loss of DA input to striatum (tail of caudate) modelled 2 ways (with same results):-
a) loss of modifiability of cortico-
striatal weights
b) proportional reduction of reward prediction error strength
Mean proportion of weights modifiable:-
YC OC 0.5 PD 0.2
(with s.d. = 0.15)

27. Modelling Process
Found parameters which gave good fit to YC performance on II task and set DA parameters for PD to produce appropriate level of nonlearners on same task
Varied OC DA values between YC and PD
Looked at fit (with these parameters) to all other data cells esp. RB task

28. Modelling II Task Results
Proportion of non-learners
Trials to criterion (learners) YC PD

29. Modelling II Task Results
Proportion of non-learners
Trials to criterion (learners) YC OC PD

30. Modelling II Task Results*
Proportion of non-learners
Trials to criterion (learners) YC OC PD

31. Model Results II Task
Performance of learners in blocks of 16 trials

32. Modelling RB Task Results
Proportion of non-learners
Trials to criterion (learners) YC OC PD

33. Modelling RB Task Results*
Proportion of non-learners
Trials to criterion (learners) YC OC PD

34. Conclusions & Future Work 1
Simplest realistic model of cortico-striatal learning captures only limited aspects of the CL data
“Bimodal” nature (learn normally vs. fail) of data simulated only under some paramter settings
No intermediate DA parameter settings in old controls which can be both PD-like for II task and YC-like for RB task

35. Conclusions & Future Work 2
Model challenges hypothesis under test: PD (and OC) deficits in some CL tasks seem unlikely to be solely due to reduced DA-related reinforcement in striatum
Findings are consistent with >1 CL system
Future model should add rule system (c.f. Ashby’s COVIS)

36. Alan Pickering CL Refs 2001-
Pickering, A.D., & Gray, J.A. (2001). Dopamine, appetitive reinforcement, and the neuropsychology of human learning: An individual differences approach. In A. Eliasz & A. Angleitner (Eds.), Advances in individual differences research (pp ). Lengerich, Germany: PABST Science Publishers.