1. More on neural similarity

2. Class project: the next steps
I have replied to many, but not yet all, of your project proposals. I’ll have replied to all by end of this week. Sorry!
What is known about this topic?
Find a recent review article (e.g. using “review” term in PubMed, Trends in Cognitive Sciences, Current Opinion in Neurobiology)
Find papers that cite this review (Google Scholar). See what Qs they are looking at.
Try to identify an open question that hasn’t been looked at yet

3. Class project: the next steps
Guiding principles:
Would the imaging provide information over and above what we can get from behaviour?
When people look the same from outside the head (behaviour), but are different inside (imaging)
Maybe imaging can catch early signs of a process which won’t manifest itself in behaviour till later?
Look at representations and mechanisms, not just what lit up
Structure of representations, e.g. via neural similarity
Try to relate neural activation to behaviour

4. Neural similarity: Does it actually matter?
Some good criteria of whether something matters:
Relation to behaviour
Captures an underlying regularity across different people

5. Why do we remember some things but forget others?
Encoding and retrieval
Encoding hypothesis:
Remembered items are better encoded
Retrieval hypothesis:
Remembered items are encoded the same, but the subsequent retrieval process grabs them more successfully

6. An early influential fMRI paper: Wagner et al., 1998
Wagner, A. D., Schacter, D. L., Rotte, M., Koutstaal, W., Maril, A., Dale, A. M., & Buckner, R. L. (1998).
Building memories: remembering and forgetting of verbal experiences as predicted by brain activity.
Science, 281(5380),

7. Wagner et al. (1998)
Greater activation at encoding time for subsequently remembered items.
See also Brewer et al., same issue of Science.
Pictures instead of words

8. Take-home message from Wagner et al. (1998)
The subsequently remembered items are better encoded
The better encoded items elicit more intense activation when they are presented

9. But what about neural pattern similarity, instead of activation intensity?
Xue, G., Dong, Q., Chen, C., Lu, Z., Mumford, J. A., & Poldrack, R. A. (2010).
Greater neural pattern similarity across repetitions is associated with better memory.
Science, 330(6000),

10. Xue et al., 2010 Test the “encoding variability hypothesis”
Greater encoding variability leads to better subsequent memory
Analogy:
To become good at tennis, practice playing against lots of different partners, not just the same opponent over and over

11. Opposite hypothesis: Encoding similarity hypothesis
To remember an item better, each presentation should induce a similar pattern of activation to the previous times
Analogy:
If you want to make a deep dent with a hammer, hit the same spot each time

12. The encoding similarity hypothesis in terms of neural similarity
Each face is presented four times
Hypothesised neural similarity of each presentation of subsequently remembered vs. forgotten faces

13. So, what did they find? Support for encoding similarity hypothesis:
The subsequently remembered faces had greater neural similarity across presentations

14. Is this actually driven by neural pattern similarity?
We already know from Wagner et al. (1998) that simply having greater activation intensity at encoding time gives better memory
Maybe that’s driving the effect here, not greater neural pattern similarity
Control: Calculate pattern similarity, after removing all the voxels that were more active for subsequently remembered items
Result: Remembered items still show greater neural pattern similarity. So, it’s not just activation intensity

15. Wanted: Population-level regularities
Say bridging spiel here

16. Wanted: Population-level regularities in the neural representation of information
Aims:
Find population-level regularities
Challenges:
How can we aggregate pattern- information across different people’s brains?

17. Locations of higher-level functional areas show variability across subjects
From Georg Langs et al. (2011), Inf Process Med Imaging.
“Learning an atlas of a cognitive process in its functional geometry”

18. Different people’s brains: alike at coarse-scale, different at fine-scale
I can align my hand to your hand, and the fingers will match up
But the fingerprints won’t match up

19. Just like literal fingerprints, neural fingerprints seem to be subject-unique
Shinkareva, Mitchell and colleagues (PLoS ONE, 2008):
Attempted both within- and across-subject decoding
Found that “a critical diagnostic portion of the neural representation of the categories and exemplars is still idiosyncratic to individual participants”

20. Hyper-alignment
A common, high-dimensional model of the representational space in human ventral temporal cortex.
Haxby JV, Guntupalli JS, Connolly AC, Halchenko YO, Conroy BR, Gobbini MI, Hanke M, Ramadge PJ.
Neuron Oct 20;72(2):
- High dimensional mapping of different subjects’ voxel spaces onto each other

21. If neural activation patterns don’t capture regularities, then what will?
We need to abstract away from subject-specific neural fingerprints of activation. But abstract towards what?
Previous neural decoding studies:
To do neural decoding, take the subjects’ neural activation and enter it into a decoder
A different approach:
Operate not on neural activation patterns, but instead on the similarity relations between those activation patterns
Neural fingerprints are subject-specific.
In order to find population-level regularities, we need to abstract away from them

22. Hypotheses: What will we gain by operating in similarity space?
Neural similarity will:
Reveal the structure of people’s representations
Reveal population-level regularities in that structure

23. How to decode in similarity-space: a simple solution
Suppose A, B and C are cities, and the similarities represent geographical distances
Which cities correspond to A, B and C?
NYC
San Diego
Boston A B C

24. Across-subject decoding via neural similarity-space
Goal:
Figure out which labels correspond to which stimulus conditions, using only condition-labels from the other subjects
Neural similarity:
Spatial correlation between activation patterns
Raizada & Connolly, J.Cog.Neuro, 2012

25. A simple solution: permute the labels, and see which permutation matches best
Raizada & Connolly, J.Cog.Neuro, 2012

26. How well does this work with actual neural data?
Dataset:
Haxby et al, Science, “Distributed and overlapping representations of faces and objects in ventral temporal cortex.”
Six subjects, eight stimulus categories:
bottles, cats, chairs, faces, houses, scissors, scrambled-pictures, shoes
Animate and inanimate objects
Voxels from ventral-temporal (VT) cortex masks
Lingual, parahippocampal, fusiform, and inferior temporal gyri
Raizada & Connolly, J.Cog.Neuro, 2012

27. How well does this work with actual neural data?
Result: 91.7% correct
44 out of 48 decodings correct
48 = 6x8: 6 subjects, 8 categories per subj
Five subjects: all eight categories correct
One subject: 4 out of 8 correct
Confusions: bottle-scissors, shoe-chair
There are 8 categories, so chance level is 1/8
Chance-level is 12.5% correct
Raizada & Connolly, J.Cog.Neuro, 2012

28. Striking convergence with a proposal from the philosophy of mind
P.M. Churchland (1998) “Conceptual similarity across sensory and neural diversity: The Fodor/Lepore challenge answered”, J. Philosophy, 95, 5-32.