1. To discuss this week What is a classifier? What is generalisation?
By an artificial algorithm, and by the brain
What are the basics of neural decoding?
Distributed vs. localised/modular representations
What is fMRI “mind reading”? Should it be called that?
What is a neural representation?

2. Homework
E.mail me some short initial thoughts (e.g. one paragraph) about what you might be interested in doing your class project on.
These initial thoughts are just for discussion, they do not “lock you in” to a given topic!
Two possible ideas:
Analyse a dataset: Have a look at openfmri.org, see if any datasets catch your eye
Think about a possible “grant proposal”-esque idea for designing a new experiment OR
Feel free to suggest something else, and we can chat about it

3. Discussion: “mind-reading”
What is a sense in which fMRI decoding can reasonably be described as "mind reading"?
And what is a sense in which it falls far short of anything deserving the description "mind reading"?

4. It’s difficult for measures inside the head to tell us more than we can get from measures outside the head
Quoting with permission from Meghan Castagno:
“What I didn't like about the second article was that it called all of this ‘mind reading.’ I think that is really dangerous considering we thought the same from differences in cardiac response and Galvanic Skin Response (especially in lie detection).”

5. Neural decoding, as it is usually done: What it does tell us, and what it doesn’t
fMRI is a noisy, indirect measure, but despite that, the signal still contains some stimulus information
What it does not tell us (as it is usually done)
We can extract some information from the fMRI signal. But is this information that the brain cares about?
How is this information mentally represented?
Does it reveal population-level regularities?
Does this information carry any meaning?

6. Is it even possible for a brain image to tell us about mental representations?
Discussion:
What is a mental representation, anyway?

7. What is a representation, anyway?

8. Some representations
“Cat”

9. What is a neural representation?
Low-level sensory areas contain concrete representations: maps (space, frequency etc.)
Tootell et al., Science, 1982

10. Mental representations
Information about the world that’s inside our heads
Perception: how does the information get there?
Action: how does it lead to behaviour?
Cognition: how is that information processed?
Language: how is it represented and communicated?
Representation: how is that information structured?
Perception
Action

11. Traps to avoid: the Cartesian Theater, and homunculi
Philosopher of mind Daniel Dennett
Homunculi
Undischarged homunculi
Image from

12. Our indirect measures of neural information processing

13. Populations of neurons process information together
Information is distributed across large populations of neurons, and across brain areas
There’s no “grandmother cell”: the one single cell that recognizes your grandmother
To really understand the brain, we’d need somehow to read the information from millions of individual neurons at once!

14. Problem 1: Neurons are fast, blood is slow
Neurons can send and receive signals in just a few milliseconds
Important events in the world happen in tens of milliseconds, and neurons can handle them. e.g. duration of formant transitions
The blood-flow response to neural firing takes around six seconds to get going, and around 18 seconds to finish
Hemodynamic Response Function (HRF)

15. Problem 2: Neurons are small, MRI measures are big
100,000,000,000 neurons in the brain
Each neuron around one hundredth of a millimeter
Typical fMRI voxel size: 3mm x 3mm x 5mm
A “voxel” is the 3D version of a pixel
So, in fMRI, we are measuring average activity of literally millions of neurons
Neighbouring neurons might be representing different things. E.g. we might be averaging together signals from /ba/ neurons and /da/ neurons

16. Distributed and overlapping representations

17. Representations that standard fMRI can handle: Localised and segregated
If you do a PubMed search for representations and fMRI,
you find figures like these:
Hand
Lips
Houses
Faces

18. Representations that are more difficult: Distributed and overlapping
/ra/
/la/
/da/
/ba/
/da/
/la/
/ba/
/ra/

19. Distinct representations may produce same overall activation
/ba/-sensitive
population of neurons
Speech area
Subtraction:
/ba/ minus /da/ = zero
/da/-sensitive
population of neurons

20. Distinct but overlapping representations: same average activation, but different local patterns
Stimuli A and B activate the same neural population, both activating shared neurons to differing degrees,
so they elicit different activation patterns
Stimulus B
Stimulus A
Average local activation is the same for both stimuli

21. Gaussian spatial smoothing to improve signal-to-noise
Distinction between representations is lost: After smoothing, there’s no difference left
Boynton
Kriegeskorte
Haxby
Raizada
/ra/
/la/
Gaussian spatial smoothing to improve signal-to-noise
Stimulus A
activation
pattern
Stimulus B
Average activation same, but spatial patterns different
Smoothed local average activation ends up the same

22. Next week Exploring in more detail, using Matlab:
Hemodynamic response function (HRF)
Convolving a time-course with an HRF
Structure of a design matrix
Computational implementation of a simple linear classifier