1. fMRI Methods Lecture 12 – Adaptation & classification

2. Neurons Neural computation Neural selectivity
Hierarchy of neural processing

3. Integration of information
Retinal ganglion cell receptive fields
Integrate
V1 neuron receptive field
(Hubel & Wiesel)

4. Grandma cell vs. distributed population
Sparse coding
Narrow selectivity

5. Grandma cell vs. distributed population
Distributed coding
Broad selectivity

6. Vision Components of visual computation: Locations in visual field
Contrast
Orientation
Spatial frequency
Direction of motion
Categories – objects, faces, houses
Neural selectivity!

7. Test selectivity
Change a stimulus attribute in a controlled manner and see how the neural response changes...

8. Neural tuning curves
Particular neurons prefer specific stimulus attributes:
“right” neuron
“left” neuron
0° 45° 90° 135° 180° 225° 270°

9. Similar tuning within a column
Organization of orientation selective neurons in primary visual cortex.
µm wide
Cortex is 2-4 mm thick

10. Resolution Each voxel = 3 mm3 40-50,000 neurons per mm3
~ 1,000,000 per voxel
Fine for distinguishing between gross neural systems in separate brain areas (“modality selectivity”).
How can we tell what intermingled neural populations within a particular area are selective for?

16. Adaptation & Classification

17. Short term memory? Allowing attention to novelty?
Adaptation
Smell
Sight
Touch
Sound
Short term memory? Allowing attention to novelty?

18. Habituation in Aplysia

19. Location of adaptation
Different forms of adaptation take place at different synaptic and cellular locations and have different time-scales…
There are multiple sensory neurons on the siphon, if adaptation is pre-synaptic will it generalize across locations?

20. Location of adaptation
Many possibilities:
Amount of neurotransmitter released.
Amount of receptors available (receptor traffic) at the post synaptic cleft.
Activity dependant ion channel changes (conductance): either pre-synaptic or post-synaptic.
Protein dependant structural synaptic changes (longer term).

21. Membrane conductance change
Tester strength (% contrast)
1.5% adapt
47% adapt
1.5% adapt
47% adapt
Contrast adaptation in V1 neurons – 100’s of milliseconds
Carandini et. al. Science 1997

22. Synapse specific adaptation
Object adaptation in IT neurons – seconds
Miller et. al. Science 1994

23. fMRI Adaptation
First presentation
Repeat

24. Models of fMRI adaptation
Adaptation carry over to higher cortical areas?
Grill-Spector et. al. TICS 2006

25. Models of fMRI adaptation
Different number of stimuli types within a block of 32 stimuli
Grill-Spector et. al. 2001

26. In an event related design
Repeats
Non-repeats

27. Selectivity, adaptation, & invariance
A true test of “high level” selectivity would be invariance across “low level” changes…
Grill-Spector et. al. 2001

28. Selectivity, adaptation, & invariance
Only Individual “face” neurons will adapt…
Generic “face” neurons will adapt…
Every neuron from retina up will adapt…
None will adapt…

30. Yellow: overlap of motor and visual adaptation
fMRI adaptation study
Repeat
Non-repeat
Mirror system areas
Yellow: overlap of motor and visual adaptation
Dinstein et. al. 2007

31. Classification

32. Typical visual area = 300 voxels
Classification
Typical visual area = 300 voxels
fMRI response

33. Classification

34. Direction selective pattern
Strong
Weak

36. Strong
Weak

37. Pause say that everything you just said about movement execution also applies to movement observation

38. How consistent are the patterns?
Trial #1
Trial #1
Trial #1
Trial #2
Trial #2
Trial #2
Trial #3
Trial #3
Trial #3
Trial #4
Trial #4
Trial #4 …. …. ….

39. Response per trial…
Y (bold) = X (model) * b (response amplitudes) + error Y
X (column for each trial)
b for each trial b1 b2 b3 b4 . b1 b2 b3 b4 . b1 b2 b3 b4 1 1 1 1 1 1 1 1 1 * = … … …
Time points

40. Response per trial…
Y (bold) = X (model) * b (response amplitudes) + error Y
X (column for each trial)
b for each trial b1 b2 b3 b4 . b1 b2 b3 b4 . b1 b2 b3 b4 * = … … …
Time points

41. Multivariate pattern classification
Responses of a single voxel to different direction trials
Response Amplitude

42. Multivariate pattern classification
Responses of two voxels to different direction trials
Right
Left
Response Amplitude (voxel 1) Up
Response Amplitude (voxel 2)

43. Multivariate pattern classification
Responses of two voxels to different direction trials
Right
Left
Response Amplitude (voxel 1) Up
Response Amplitude (voxel 2)

44. Multivariate pattern classification
Decode direction using brain pattern
Right
Left
Response Amplitude (voxel 1)
Left? Up
Response Amplitude (voxel 2)

45. Multivariate pattern classification
Decode direction using brain pattern
Right
Left
Response Amplitude (voxel 1) Up
Response Amplitude (voxel 2)

46. Multivariate pattern classification
Response of voxel #1
Response of voxel 2

47. Decode trials (leave one out)…. …. ….

48. Decode direction trials
Area MT
Motor cortex
Decoding accuracy
Chance

49. Decoding orientation in visual areas
Kamitani et. al. Nat. Neurosci 2005

50. Decoding orientation in visual areas

51. Decoding orientation in visual areas

52. Decoding attended stimulus
The orientation that the subject attends is evident in the distributed response pattern of V1 neurons!

53. Classify movement identity
Dinstein et. al. 2008

54. Classify movement identity
Central sulcus
Decoding accuracy
Executed
Observed

55. Classify movement identity
Early visual areas
Decoding accuracy
Executed
Observed

56. Classify movement identity
Anterior IPS
Left
Right
Decoding accuracy
Executed
Observed

57. To the lab!