1. Modeling Large Scale Neural Systems Barry Horwitz Brain Imaging & Modeling Section National Institute on Deafness & Other Communication Disorders National Institutes of Health
N1245

2. Collaborators Malle Tagamets Fatima Husain Theresa Long Brent Warner
Julie Fitzer
Jieun Kim
Allen Braun
Yukiko Kikuchi
Mort Mishkin
Feng Rong
Jose Contreras-Vidal
N1245

3. Methods to Understand Neural Basis of Human Cognition
1. Brain lesions & cognitive neuropsychology
2. Pharmacological and genetic studies
3. Electrophysiological recordings in primates (mammals)
4. Transcranial magnetic stimulation
5. Functional neuroimaging
Hemodynamic-metabolic methods (PET, fMRI)
Electric-magnetic methods (EEG, MEG)
N1245
All these data are generally incommensurate with one another.

4. Methods to Understand Neural Basis of Human Cognition
1. Brain lesions & cognitive neuropsychology
2. Electrophysiological and lesion studies in primates (mammals)
3. Pharmacological and genetic studies
4. Transcranial magnetic stimulation
5. Functional neuroimaging
Hemodynamic-metabolic methods (PET, fMRI)
Electric-magnetic methods (EEG, MEG)
N1245
Functional neuroimaging is unique – most of the other techniques permit one to examine only one neural entity at a time. Functional brain imaging allows one to study neural networks directly.

5. Problems with Relating Hemodynamic Data to Underlying Neural Activity
1. Spatial resolution - each PET or fMRI resolvable element contains multiple and diverse neuronal populations.
2. Temporal resolution - temporal resolution of neuronal activity is on the order of milliseconds; PET and fMRI (because of hemodynamic delay) is on the order of seconds; fast transients may be invisible to PET/fMRI.
3. Synaptic vs. neuronal activity - electrical activity comes from cell body firings, PET/fMRI reflect primarily the activity of synapses; excitatory vs. inhibitory.
4. Connectivity - PET/fMRI activity is a mixture of local and afferent synaptic activity.
N1247

6. Large-Scale Neural Modeling
Goal: Construct a large-scale, neurobiologically
realistic neural model that can perform tasks like those studied by PET and fMRI.
Multiple, interconnected brain regions (feedforward and feedback connections).
Each region consists of multiple neuronal units (cortical column).
The basic unit consists of an excitatory-inhibitory pair.
Model can perform multiple tasks (e.g., DMS for shape, control task).
Dynamic behavior of excitatory units in each region matches that observed by primate electrophysiological studies.
Synaptic activity (both excitatory and inhibitory), integrated spatially and temporally, represents rCBF/BOLD.
N1233

7. Uses for Large-Scale Modeling
Understand how cognitive and sensorimotor processes are implemented neurally (forward modeling).
Test experimental design and data analysis methods.
Method to understand the neural substrate for high-level concepts.
Method to combine multimodality information (e.g., fMRI, MEG, lesion).
N1247

8. Perceptual Objects Perceptual Object
Subject to figure-ground separation*
Examples of visual objects
Nameable object (e.g., dog, table)
Delimited pattern
Examples of auditory objects**
Word
Melodic fragment
Definable & delimited environmental sound
N1245
*Kubovy and Van Valkenburg, Cognition, 2001
** Griffiths and Warren, Nat. Rev. Neurosci., 2004

9. From Signal to Percept to Concept
(higher cortical levels)
Percept (image)
(primary cortex)
N1245
Signal
(receptors)
Roy Patterson et al.

10. Neuroanatomy for Visual and Auditory Object Processing
Kass et al.

11. Delayed Matched-to-Sample Tasks
Stimulus 1
Stimulus 2
Delay
Response
ITI, next trial
TIME
Frequency
Shape
Tonal pattern

12. Cerebral Cortex 8: (1998)

13. Regions of the Visual Model
LGN (stimulus)
N1245
V1/V2
Prefrontal V4 IT
(Tagamets & Horwitz, Cerebral Cortex, 1998)

14. Basic Unit of Model and Between-Area Connections
(Cortical column)
60% E
15%
15%
10% I
1. One excitatory (E) and one
inhibitory (I) element per unit.
2. Local connections based on
anatomical data.
3. Total afferent input ~ 10-15%
local connections.
4. Sigmoidal activation rule.

15. PFC IT FS LGN D1 FR D2 Attention V1/V2 V4 Horizontal selective units
Excitatory EE
Inhibitory EI
Horizontal selective
units FS
Corner selective units IT
LGN D1 FR
Vertical selective
units
Vertical selective units D2
Attention
PFC
V1/V2 V4

16. The Sigmoidal Activation Rule
where å + = k I ki E iI t w in ) (
Multiparameter differential equation
Δ = rate of increase
d = rate of decay

17. Working Memory Module (IT-PF component)
PF-s IT
PF-d1
PF-r
(and other areas)
excitatory
inhibitory
ee
ei
PF-d2
s = cue-selective
d1 = delay
d2 = delay+cue
r = response
Modulator of Attention

18. Stimulus and Response
N1245

19. Delay Period
N1245

20. Response to Match
N1245

21. Visual Model

22. Results and Conclusions: Visual Model
Simulations with the Full Model
(% CHANGE WITHIN AREAS)
High Attention to Shape - Low Attention to Degraded Shape
V1/V V IT Prefrontal
+3.1% % % %
Experimental Results (Haxby et al., 1995)
+2.7% % % %
Conclusions
1. Electrical activities in each region match exp. results in primates.
2. PET activities in each region match exp. results in humans.
3. Our hypothesis about how different frontal neuronal populations interact is supported.
4. Our hypothesis about relation between integrated synaptic activity and PET/fMRI data is supported.
5. Hypothesis about role of top-down processing is supported.
N1238

23. NeuroImage 21: (2004)

24. Regions of the Auditory Model
(Husain et al., Neuroimage, 21: , 2004) Ai
Aii
STG/STS D1 D2 FR FS
PFC
MGN

25. Network Diagram of Auditory Model
MGN Up
Selective
Units
Down
Contour
STG/
STS FS
Freq
Time
Freq
Time Up
Selective
Units
Freq
Time
Freq
Time Up
Selective
Units FS FS FS
Freq
Time
Freq
Time
Contour
Selective
Units
STG/
STS
STG/
STS
STG/
STS D1 D1 D1 R R
MGN
Down
Selective
Units D2 D2 D2
Down
Selective
Units
PFC Ai
Attention
Aii
Excitatory E  E
Inhibitory E  I
FS = cue selective
D1 = delay selective
D2 = delay+cue selective
R = response selective

26. Optical imaging - Chinchilla
Simulated & Measured Activity in Auditory Regions: Increasing Temporal Windows of Integration
Temporal window of increases along the Ai-Aii-STG/STS pathway.
--- stimulus Ai
Aii ST
Optical imaging - Chinchilla
(Harrison et al., 2000)
Neural Firing Rate 10
Time-
steps
Time
0.5
1.0
Input Ai
Aii ST
Timesteps
100 timesteps
Intensity

27. Monkey call-related activitywn 40 5
-500
500
1000 10 20 30 40 6
-500
500
1000 10 20 30 40 1
-500
500
1000 10 20 30 40 2 3
-500
500
1000 10 20 30 40 4 30 20 10
-500
500
1000
-500
500
1000 10 20 30 40 7 12
-500
500
1000 10 20 30 40 13 8
-500
500
1000 10 20 30 40 9 10
-500
500
1000 20 30 40 11
-500
500
1000 10 20 30 40 20
-500
500
1000 10 30 40 14
-500
500
1000 10 20 30 40 15 16
-500
500
1000 10 20 30 40 17
-500
500
1000 10 20 30 40 18
-500
500
1000 10 20 30 40 19
-500
500
1000 10 20 30 40 MC 27
-500
500
1000 10 20 30 40 21
-500
500
1000 10 20 30 40 22
-500
500
1000 10 20 30 40 23
-500
500
1000 10 20 30 40 24 25
-500
500
1000 10 20 30 40 26
-500
500
1000 10 20 30 40 10 20 30 40 34
-500
500
1000 28
-500
500
1000 10 20 30 40 29
-500
500
1000 10 20 30 40 31 32
-500
500
1000 10 20 30 40 33
others
complex 40
-500
500
1000 10 20 30 41 35
-500
500
1000 10 20 30 40 36
-500
500
1000 10 20 30 40 37 38
-500
500
1000 10 20 30 40 39 FM 42
-500
500
1000 10 20 30 40 43 44 PT
P < 0.01
XD S
SPK ch# 4 ID#9

28. Simulation of fMRI Experiments
fMRI activity is simulated by spatial and temporal integration of the absolute value of the synaptic activity over 50 msec (which represents the time needed to acquire an fMRI slice).
This time course is then convolved with a Poisson function representing the hemodynamic delay.
The resulting function is then sampled every Tr sec (volume acquisition time) to yield the simulated fMRI activity during each scan series.
(Horwitz and Tagamets, Human Brain Mapp., 1999)

29. BLOCK design paradigm; task blocks interspersed with rest.
Experimental Details
Timeline of a Trial
Stimulus 1
0.35 sec
Delay
1.0 sec
Stimulus 2
0.35 sec
Response
2.0 sec
Next Trial
Task: Subjects discriminate between the two sounds they hear based on whether the two sounds are exactly the same.
BLOCK design paradigm; task blocks interspersed with rest.
Subjects: 12 normal, right-handed, American English speakers (5 women, 7 men).
Training: ½ hour before scanning. Subjects performed at 85% correct.
Behavioral data collected via button presses.
22 axial slices, collected on a 1.5 Tesla GE scanner.
Analysis conducted using the statistical parametric mapping (SPM99) software. Preprocessing, fixed-effects analysis

30. Auditory Stimuli vs. Rest
Tones R L
Tonal Contours

31. Regions of Interest for Auditory Model
z=15
z= 9
Ai Aii (red), PFC(green) ST
z= 3
Left Ai
Aii ST
PFC
z = 9
z = 3
z=6
Right

32. Model vs. Experiment: % Signal Change (Tonal Contours-Tones)
100.0
80.0
Expt_right
Percentage Signal Change
60.0
Model
40.0
20.0
0.0 Ai
Aii ST
PFC

33. Model vs. Experiment: % Signal Change (Tonal Contours-Tones)
100.0
80.0
Expt_right
Percentage Signal Change
60.0
Model
40.0
20.0
0.0 Ai
Aii ST
PFC

34. Network Diagram of Auditory Model
MGN Up
Selective
Units
Down
Contour
STG/
STS FS
Freq
Time
Freq
Time Up
Selective
Units
Freq
Time
Freq
Time Up
Selective
Units FS FS FS
Freq
Time
Freq
Time
Contour
Selective
Units
STG/
STS
STG/
STS
STG/
STS D1 D1 D1 R R
MGN
Down
Selective
Units D2 D2 D2
Down
Selective
Units
PFC Ai
Attention
Aii
Excitatory E  E
Inhibitory E  I
FS = cue selective
D1 = delay selective
D2 = delay+cue selective
R = response selective

35. Contour-selective neuron
Brief Explanation: Contour-selective neuron
Contour-selective neurons are selective to changes in sweep direction occurring within their time window. The duration of this sound is 300 ms. This neuron increased in activity as soon as the FM sweep changed the direction of its frequency. However, if you take a look at the activities of the same neuron to the stimuli with the same components, you can see that this neuron didn’t respond to the mere FM down or up sweep, or the pure tone at the same frequency. To elicit the activity of this neuron, the change of the direction of sweep is necessary.
We cannot explain much about this neuron with the summation of neurons which we can observe in A1.
This type of neuron might play a role in the linkage between the neurons which represent the tonotopic map and the neurons which engage in the higher auditory representations, such as species-specific vocalization processing.

36. General Conclusions
We have constructed a large-scale, neurobiologically realistic model of cortical processing of auditory objects.
Simulated neuronal activities agree with experimental findings.
Simulated fMRI data agree with experimental findings.
Model can account for a number of perceptual grouping observations (human psychophysical performance).
Model predicted existence of a type of neuron found in primate cortex.
Simulated MEG looks like it will agree with experimental MEG data.