1. Bonaiuto, jimmy@vis.caltech.edu Synthetic Brain Imaging: A Computational Interface Between Electrophysiology and Neuroimaging James Bonaiuto (work with Michael Arbib, USC) Andersen Laboratory California Institute of Technology

2. Bonaiuto, jimmy@vis.caltech.edu The Challenge of Data Integration and Interpretation  Two of the major problems facing cognitive neuroscientists are data interpretation and comparison across modalities  Neuroimaging studies are typically designed to test some conceptual model of the interactions between the brain regions involved in a task.  The results are usually evaluated using an ad-hoc verbal analysis and compared to neurophysiological data. ?≈?≈ Rizzolatti et al (1995) Buccino et al (2004)

3. Bonaiuto, jimmy@vis.caltech.edu From Neural Activity to BOLD Response Neural activitySignallingVascular response Vascular tone (reactivity) Autoregulation Metabolic signalling BOLD signal glia arteriole venule B 0 field Synaptic signalling Blood flow, oxygenation and volume BOLD signal ≠ Neuron firing (figure by Richard Wise)

4. Bonaiuto, jimmy@vis.caltech.edu Synthetic Brain Imaging  Synthetic Brain Imaging was developed to address the disconnect between experimental modalities  Computational models based on neurophysiological data are used to generate simulated neuroimaging signals:  regional cerebral blood flow (rCBF)  blood oxygen level-dependent (BOLD) responses  This technique has been used to successfully predict:  A projection from the PFC to the anterior intraparietal region AIP (Arbib, Fagg & Grafton, 2002)  Populations of sound contour-selective cells in secondary auditory cortex (Husain et al., 2004)

5. Bonaiuto, jimmy@vis.caltech.edu  It depends on the data the model is intended to address  Early synthetic brain imaging approaches used firing rate models (Arbib et al., 1995; Horwitz et al,. 1998)  We require the simplest possible model that can produce a wide range of firing patterns What is the Appropriate Neural Model? (Izhikevich, 2004)

6. Bonaiuto, jimmy@vis.caltech.edu Neurophysiology and fMRI  Under some conditions spiking activity and CBF dissociate  In general, local field potential (LFP) is a better predictor of BOLD than spiking activity  LFP most directly reflects synaptic rather than spiking activity  We therefore require a model with realistic synaptic activity (Goense & Logothetis, 2008) (Lauritzen et al., 2003)

7. Bonaiuto, jimmy@vis.caltech.edu  We use the sum of synaptic conductances as the measure of neural activity  Several mechanisms coexist to regulate blood flow  neuron-astrocyte pathway (Koehler et al., 2006)  vasomotor GABAergic interneurons (Cauli et al., 2004)  nitric oxide diffusion (Metea & Newman, 2006)  We use a generic blood flow-inducing signal that subsumes neurogenic and diffusive components (Friston et al., 2000)  A linear function of neural activity with signal decay and autoregulatory feedback from blood flow Neurovascular Coupling  =gain parameter u 0 =baseline synaptic activity  b =decay time constant f in =blood flow  f =feedback time constant

8. Bonaiuto, jimmy@vis.caltech.edu Vascular Signal Generation: Balloon Model We use Friston & Buxton’s balloon model to simulate the vascular response to neural activity and generate simulated PET or fMRI signals

9. Bonaiuto, jimmy@vis.caltech.edu A New Synthetic Brain Imaging Model  Izhikevich neurons with realistic synaptic dynamics and noise  Total synaptic conductance for all synapses in a voxel used to generate to a generic blood flow- inducing signal  Use normalized blood flow-inducing signal as input to the Balloon model

10. Bonaiuto, jimmy@vis.caltech.edu Basic Network Architecture Pyramidal Neuron Firing Rate (Hz) { Response latency – Populations of pyramidal neurons and inhibitory interneurons – Center-surround connectivity implements winner-take-all dynamic – Depending on the input, there can be a considerable latency before the network settles on a stable winner Membrane Potential (mV) 0 100 1 Neuron 0 2.0 Time (s) -80 30 0 -50

11. Bonaiuto, jimmy@vis.caltech.edu Example 1: Random Dot Motion Discrimination  The random dot motion direction discrimination task is commonly used to study perceptual decision-making  Task: saccade in the net movement direction of a field of randomly moving dots  Stimulus coherence: percentage of dots moving in the same direction  This task is useful in demonstrating the power of synthetic brain imaging because  A well-defined network of brain regions is involved (MT, LIP, FEF)  There exists neural recording, microstimulation, behavioral and imaging data using the task in humans and non-human primates

12. Bonaiuto, jimmy@vis.caltech.edu Model Derivation  We used three connected WTA networks to simulate the MT- LIP-FEF network  Neural parameters were set using values from experimental data  Network parameters were set using a genetic algorithm that used the model’s fit to neural recording and behavioral data as the fitness function (Gold & Shadlen, 2007)

13. Bonaiuto, jimmy@vis.caltech.edu Results: Neural Activity Stimulus Coherence 3.2%12.8%51.2%  Pyramidal neurons in LIP converge on a population code centered on the chosen saccade direction  Response time was interpreted as the time taken for max firing rate in FEF to reach 100Hz (Hanes & Schall, 1996)

14. Bonaiuto, jimmy@vis.caltech.edu Results: Behavioral Measures  Response time and accuracy were fit to the same psychometric and chronometric functions used to analyze human data  Fitted parameters were within the range of human performance Model PerformanceHuman Behavioral Data (Palmer et al., 2005)

15. Bonaiuto, jimmy@vis.caltech.edu Results: Microstimulation Simulations Simulation Results Monkey Data Control MT Stim LIP Stim  MT microstimulation biased decision process and reaction time  LIP microstimulation had the same effect, but to a lesser extent  The same results are found in monkey microstimulation experiments (Hanks et al., 2006)

16. Bonaiuto, jimmy@vis.caltech.edu Results: Synthetic fMRI MT LIP FEF Synthetic fMRI Human fMRI (Rees, Friston & Koch 2000)  The model replicated human fMRI data that only found a positive correlation between BOLD response and stimulus coherence in MT  In the model this is because intraregional processing (WTA) dominates LIP and FEF activity and is roughly the same at each coherence level

17. Bonaiuto, jimmy@vis.caltech.edu Example 1: Summary  A basic neural microcircuit was connected in a network based on anatomical considerations  Neural parameters were set using values from experimental data. Network parameters were set using a genetic algorithm that fit the firing rate and model behavior to neural recording and psychophysical data  The model was validated by replicating microstimulation and fMRI studies

18. Bonaiuto, jimmy@vis.caltech.edu Example 2: Reach Target Selection We developed a model of the parieto-frontal reach circuit with each region based on macaque neurophysiological data and interregional connections constrained by tract-tracing studies The output of the dorsal premotor region was decoded and used to control a simulated arm/hand

19. Bonaiuto, jimmy@vis.caltech.edu Synthetic PET: Comparison with Experimental Data  % change in rCBF in F2, F6, V6a, and PFC matches published neuroimaging data (Savaki et al., 1997, left)  Differences in activity in V4 highlight a PFC→V4 connection overlooked in model construction  Updated model activity closely matches published data (right) 0.0 0.8 0.4 Signal % Change F6F2LIPV6APFCV4F6F2LIPV6APFCV4

20. Bonaiuto, jimmy@vis.caltech.edu Example 2: Summary A virtuous cycle of models and experiments  We used available connectivity and neural recording data to develop a model of reach target selection  Synthetic brain imaging was used to compare the global model activity to metabolic signals in the monkey brain  This comparison was used to update the model to include feedback connections from PFC to V4

21. Bonaiuto, jimmy@vis.caltech.edu Peigneux et al (2004)  Looked at familiar vs novel gesture imitation  Found that brain areas associated with gesture recognition and production were not more active for familiar vs novel imitation Example 3: Imaging of a Cognitive Model of Apraxia Red: visuo-gestural coding Green: input praxicon Blue: output praxicon

22. Bonaiuto, jimmy@vis.caltech.edu Synthetic PET on a Cognitive Model of Apraxia

23. Bonaiuto, jimmy@vis.caltech.edu Example 3: Summary  A model of gesture recognition and imitation was used to generate PET predictions that conflict with those generated in an ad-hoc manner using a conceptual model  The model is simple, but nonetheless more complex than the conceptual one used by experimentalists  Data necessary to constrain the model do not exist – the model demonstrates one possibility  In this situation multiple competing models should be developed and used to determine an experiment that could disambiguate them

24. Bonaiuto, jimmy@vis.caltech.edu Summary Synthetic imaging can bridge the gap between electrophysiology and neuroimaging We gave three examples using this technique to  Validate a neural model  Refine a neural model  Offer a novel interpretation of experimental data Future studies could use this technique to generate predictors for fMRI analysis

25. Bonaiuto, jimmy@vis.caltech.edu Arbib Lab Rob Schuler Itti Lab David Berg Farhan Baluch Funding NSF Sloan Foundation Thank you