1. Multimodal Brain Imaging Wellcome Trust Centre for Neuroimaging, University College, London Guillaume Flandin, CEA, Paris Nelson Trujillo-Barreto, CNC, Havana Will Penny

2. Experimental Manipulation Neuronal Activity MEG,EEG Optical Imaging PET fMRI Single/multi-unit recordings Spatial convolution via Maxwell’s equations Temporal convolution via Hemodynamic/Balloon models FORWARD MODELS Sensorimotor Memory Language Emotion Social cognition

3. Experimental Manipulation Neuronal Activity MEG,EEG fMRI Spatial deconvolution via beamformers INVERSION

4. MEG/EEG: Source reconstruction Source model Forward model Inverse method Registration ‘Imaging’ ‘ECD’ Data Anatomy

5. Sources ‘Imaging’ ‘Equivalent Current Dipoles’ (ECD) Source Reconstruction MEG data EEG data MEG/EEG: Source reconstruction

6. Experimental Manipulation Neuronal Activity MEG,EEG fMRI Spatial deconvolution via beamformers Temporal deconvolution via model fitting/inversion INVERSION

7. fMRI Forward Model Seconds

8. fMRI model fitting Motion Correction Smoothing Spatial Normalisation General Linear Model Effect Map fMRI time-series Parameter Estimates Design matrix Anatomical Reference

9. Experimental Manipulation Neuronal Activity MEG,EEG fMRI Spatial deconvolution via beamformers Temporal deconvolution via model fitting/inversion INVERSION 1. Spatio-temporal deconvolution 2. Probabilistic treatment

10. Overview Spatio-temporal deconvolution for M/EEG Spatio-temporal deconvolution for fMRI Towards models for multimodal imaging

11. Spatio-temporal deconvolution for M/EEG Add temporal constraints in the form of a General Linear Model to describe the temporal evolution of the signal. Puts M/EEG analysis into same framework as PET/fMRI analysis. Work with Nelson. Described in chapter of new SPM book: Friston et al. Statistical Parametric Mapping, Elsevier, London, 2006

13. Approximate Bayesian Inference Repeat Update source estimates, q(j) Update regression coefficients, q(w) Update spatial precisions, q(  ) Update temporal precisions, q( ) Update sensor precisions, q(  ) Until change in F is small L F KL

15. Corr(R3,R4)=0.47

18. o + 500ms Low Symmetry Low Asymmetry High Symmetry High Asymmetry Phase 1 Time 600ms + 700ms + o 2456ms + FaFa + SbSb UbUb + SaSa Henson R. et al., Cerebral Cortex, 2005

19. B8 A1 Faces minus Scrambled Faces 170ms post-stimulus

20. B8A1 Faces Scrambled Faces

21. Daubechies Cubic Splines Wavelets

22. 28 Basis Functions 30 Basis Functions Daubechies-4

23. ERP Faces ERP Scrambled

24. t = 170 ms

25. Faces – Scrambled faces: Difference of absolute values

26. Spatio-temporal deconvolution for fMRI Temporal evolution is described by GLM in the usual way. Add spatial constraints on regression coefficients in the form of a spatial basis set eg. spatial wavelets. Automatically select the appropriate basis subset using a mixture prior which switches off irrelevant bases. Embed this in a probabilistic model. Work with Guillaume. In Neuroimage.

27. Spatial Model eg. Wavelets

28. Mixture prior on wavelet coefficients (1)Wavelet switches: d=1 if coefficient is ON. Occurs with probability  (2)If switch is on, draw z from the fat Gaussian.

29. Probabilistic Model for fMRI fMRI data General Linear Model Wavelet coefficients Temporal Model Spatial Model Wavelet switches Switch priors

31. Compare to (i) GMRF prior used in M/EEG and (ii) no prior

32. Inversion using wavelet priors is faster than using standard EEG priors

33. Results on face fMRI data

34. Towards multimodal imaging Use simultaneous EEG- fMRI to identify relationship Between EEG and BOLD (MMN and Flicker paradigms) EEG is compromised -> artifact removal Testing the `heuristic’ Start work on specifying generative models Ongoing work with Felix Blankenburg and James Kilner

35. fMRI results

37. We have “synchronized sEEG-fMRI” – MR clock triggers both fMRI and EEG acquisition; after each trigger we get 1 slice of fMRI and 65ms worth of EEG. Synchronisation makes removal of GA artefact easier MRI Gradient artefact removal from EEG

38. Ballistocardiogram removal Could identify QRS complex from ECG to set up a ‘BCG window’ for subsequent processing

39. Ballistocardiogram removal

41. The EEG-BOLD heuristic (Kilner, Mattout, Henson & Friston) contends that increases in average EEG frequency predict BOLD activation. g(w) = spectral density Testing the heuristic

42. Log of Bayes factor for Heuristic versus Null

43. Log of Bayes factor for Heuristic versus Alpha

44. Probabilistic model for EEG-fMRI

45. THANK-YOU FOR YOUR ATTENTION !