1. Multimodal Integration and Inter-subject Registration surfer. nmr. mgh
Multimodal Integration and Inter-subject Registration surfer.nmr.mgh.harvard.edu

2. Overview Intra-subject registration Affine transformations
Registration, Automatic and Manual
fMRI Integration
fMRI Analysis Intro
Registration
Viewing on Volume and Surface
ROI analyses
Surface-based group analysis
Inter-subject registration

3. Theory of Affine Spatial Transforms
Anatomical (1x1x1.1mm, 256x256x128, Sag)
Scanner Acquisition
Functional (3x3x5mm, 64x64x30, Axial)

4. Theory of Affine Spatial Transforms
Conformed Anatomical Space 1x1x1mm, 256x256x256, Cor
Native Anatomical Space 1x1x1.1mm, 256x256x128, Sag
“Anatomical Space”
orig.mgz
Surfaces
Parcellations
Segmentations

5. Theory of Affine Spatial Transforms
“Anatomical Space”
Native Functional Space 3x3x5mm, 64x64x30, Axial
???
Conformed Anatomical Space 1x1x1mm, 256x256x256, Cor

6. Multi-modal Intra-subject Registration
FreeSurfer Anatomical (orig)
Template Functional
Note: Registering the template functional volume to the anatomical volume
is sufficient to register the template to the surface.

7. FreeSurfer Registration and
Template Volume
FreeSurfer
Subject-Specific
Volumes
Surfaces
Thickness
ROIs
Template Volume
fMRI
DTI
ASL
PET …
Registration
Template Volume:
In voxel-for-voxel registration with parameter map
Best gray-white contrast

8. FreeSurfer Registration Matrix
4x4 Matrix
As many as 12 DOF (usually 6)
Simple Text file
Coordinate system not easy to explain
mgh v2
e e e e-01
e e e e+01
e e e e+00
round

9. FreeSurfer Registration Matrix
4x4 Matrix
As many as 12 DOF (usually 6)
Simple Text file
Coordinate system not easy to explain
mgh v2
e e e e-01
e e e e+01
e e e e+00
round
 FreeSurfer subject name
 Functional In-plane resolution mm
 Functional Between-plane resolution mm
 Intensity (for visualization)
 Legacy

10. Automatic Intra-subject Registration
bbregister \
-–subject bert \
–-mov mmtemplate.nii \
--bold \
--init-fsl \
–-reg register.dat
 Command name
 FreeSurfer subject name
 Multimodal template volume
 Multimodal contrast
 Initialize with FSL-FLIRT
 Output registration file
BB = Boundary-based, about 5 min.
Registers template to conformed anatomical of given subject (bert)
Registration is initialized with FSL-FLIRT
6 DOF
Initialization also with --init-spm and --init-header
About 5 min

11. Manual Intra-subject Registration
tkregister2 \
–-mov mmtemplate.nii \
–-reg register.dat \
--surfs
 Command name
 Multimodal template volume
 registration file
 Display white surf
Note similarity to bbregister command
Subject not needed (alread in register.dat file)
tkregister2 --help

12. Manual Intra-subject Registration
Visually inspect registration
Manually edit registration (6 DOF)
Cf Manual Talairach registration
Green line is white surface
tkregister2 --help

13. Tips Rigid = 6 DOF = No stretching
Use CSF to get a sense of where the folds are
Avoid using B0 distortion regions
Avoid using ventricles
Warning about “edge” of the brain
Same Subject, Left-Right Flips

14. } Command-line Tools Automatic Registration: bbregister --help
fslregister –help
spmregister –help
reg-feat2anat –help
Manual Registration:
tkregister2 --help
Transformations:
mri_vol2surf --help
mri_vol2vol --help
mri_label2vol --help
mri_surf2vol --help }
FreeSurfer Scripts

15. fMRI Integration Visualize individual fMRI results on surface volume
ROI Volume Study: Count number of voxels above threshold in an anatomical ROI
ROI Intensity Study: Average HRF inside of an ROI
Surface-based fMRI group analysis

16. Hemodynamic Response (BOLD)
Time-to-Peak (~6sec)
Dispersion
TR (~2sec)
Equilibrium
(~16-32sec)
Undershoot
Delay (~1-2sec)

17. Multiple Presentations/Averaging
Individual Output: HRF Amp, HRF Var, p/z/t/F

18. fMRI Preprocessing Overview
Motion Correction (MC Template)
Use for registration template
bbregister --mov mctemplate.nii --s subject --init-fsl --reg register.dat
tkregister2 --mov mctemplate.nii --reg register.dat --surf
Do not use nonlinear resampling to talairach/MNI space
Do not spatially smooth (3d)

19. fMRI Analysis Overview
First-Level (Individual) Analysis
HRF Amplitude (or Contrast of Amplitudes)
cope (FSL),
CON (SPM),
ces (FSFAST)
Variance of Amplitude
varcope (FSL), ??? (SPM), cesvar (FSFAST)
Activation/Significance Maps: z t F
sig (-log10(p))
All in alignment with MC Template!!!!

20. Template and Map
Functional
Template
Template+
Map

21. Volume Viewing tkmedit subject orig.mgz –aux brain.mgz –aparc+aseg \
-overlay sig.nii –reg register.dat \
-fthresh 2 –fmax 4
sig.nii – significance map in native functional space. could have been z, t, or F map as well.
register.dat – freesurfer registration file
fthresh – lower threshold (value depends on map). You can change this in the interface.
fmax – saturation threshold. (value depends on map). You can change this in the interface.
aparc+aseg – display aparc+aseg.mgz. Can load this from the interface too.

22. Volume Viewing Red/Yellow + Blue/Cyan - Seg Opacity ROI Average
ROI Count
place holder

23. Sampling onto the Surface
White/Gray
Pial

24. Sampling onto the Surface
White/Gray
Pial
White/Gray
Pial
Half Way
Average
Projection Fraction
--projfrac 0.5

25. Sampling on the Surface

26. Sampling on the Surface: Projection Fraction
-0.1
0.0 (white)
+0.1
+0.3
+0.5
+0.7
+0.9
+1.0 (pial)
+1.1

27. Surface Viewing Resample HRF Contrast Significance to left hemisphere
mri_vol2surf \
--mov sig.nii \
--reg register.dat \
--hemi lh \
--projfrac 0.5 \
--o lh.sig.mgh
 map in native functional space
 freesurfer registration file
 hemisphere
 projection fraction (half)
 output (Nvertices-x-1 mgh format)
Note similarity to bbregister and tkregister2 commands.
Load HRF Contrast Significance as overlay
tksurfer subject lh inflated -aparc –overlay lh.sig.mgh

28. Surface Viewing Red/Yellow +,Blue/Cyan - Parcellation Outline
ROI Average
ROI Count

29. ROI fMRI Analyses: Intensity Volume Full Anatomical ROI
Functionally Constrained ROI
Volume

30. Average HRF within Full Anatomical ROI
Eg, average functional HRF amplitudes from all voxels inside of post-central gyrus (red) regardless of whether a voxel is significant or not.
Resampling puts ces into voxel-for-voxel registration with aseg.mgz

31. Average HRF within Full Anatomical ROI
Resample HRF Contrast to anatomical space
mri_vol2vol \
--mov ces.nii \
--reg register.dat \
--interp nearest \
--fstarg \
--o ces.anat.mgh
 Command name
 HRF map in functional space
 FreeSurfer Registration File
 Nearest neighbor interpolation
 Specify anatomical output space
 Output file in anatomical space
Note similarity to bbregister, tkregister2, and mri_vol2surf commands.
Average HRF Contrast within ROIs
mri_segstats \
--seg $SUBJECTS_DIR/subject/mri/aseg.mgz \
--ctab $FREESURFER_HOME/FreeSurferColorLUT.txt \
--i ces.anat.mgh --sum ces.aseg.stats
Resampling puts ces into voxel-for-voxel registration with aseg.mgz

32. Average HRF within Full Anatomical ROI
Average HRF Contrast within ROIs
mri_segstats \
--seg $SUBJECTS_DIR/subject/mri/aseg.mgz \
--ctab $FREESURFER_HOME/FreeSurferColorLUT.txt \
--i ces.anat.mgh --sum ces.aseg.stats
Notes:
--seg is the segmentation (eg, aseg.mgz, aparc+aseg.mgz, etc)
--ctab is matching color lookup table
Output File: ces.aseg.stats
simple text file with same format aseg.stats
multiple subjects can be combined with asegstats2table

33. Average HRF within a Functionally Active area inside of an Anatomical ROI
Resampling puts ces into voxel-for-voxel registration with aseg.mgz
Eg, average functional HRF amplitudes from voxels inside of post-central gyrus (red) for voxels that have
p<.01 (sig>2) regardless of sign (yellow or blue), or
p<.01 (sig>2) for positive activation (yellow only), or
p<.01 (sig>2) for negative activation (blue only)

34. Average HRF only within the Functionally Active Area of an Anatomical ROI, Count Voxels above threshold in each ROI
Resample HRF Contrast Significance to anatomical space
mri_vol2vol \
--mov sig.nii \
--reg register.dat \
--interp nearest \
--fstarg \
--o sig.anat.mgh
Average HRF Contrast within functionally constrained ROIs
mri_segstats \
--seg $SUBJECTS_DIR/subject/mri/aseg.mgz \
--ctab $FREESURFER_HOME/FreeSurferColorLUT.txt \
--i ces.anat.mgh --sum ces.aseg.mask.stats \
--mask sig.anat.mgh --mask-thresh 2 --mask-sign abs

35. Average HRF only within the Functionally Active Area of an Anatomical ROI, Count Voxels above threshold in each ROI
mri_segstats \
--seg $SUBJECTS_DIR/subject/mri/aseg.mgz \
--ctab $FREESURFER_HOME/FreeSurferColorLUT.txt \
--i ces.anat.mgh --sum ces.aseg.mask.stats \
--mask sig.anat.mgh --mask-thresh 2 --mask-sign abs
Volume in stats file is volume above threshold (may be 0)
Sign is important for Average!
abs, pos, or neg
pos will always result in positive HRF average
neg will always result in negative HRF average
abs ????
Careful to avoid circularity
Can use a different contrast to mask

36. Surface-based Group Analysis
mris_preproc \
--hemi lh \
--o lh.fsaverage.ces.mgh \
--iv subject1/ces.nii subject1func/register.dat \
--iv subject2/ces.nii subject2func/register.dat \
--iv subject3/ces.nii subject3func/register.dat \
...
After that, everything else is the same as a thickness study …
mris_fwhm --i lh.fsaverage.ces.mgh --fwhm 10 \
--o lh.fsaverage.ces.sm10.mgh --cortex
mri_glmfit --surf fsaverage lh –cortex \
--y lh.fsaverage.ces.sm10.mgh ...

37. Tutorial Registration – manual and automatic registration
fMRI Integration (Sensorimotor Paradigm)
Individual
Volume view sig
Surface view sig
ROI analysis with and without functional constraint
Group
mris_preproc
ROI analysis (asegstats2table)

38. Another type of problem: inter-subject uni-modal registration
Surface-based (2D) registration does an excellent job of aligning cortical folds, but does not apply to non-cortical structures (e.g. basal ganglia).
Volumetric (3D) registration applies to the entire brain but does not, in general, align folding patterns.
Goal: combined their strength

39. Why aligning folds in the volume is hard…
Bruce’s brain affine transformed and mapped on Doug’s
Affine – does not provide the a good initialization; does not bring folds close enough for the basin of attraction
Affine transform of surfaces from one subject mapped to another.

40. pial surface WM surface Template Affine Nonlinear
Mention (red , yellow)

41. Combined volumetric and surface-based registration (CVS)
Spherical alignment
Elastic propagation of
cortical registration results
in the 3D volume
Volumetric alignment of sub-cortical regions
sub-cortical alignment using cortical correspondence as initialization

42. Resulting morph
Template
Elastic
CVS

43. Template HAMMER FLIRT CVS
Examples of results obtained in Inter-Subject Experiment 2. The first column displays the template segmentation and the second, third and
fourth columns represent the deformed segmentation volumes resulting from FLIRT (affine), HAMMER and our new combined surface- and intensity-based registration. In all the images, the surfaces shown are those of the template (pial surfaces in red and gray/white surfaces in yellow).

44. Extended Jaccard Coefficient (23) measures computed for Experiment 1
Extended Jaccard Coefficient (23) measures computed for Experiment 1. The last column of the cortical measures represents the overlap measure
computed in the surface space. The vertical lines represent the standard error of the mean of the measurement.
Extended Jaccard Coefficient measures: 20 cortical and 21 sub-cortical labels. (The vertical lines represent the standard error of the mean of the measurement.)
G.M. Postelnicu*, L . Zöllei*, B. Fischl: "Combined Volumetric and Surface Registration", IEEE Transactions on Medical Imaging (TMI), Vol 28 (4), April 2009, p

45. mri_cvs_register --mov subjid
registering the subject to, by default, the CVS atlas space
make sure that the SUBJECTS_DIR for subjid is correctly set
Optional Arguments
-- template subjid : subjid for template subject
-- templatedir dir : recon directory for template
(default is SUBJECTS_DIR)
--outdir dir : output directory for all the results
(default is SUBJECTS_DIR/subjid/cvs)
… and many more: use --help
-I like how you have "moving" in italics. Will you cover how commands like vol2vol and flirt are usually asking for a mov, ref, template, etc.? Some of those terms are interchangeable and it can be confusing when different commands use different terms.

46. mri_cvs_register
Optional Arguments (cont) --step1 Only do step 1 (spherical registration). --step2 Only do step 2 (elastic registration). --step3 Only do step 3 (volumetric registration). --noaseg Do not use aseg volumes in the volumetric registration pipeline (default is 0). Setting this option could shorten significantly the time of registration, however, might also take away from the accuracy of the final results.

47. mri_cvs_register
Optional Arguments (cont) --nocleanup Do not delete temporary files (default is 0). --keepelreg Do not delete elastic registration (default is 0) outcome. --cleanall Recompute all CVS-related morphs that might have been computed prior to the current CVS run (def = 0). --cleansurfreg Recompute CVS-related surface registration morphs that might have been computed prior to the current CVS run (def = 0). --cleanelreg Overwrite /recompute the CVS-related elastic registration morph that might have been computed prior to the current CVS run (default is 0). --cleanvolreg Overwrite / recompute CVS-related volumetric morphs that might have been computed prior to the current CVS run (default is 0).

48. CVS atlas

49. related commands mri_cvs_check mri_cvs_data_copy mri_vol2vol
checking whether all files needed for a successful CVS registration are present
mri_cvs_data_copy
copying the CVS-relevant recon directories over to a new location
mri_vol2vol
applying the CVS registration morph to files corresponding to the moving subject

50. Applying CVS morphs mri_vol2vol applying CVS morph to aseg file
mri_vol2vol --targ templateid --m3z morph.m3z \
--noDefM3zPath --mov asegvol \
--o asegvol2CVS --interp nearest \
--no-save-reg
applying morph to corresponding diffusion file
mri_vol2vol --targ templateid --m3z morph.m3z
--noDefM3zPath --reg 2anat.register.dat
--mov diffvol --o diffvol2CVS
paths
$FREESURFER_HOME/subjects/cvs_avg35

51. Applying CVS morphs
applyMorph: use with older style CVS morphs (.tm3d)
applying CVS morph to aseg file
applyMorph --template templateid --transform morph.tm3d\
vol asegvol asegvol2CVS nearest
applying morph to corresponding diffusion file
applyMorph --template templateid –transform morph.tm3d
vol diffvol2anat diffvol2CVS linear
paths
$FREESURFER_HOME/subjects/cvs_avg35

52. Application of CVS to tractography
Goal: fiber bundle alignment
Study: compare CVS to methods directly aligning DWI-derived scalar volumes
Conclusion: high accuracy cross-subject registration based on structural MRI images can provide improved alignment
Zöllei, Stevens, Huber, Kakunoori, Fischl: “Improved Tractography Alignment Using Combined Volumetric and Surface Registration”,  NeuroImage 51 (2010),

53. Average tracts after registration mapped to the template displayed with iso-surfaces
FLIRT
FA-FNIRT
CVS
The probability maps were thresholded at 0.1

54. Mean Hausdorff distance measures for three fiber bundles
CST
ILF
UNCINATE

55. Functional MRI analysis in CVS space
Collaboration with Kami Koldewyn, Joshua Julien and Nancy Kanwisher at MIT

56. Ongoing development
Improve CVS capability to register ex-vivo to in-vivo acquisitions
Implemented MI-based volumetric registration (for CVS step 3) to accommodate intensity profile differences
Qualitative preliminary results on 4 subjects
L. Zöllei, Allison Stevens, Bruce Fischl: Non-linear Registration of Intra-subject Ex-vivo and In-vivo Brain Acquisitions, Human Brain Mapping, June 2010
L. Zöllei, B. Fischl, Automatic segmentation of ex-vivo MRI images using CVS in FreeSurfer, HBM 2011

57. Subject 1
Target (in-vivo) Masked target step CVS CVS with MI

58. registration tool summary
mris_register
fslregister: bet + flirt
bbregister
mri_robust_register
mri_cvs_register
mri_nl_align
Some require recons!
-with the registration tool summary slide, will you cover the strengths and weaknesses of each or when it is most appropriate to use them? Like
bbregister and cvs can only be used if you have surfaces, fsl takes into account X while robust_register takes into account Y so you would choose one over the other in such Z situations.

59. registration morph summary
.dat, .lta, .xfm, .fslmat: encode rigid and affine transformations
mri_vol2vol
sphere.reg: encodes spherical morph
mris_resample
.m3z, .tm3d: encode nonlinear volumetric morphs
mri_vol2vol, applyMorph
Some require recons!

60. Summary Multi/Cross-modal map (HRF Amplitude, FA)
Multimodal Integration requires a Template
A Template is:
Same size as multimodal map
In Voxel-to-voxel alignment with map
Has better anatomical contrast
Baseline functional
Low-B DTI
Usually a MC template
Volume and Intensity ROI Analyses
Functionally-constrained ROI