1. NA-MIC National Alliance for Medical Image Computing http://na-mic.org fMRI Analysis ; Registration William (Sandy) Wells

2. National Alliance for Medical Image Computing http://na-mic.org Theme fMRI Registration algorithmic point of view... –where are we now –where are we going –where should we be going ?

3. National Alliance for Medical Image Computing http://na-mic.org fMRI Part fMRI in Slicer –Current Status –what’s next fMRI directions –Example

4. National Alliance for Medical Image Computing http://na-mic.org Slicer fMRI Goals Advanced GUI and Interactive Visualization Environment –iBrowser Platform for Activation Detector Research –fMRIEngine Framework for fMRI Integration with Other Modalities W. Plesniak, S. Pieper, W. Wells

5. National Alliance for Medical Image Computing http://na-mic.org ibrowser: time-series GUI W. Plesniak, S. Pieper, W. Wells name order visibility copy delete hold viewer FG viewer BG animation & viewing manual indexing * GUI controls for indexing, animating, operating on sequences of volumes; * GUI panel provides a graphical schematic of loaded volume sequences.

6. National Alliance for Medical Image Computing http://na-mic.org ibrowser: GUI updates Viewer W. Plesniak, S. Pieper, W. Wells

7. National Alliance for Medical Image Computing http://na-mic.org ibrowser: other applications visualizing animated cardiac perfusion study W. Plesniak, S. Pieper, W. Wells

8. National Alliance for Medical Image Computing http://na-mic.org ibrowser: current / future work design mock-up interval browser: early version in Slicer 2.4 Data loading: Analyze, DICOM, BXH; Data organization: persistent GUI organizes and indexes multi-volume data; Data processing: multi-volume window, level and threshold; reorient; shuffle; MRML extension in progress: vtkMRMLVolumeCollectionNode; vtkMRMLVolumeGroupNode; vtkMRMLVolumeRefNode Ongoing & future work: additional processing capabilities as non-rigid registration; representing collections of other data types (models, events, etc.) as illustrated. W. Plesniak, S. Pieper, W. Wells

9. National Alliance for Medical Image Computing http://na-mic.org fMRIEngine: overview activation detection: model specification: visualization: information: GLM-based MI-based design matrix signal modeling spatial priors voxel time-course and experimental protocol activation overlays help tool tips The fMRI Engine: reads volume series in Analyze, DICOM and BXH format; detects and visualizes activation in fMRI data. will contain suite of activation detectors (currently GLM) and modeling options includes tools to visualize statistical activation data H. Liu, W. Plesniak, S. Pieper, W. Wells, C. Wible

10. National Alliance for Medical Image Computing http://na-mic.org fMRIEngine: current status H. Liu, W. Plesniak, S. Pieper, W. Wells, C. Wible Data loading: Loads Analyze (3D and 4D), DICOM, BXH (BIAC XML Header) format data; or imports from Interval Browser; Protocol specification: input block design via GUI or load/save in text file; Activation computing: GLM detection, currently supports block design protocol and single regressor;

11. National Alliance for Medical Image Computing http://na-mic.org fMRIEngine: visualization H. Liu, W. Plesniak, S. Pieper, W. Wells, C. Wible generates color-coded parametric map of activation 3D visualization of activation in the context of subject’s own anatomy or in a standardized morphological space; provides interactive activation filtering and background masking.

12. National Alliance for Medical Image Computing http://na-mic.org fMRIEngine: data inspection H. Liu, W. Plesniak, S. Pieper, W. Wells, C. Wible Average voxel timecourse for all volumes in each of two experimental conditions Measured voxel timecourse over entire protocol interactive inspection of voxel timecourse.

13. National Alliance for Medical Image Computing http://na-mic.org fMRI in Slicer where are we going –Flesh out basic capabilities –Ising Prior on Activation –Non-Standard Detectors

14. National Alliance for Medical Image Computing http://na-mic.org Ising Prior Eric Cosman Wanmei Ou Polina Golland

15. National Alliance for Medical Image Computing http://na-mic.org GLM / Ising Exact MAP Activity Detection in fMRI Using a GLM with an Ising Spatial Prior Eric Cosman, Jr., Wanmei Ou, John Fisher, William Wells

16. National Alliance for Medical Image Computing http://na-mic.org GLM / Ising… Alternative model for noise reduction Ising MRF prior model In GLM / Hypothesis Test Fast exact solution: Portius, Grieg Result is F-statistic

17. National Alliance for Medical Image Computing http://na-mic.org GLM / Ising Results Eric Cosman, Jr., Wanmei Ou

18. National Alliance for Medical Image Computing http://na-mic.org GLM / Ising… Wanmei Ou –MIT MS Project with Polina Golland –Empirical Characterization of fMRI detectors Larry Panych data GLM / MI MARKOV spatial models –Exact –Approximate

19. National Alliance for Medical Image Computing http://na-mic.org GLM / Ising: Empirical Results Eric Cosman, Jr., Wanmei Ou ROC analysis on realistic synthetic data Low SNR regime: Gaussian Blur is Better High SNR regime: Ising Prior is Better

20. National Alliance for Medical Image Computing http://na-mic.org fMRI Data Processing Summer 2005 – Technology Transfer from Wanmei Ou to Slicer

21. National Alliance for Medical Image Computing http://na-mic.org fMRI Data Analysis Status Under control –Voxel level activation detection –Blob activation detection –Population analysis of voxel activations In Talairach space

22. National Alliance for Medical Image Computing http://na-mic.org fMRI Data Analysis Status Need Work –Population analysis of voxel activations In Atlas Space with High Homological Accuracy –REGISTRATION –Advanced Multi-Variate Analysis Connectivity Analysis System Fitting –SEM –MVAR –Population Multi-Variate Analysis

23. National Alliance for Medical Image Computing http://na-mic.org I do not have permission to distribute Eric Cosman’s thesis research slides.

24. National Alliance for Medical Image Computing http://na-mic.org Registration Part Intensity-Based registration Summary of State of Art Where to next?

25. National Alliance for Medical Image Computing http://na-mic.org Estimate Relationship Among two Signals U : a signal V  : another signal, transformed by 

26. National Alliance for Medical Image Computing http://na-mic.org Estimate Relationship Among two Signals If p(U,V) is Gaussian –Then best f is correlation (or squared difference)

27. National Alliance for Medical Image Computing http://na-mic.org Estimate Relationship Among two Signals If p(U,V) is UNKNOWN –Look for strongest statistical relationship among the signals I : Mutual Information

28. National Alliance for Medical Image Computing http://na-mic.org Mutual Information (MI) H : entropy –measures information content I : Mutual Information - a statistic that measures lack of statistical independence

29. National Alliance for Medical Image Computing http://na-mic.org MI Registration Default Method for Multi-Modal Medical Image Registration Viola Wells et al. circa 96 –Collignon, and Hill & Hawkes Pluim et al. Survey, 2003: More than 160 published applications

30. National Alliance for Medical Image Computing http://na-mic.org Example MRT Rigid Registration Pre-operative SPGR MRIIntra-operative T2-weighted MRI Provided by D. Gering

31. National Alliance for Medical Image Computing http://na-mic.org Before Registration After Registration Provided by D. Gering Example MRT Rigid Registration

32. National Alliance for Medical Image Computing http://na-mic.org Registration of Video and 3D Model Paul Viola MIT PhD Thesis 1996 Provided by Paul Viola

33. National Alliance for Medical Image Computing http://na-mic.org Entropy Measures for Joint Registration Motivation: Atlas Formation Advantage of Entropy Criteria –Accommodates multi modal result Multiple peaks in probability Variant Anatomies

34. National Alliance for Medical Image Computing http://na-mic.org Entropy Measures for Joint Registration Congealing –Erik Learned-Miller MIT PhD Handwriting Recognition, mostly –Iterative Non-Rigid Registration For low total entropy result

35. National Alliance for Medical Image Computing http://na-mic.org Congealing Movie Provided by Erik Learned-Miller

36. National Alliance for Medical Image Computing http://na-mic.org T1T1 T5T5 T4T4 T3T3 T2T2 T6T6 T7T7 TNTN … Congealing: simultaneous non-rigid registration of N input images Goal: find “central tendancy”: Provided by Lilla Zollei

37. National Alliance for Medical Image Computing http://na-mic.org A Binary Entropy Measure to Assess Nonrigid Registration Algorithms Simon K. Warfield, Jan Rexilius, Petra S. Huppi, Terrie E. Inder, Erik G. Miller, William M. Wells, Gary P. Zientara, Ferenc Jolesz and Ron Kikinis

38. National Alliance for Medical Image Computing http://na-mic.org Results: Nonrigid MEAM CSF 0.04bpv SGM 0.05bpv CGM 0.15bpv Myelin 0.04bpv UWM 0.14bpv Average SPGR Provided by Simon Warfield

39. National Alliance for Medical Image Computing http://na-mic.org Atlas and Registration Technology Lilla Zollei : PhD Thesis 2005 –Multi-way fusion for atlas formation Congealing – total entropy reduction Conventional MRI DT-MRI

40. National Alliance for Medical Image Computing http://na-mic.org DTI registration How to estimate Entropy of a collection of Diffusion Tensors –How to estimate a Density on a collection of Diffusion Tensors How to apply a transformation to a tensor-valued dataset?

41. National Alliance for Medical Image Computing http://na-mic.org Stronger Models for Registration MI is effective when joint intensity models are unknown More Robustness may be obtained with concrete application-specific models

42. National Alliance for Medical Image Computing http://na-mic.org KL approach to registration D KL : Kullback-Leibler Distance P o (U,V) : Known Joint Intensity Model P(U,V  ) : New Image Data offset by hypothesized transformation  : Transformation Parameters

43. National Alliance for Medical Image Computing http://na-mic.org Multi-Modality image registration by minimising Kullback-Leibler distance ACS Chung, WM Wells, WEL Grimson, A Norbash MICCAI 2002 2D/3D DSA to MRA Provided by Albert Chung KLD Registration Example…

44. National Alliance for Medical Image Computing http://na-mic.org KL Registration Example Rigid Registration of Echoplanar and Conventional Magnetic Resonance Images by Minimizing the Kullback-Leibler Distance Second International Workshop on Biomedical Image Registration Philadelphia 2003 Salil Soman, Albert C.S. Chung, W Eric L Grimson, William M Wells III Also: MIT EECS MS Thesis by Salil Soman (same title)

45. National Alliance for Medical Image Computing http://na-mic.org KLD Registration Results Expert’s Pose KLD AVG Pose JE AVG Pose Provided by Salil Soman

46. National Alliance for Medical Image Computing http://na-mic.org Recent results… Multi-resolution multi-modal image registration based on prior joint intensity distributions and Kullback-Leibler distance Albert C.S. Chung, Rui Gan and William M. Wells III IEEE TMI Submitted

47. National Alliance for Medical Image Computing http://na-mic.org MR CT: Vanderbilt Data Provided by Albert Chung

48. National Alliance for Medical Image Computing http://na-mic.org MR-CT Probing MIMLaKLD Low res translation Full res translation Rotation Provided by Albert Chung

49. National Alliance for Medical Image Computing http://na-mic.org Registration Technology Under Control: –3D – 3D Rigid, Intensity Based –Non-Rigid Daniel Reuckert B-Spline Mesh “FFD” Linear Elastic Deformation Model –Warfield, others –Among Cortices: FreeSurfer –Inflate to sphere –Match spherical representations

50. National Alliance for Medical Image Computing http://na-mic.org Registration Technology More Work Needed: –EPI Registration Distortions, Voids MR Physics? –Registration of Diffusion MRI (WM) Recent: CF Westin, HJ Park Probably volumetric –Population Fusion Automatic construction of statistical atlases having High Homological Accuracy –Joint Registration Cortex: Surface Based WM: Volumetric

51. National Alliance for Medical Image Computing http://na-mic.org Thanks… Students and Collaborators Ron Kikinis Allen Tanenbaum Sanjay Manandhar Tina Kapur Many others…