1. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Techniques for Visualization of Anatomy and Function from MRI Steve Pieper, PhD VizBi 2010 Heidelberg

2. National Alliance for Medical Image Computing http://na-mic.org Outline Motivation Standard Clinical MRI Visualization Research Imaging –Registration –Segmentation and Parcellation –Functional Imaging Applications and Open Issues

3. National Alliance for Medical Image Computing http://na-mic.org Example: Morphometry Group Statistics Regional Cortical Thickness Correlation with Aging and Cognitive Impairment –Greater Sensitivity than Clinical Tests –Localize Disease within Brain Anatomy –Impact of Treatment on Imaging Biomarkers Fischl, Greve, et al - MGH

4. National Alliance for Medical Image Computing http://na-mic.org Example: Multimodality Image Guided Neurosurgery Detailed Pre-Operative Model –Registered to Tracked Instruments –Superimposed on Intraoperative MRI Extracted Anatomy and Function –Roadmap for Surgical Decision Making What is the Best Surgical Approach to Preserve Function? Golby, Kikinis, Lemaire, BWH Neurosurgery

5. National Alliance for Medical Image Computing http://na-mic.org MRI Context MRI: Typically Volumetric at ~1mm Resolution –Unlike CT, No Physical Units for MR –Contrast Determined by Scan Protocol Much of clinical MRI is pattern recognition –Seeing things that “don’t look right” Anatomical Conventions –Like Maps having North at the Top –Axial (Transverse), Sagittal, Coronal Analysis Results –Spatially Localized Data –Often visualization is used as a ‘reasonableness check’ on the automated calculations Wikipedia

6. National Alliance for Medical Image Computing http://na-mic.org Basic Clinical Visualization Window/Level Corner Annotations Pseudocolor Mosaic/Lightbox Cine

7. National Alliance for Medical Image Computing http://na-mic.org 3D Clinical Visualization MR Angiogram MIP from Siemens Leonardo Workstation Ray Casting Through Volume –Summation (Simulated X-Ray) –MIP (Maxiumum Intensity Projection) –SSD (Shaded Surface Display) Color Transfer Function and Opacity Transfer Function Pseudocolor + Gradient Lighting Less Applicable for MRI than CT Reference Labels for Standard Views –Left/Right, Anterior/Posterior, Inferior/Superior

8. National Alliance for Medical Image Computing http://na-mic.org MRI Contrast Enhancement Vascular / Oral: Gadolinium, Iron Oxide, Manganese –Change MR Properties of Tissues Dynamic Contrast Tissue Enhancement (4D) –Vascularized Tumor –Stroke Related Ischemia Perfusion, Diffusion, Bloodflow Mismatch Indicates Brain Tissue that May Recover Gilbert et al University of Wisconsin Gonzalez et al MGH

9. National Alliance for Medical Image Computing http://na-mic.org Longitudinal Imaging (4D) Volumes Acquired Over Multiple Visits –“Watchful Waiting” Prior to Intervention –Monitor Treatment (Multiple Sclerosis, Cancer, Lupus…) Comparison View –Linked Cursors –Subtraction Imaging and Quantification Guttmann, Meier, Fedorov – BWH Miller - GE

10. National Alliance for Medical Image Computing http://na-mic.org Registration Intra-subject –Pre-Intra-Post Procedure –Longitudinal Tracking of Disease Progression Inter-subject –Support Group Comparison (fMRI) –Map Anatomical Atlas to Individual Degrees of Freedom (DOFs) –Rigid (Rotation + Translation) –Similarity (Rigid + Uniform Scale) –Affine (Rigid + Nonuniform Scale and Shear) –Polyaffine (Locally Affine Interpolation) –B-Spline (Cubic Displacement) –Vector Field Sota - BWH

11. National Alliance for Medical Image Computing http://na-mic.org Registration Considerations Registration is Typically an Ill-Posed Problem –Requires Nonlinear Optimization –Distortions, Anatomy, Pathology Means No Exact Correspondence –Multimodality Registration Requires Statistical Metric (e.g. Mutual Information) –When No “Right” Answer Only What “Looks Right” or What is Reproducible and Statistically Significant Insight Toolkit (ikt.org) for Details and Software Fixed Image Moving Image Metric Transform Interpolator Optimizer Registration Method Ibanez - Kitware

12. National Alliance for Medical Image Computing http://na-mic.org Registration Visualization Cross Fade / Toggle Color / Skeleton Overlay Checkerboard Vector Fields

13. National Alliance for Medical Image Computing http://na-mic.org Segmentation Definition: Assignment of Anatomical Labels to Image Regions –Not an Exact Science Anatomists Disagree Definition Depends on Scale Techniques –Intensity Driven: Function of Image Measurements Thresholding is Most Common (Typically Bad for MRI) –Rule Based: E.g. “Skin is always on the outside” Obviously not always the case in clinical scans –Atlas Driven: Registration of Manually Labeled Data Also difficult for clinical scans –Hybrid Approaches Typically Required E.g. Expectation Maximization (EM) Pohl – IBM Kikinis, Shenton - BWH

14. National Alliance for Medical Image Computing http://na-mic.org Segmentation Visualization Label Map Overlay –Cross Fade / Toggle –Solid or Outline 3D Surface Models –Leverage Commodity Graphics Cards –Material Properties, Lighting, Transparency…

15. National Alliance for Medical Image Computing http://na-mic.org Parcellation Functional or Anatomical Subdivisions (e.g. Cerebral Hemisphere Surface) Obtained from Curvature and Landmarks on Surface –Flattened to Plane or Sphere for Tractability –Displayed Inflated to Show Sulci (Valleys) and Gyri (Ridges) Fischl et al - MGH

16. National Alliance for Medical Image Computing http://na-mic.org Functional MRI Blood Oxygen Level Dependent (BOLD) Volume Time Series ~2mm Resolution Activation Statistics Volume: Typically Generalized Linear Model (GLM) of –Hemodynamic Response Function (HRF) –Stimulus / Paradigm –Intensity Trends –Physiology –Motion… Statistics Become Difficult –Noisy Data –Multiple Comparisons / Multiple Regressors –Detects Metabolism, not Neural Activity Directly Plesniak, Liu - BWH Finger Tapping

17. National Alliance for Medical Image Computing http://na-mic.org fMRI Visualization Statistics Volume Multi Volume Rendering Cortical Surface Map Group Comparisons in Atlas Space ( Spatial Normalization) –Group Contrasts (Left/Right, Schizophrenics/Normal Control, Active/Resting…) –Positive and Negative Correlations Plesniak, Liu – BWH Hernell – Linköpings U.

18. National Alliance for Medical Image Computing http://na-mic.org Multi-Modality Imaging Integrated Visualization of What is Known About the Subject –Anatomical Space as Common Coordinate System –Segmented Anatomy and Volume Rendering for Context –Statistics Volumes –Interactive Visualization (View, Visibility, Cropping, Slicing… ) Image Guided Therapies Plesniak, Aucoin et al - BWH Jakab and Berenyi - University of Debrecen

19. National Alliance for Medical Image Computing http://na-mic.org Group Comparisons Visualization for MRI Data Mining –Images, Clinical Data, Demographics on Web Database –Target Population and Hypothesis Specify Batch Computation –Group Statistics Overlay on Inflated Cortical Surface of Atlas –Click to Get Scatter Plot of Subjects Supports Alzheimer’s Disease Research Anatomy Labels Linked to Web Resources and Journal Papers Morphometry BIRN Consortium

20. National Alliance for Medical Image Computing http://na-mic.org Open Challenges in MRI Visualization Stereo / Virtual Reality (VR) / Augmented Reality (AR) / Telepresence –No real traction in spite of significant investment –Probably due to lack of mainstream hardware Dynamic multimodal volume rendering –Large volumes – CUDA vs. Cluster –Encode segmentation in transfer function General Information Overload Display of Uncertainty in Analysis Results –Many techniques require discretization