1. Constructing Image Graphs for Segmenting Lesions in Brain MRI May 29, 2007 Marcel Prastawa, Guido Gerig Department of Computer Science UNC Chapel Hill

2. Constructing Image Graphs for Segmenting Lesions in Brain MRI 2 Goal Segmentation of “lesions”: –Abnormal tissue associated with neurodegeneration –Small patches Clinical applications: lupus (NAMIC), MS, aging, depression, NF1 –Different appearances, locations, and shapes –Method needs to be adaptable Example:

3. Constructing Image Graphs for Segmenting Lesions in Brain MRI 3 Outline Background –Goal –Image Graph –Previous Work –Overview Methodology Results Conclusions and Future Work

4. Constructing Image Graphs for Segmenting Lesions in Brain MRI 4 Challenges Lesions are relatively small Wide variety of shape Partial voluming can be confused as lesions Requires knowledge of neighboring structures Voxel classification typically fails Common MRF scheme oversmooths segmentation, hard to balance Proposed solution: Use a hierarchical graph representation

5. Constructing Image Graphs for Segmenting Lesions in Brain MRI 5 Manage hierarchical information Image Graph WMGMCSFLesion Object Atom / Supervoxel / Neighborhood Voxel A1 A2A3 v1 v2v3 Segmentation = determining info at nodes and edges

6. Constructing Image Graphs for Segmenting Lesions in Brain MRI 6 Previous Work [1/3] [Barbu et al, PAMI 2005] Image segmentation by graph clustering Group similar regions using Swendsen-Wang cuts For natural images, no anatomical prior

7. Constructing Image Graphs for Segmenting Lesions in Brain MRI 7 Previous Work [2/3] [Corso, Zhuowen Tu*, et al, IPMI 2007 (UCLA Loni)] Segmentation of subcortical structures through graph shifts Training using boosting No pathological class *DDMCMC discriminative model guided generative model computing

8. Constructing Image Graphs for Segmenting Lesions in Brain MRI 8 Previous Work [3/3] Marcel Prastawa PhD: “An MRI Segmentation Framework for Brains with Anatomical Deviations” –EMS modulated by probabilistic brain atlas: –Nonparametric statistics –Robust clustering –Separation of pathology from healthy (tumor, edema, myelination,..) –ITK implementation: GUI and XML scripts for large throughput –Rigorous validation (repeatability, validity, traveling phantom etc.) –Tested on over 1500 brain MRI 1.Marcel Prastawa, John H. Gilmore, Weili Lin, Guido Gerig, Automatic Segmentation of MR Images of the Developing Newborn Brain, Medical Image Analysis Vol 9, October 2005, pages 457-466 2.John H. Gilmore, Weili Lin, Marcel W. Prastawa, Christopher B. Looney, Y. Sampath K. Vetsa, Rebecca C. Knickmeyer, Dianne Evans, J. Keith Smith, Robert M. Hamer, Jeffrey A. Lieberman, Guido Gerig, Cerebral Asymmetry, Sexual Dimorphism, and Regional Gray Matter Growth in the Neonatal Brain, Accepted by J of Neuroscience, Oct 2006 3.Bénédicte Mortamet, Donglin Zeng, Guido Gerig, Marcel Prastawa, and Elizabeth Bullitt. Effects of Healthy Aging Measured By Intracranial Compartment Volumes Using a Designed MR Brain Database. Lecture Notes in Computer Science LNCS 3749, Oct. 2005, pp. 383 -- 391 4.Marcel Prastawa, Elizabeth Bullitt, Sean Ho, Guido Gerig, A Brain Tumor Segmentation Framework Based On Outlier Detection, Medical Image Analysis Vol. 8, Issue 3, Sept. 2004, pages 275-283 5.Marcel Prastawa, John Gilmore, Weili Lin, and Guido Gerig, Automatic Segmentation of Neonatal Brain MRI, Lecture Notes in Computer Science LNCS 3216, Springer Verlag, pp. 10-17, 2004 6.Marcel Prastawa, Elizabeth Bullitt, Nathan Moon, Koen van Leemput, and Guido Gerig, Automatic Brain Tumor Segmentation by Subject Specific Modification of Atlas Priors. Academic Radiology, Vol. 10 pp. 1341-1348 Dec. 2003 7.Marcel Prastawa, Elizabeth Bullitt, Sean Ho, Guido Gerig, Robust Estimation for Brain Tumor Segmentation, Lecture Notes in Computer Science LNCS 2879, pp. 530-537, Nov. 2003

9. Constructing Image Graphs for Segmenting Lesions in Brain MRI 9 Method Overview WMGMCSFLesion Atlas based training Data driven clustering + anatomy A1 A2A3 v1 v2v3 Bayesian classification Top-down Bottom-up interface

10. Constructing Image Graphs for Segmenting Lesions in Brain MRI 10 Outline Background Methodology Results Conclusions and Future Work

11. Constructing Image Graphs for Segmenting Lesions in Brain MRI 11 Object Level Training based on prior knowledge: brain atlas Use for sampling and as priors No lesion model –Lesion prior = fraction of wm or gm priors WMGMCSFLesion ICBM/MNI atlas, average of 152 healthy adult subjects

12. Constructing Image Graphs for Segmenting Lesions in Brain MRI 12 Outlier Detection Lesion training data obtained via outlier detection Robust estimation using MCD (minimum covariance determ.) WM example: Use outlier samples that fit user defined rule for lesion T1 T2 before after

13. Constructing Image Graphs for Segmenting Lesions in Brain MRI 13 Lesion Rules User defined rule for different lesion [van Leemput, TMI 2001] Embedded Python interpreter, any function with variables for voxel data (i1, … in) and training data (mu1_1 … mu#d_#c) Example rules: – MS lesion for [T1, T2, FLAIR]: (i2 > mu2_2) and (i3 > mu3_1) and (i3 > mu3_2) Radiology terms: Lesion is brighter than gm in T2, brighter than wm in Flair, and lesion is brighter than gm in Flair – NF1 lesion for [T1, T2, PD]: i2 > mu2_2 Can use arithmetic: (i2/i3 > mu2_2/mu3_2) Adaptable: input parameter, can have user def. functions, etc

14. Constructing Image Graphs for Segmenting Lesions in Brain MRI 14 Atom: group of voxels that are perceptually similar Group neighboring voxels that: 1.Look similar 2.Located close to each other 3.Belong to the same category Combining 1, 2 leads to data-driven schemes Atom Assignments CSF A1 A2A3 v1 v2v3

15. Constructing Image Graphs for Segmenting Lesions in Brain MRI 15 Initial Voxel Grouping Group voxels that are similar and close to each other Use watershed algorithm: Input for watershed transform is gradient magnitude image (pictures from Matlab tutorial manual)

16. Constructing Image Graphs for Segmenting Lesions in Brain MRI 16 Multimodal Image Gradient [Lee & Cok, IEEE TSP 1991] on gradients of vector field Use largest singular value of Jacobian matrix (DTI analogy: use λ1 vs MD) Example gradient image:

17. Constructing Image Graphs for Segmenting Lesions in Brain MRI 17 Communication between different levels in the hierarchy: Information Flow CSF A2 v2 1.Appearance parameters (mean, covar) 2.Atlas priors 1.Boundary adjustments 2.Split / merge atoms Appearance parameter adjustments

18. Constructing Image Graphs for Segmenting Lesions in Brain MRI 18 Object-Atom and Object-Voxel Interface Object passes down intensity parameters and atlas priors In atom level, image represented as flat patches Compute class posterior probabilities of each voxel and atom CSF A2 v2

19. Constructing Image Graphs for Segmenting Lesions in Brain MRI 19 Voxel-Atom Interface Change voxel grouping based on anatomy Possible adjustments: –Split/merge voxel groups –Boundary shift Split / merge not yet implemented (clustering) Boundary shift: –Every voxel in boundary between atoms get assigned to atom with nearest Kullback-Leibler (KL) distance –Simulate region competition (SNAP) CSF S2 v2

20. Constructing Image Graphs for Segmenting Lesions in Brain MRI 20 Atom-Object Interface Atom posteriors determine classification of every child voxel May have conflict between voxel and atom classication Resolve by adjusting global parameters Rationale: similar voxels must have the same classification, if not then need to accommodate violating voxels Currently implemented by adjusting covariance CSF S2 v2

21. Constructing Image Graphs for Segmenting Lesions in Brain MRI 21 Bias Correction MR images present intensity inhomogeneities or bias fields (“vignetting”) Bias corrected using polynomial fit Polynomial Fit

22. Constructing Image Graphs for Segmenting Lesions in Brain MRI 22 Method Summary CSF A2 v2 Bias Correction

23. Constructing Image Graphs for Segmenting Lesions in Brain MRI 23 Outline Background Methodology Results Conclusions and Future Work

24. Constructing Image Graphs for Segmenting Lesions in Brain MRI 24 Duke C1011A3 Depression Study Low contrast MRI

25. Constructing Image Graphs for Segmenting Lesions in Brain MRI 25 Voxel Only vs Hierarchical Classification Low tissue contast Duke C1011A3 data: FLAIRVoxel-onlyHierarchical

26. Constructing Image Graphs for Segmenting Lesions in Brain MRI 26 Duke C1011 voxel only hierarchical

27. Constructing Image Graphs for Segmenting Lesions in Brain MRI 27 Duke CRC-Oct04 (Aging/Depression)

28. Constructing Image Graphs for Segmenting Lesions in Brain MRI 28 Duke CRC

29. Constructing Image Graphs for Segmenting Lesions in Brain MRI 29 Multi-channel Segmentation T1wT2wFlairLabels Segmentation uses signature of all channels combined, using user-specified rules.

30. Constructing Image Graphs for Segmenting Lesions in Brain MRI 30 Outline Background Methodology Results Conclusions and Future Work

31. Constructing Image Graphs for Segmenting Lesions in Brain MRI 31 Conclusions Segmentation using hierarchical scheme Integrate top-down atlas-based approach and bottom-up data driven approach Segments small abnormal regions OK results on obvious high contrast lesions

32. Constructing Image Graphs for Segmenting Lesions in Brain MRI 32 Future Work Splitting / merging of atoms Improve classification scheme using non-parametric kernel densities Improve global parameter adjustment scheme Partial voluming? Tests/Adapt to lesions in NAMIC MIND DBP (lupus) Validation

33. Constructing Image Graphs for Segmenting Lesions in Brain MRI 33 Example NPSLE Lesion Hypointense on T1Hyperintense T2Hyperintense on FLAIR H Jeremy Bockholt, Charles Gasparovic The MIND Institute / UNM Albuquerque, NM