1. DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical Imaging Department of Radiology, MGH Harvard Medical School, USA Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology, USA

2. MRI of the brain Magnetic resonance imaging: – Harmless – Three dimensional (3-D) – High soft tissue contrast – High spatial resolution – Extremely versatile – Possibly multi-spectral Ideal for studying the living human brain “voxel” Koen Van Leemput DTU Medical Visionday May 27, 2009

3. Segmentation of brain MRI – Delineating structures of interest in the images – Segmentation is important: Basic neuroscience Uncovering disease mechanisms Diagnosis, treatment planning, and follow-up Clinical drug trials … – Automated computational methods are needed Koen Van Leemput DTU Medical Visionday May 27, 2009

4. Overview Segmentation basics: modeling and inference Modeling MRI bias fields Mesh-based brain atlases Whole-brain segmentation Koen Van Leemput DTU Medical Visionday May 27, 2009

5. Overview Segmentation basics: modeling and inference Modeling MRI bias fields Mesh-based brain atlases Whole-brain segmentation Koen Van Leemput DTU Medical Visionday May 27, 2009

6. MRI image The problem to be solved Koen Van Leemput DTU Medical Visionday May 27, 2009

7. MRI image The problem to be solved Koen Van Leemput DTU Medical Visionday May 27, 2009 Label image

8. One solution: generative modeling – Formulate a statistical model of how an MRI image is formed – The model depends on some parameters “labeling model” “imaging model” Koen Van Leemput DTU Medical Visionday May 27, 2009 MRI image Label image

9. MRI image Segmentation = inverse problem Koen Van Leemput DTU Medical Visionday May 27, 2009 Label image

10. MRI image Segmentation = inverse problem Koen Van Leemput DTU Medical Visionday May 27, 2009 Label image Bayesian inference – Start from our statistical model of image formation – Play with the mathematical rules of probability

11. Practical approximation Involves two optimizations: – First estimate the optimal model parameters – Then find the optimal segmentation based on those parameter estimates Bayesian inference Koen Van Leemput DTU Medical Visionday May 27, 2009

12. Example: Gaussian mixture model – The label in each voxel is drawn independently with a probability for tissue type k – Assume a uniform prior for the labeling model parameters “labeling model” “imaging model” Koen Van Leemput DTU Medical Visionday May 27, 2009 MRI image Label image

13. Example: Gaussian mixture model – The intensity in each voxel is drawn independently from a Gaussian distribution associated with its label – The imaging model parameters are the mean and variance of each Gaussian: – Assume a uniform prior “labeling model” “imaging model” Koen Van Leemput DTU Medical Visionday May 27, 2009 MRI image Label image

14. Model parameters are unknown Example: Gaussian mixture model Koen Van Leemput DTU Medical Visionday May 27, 2009 Mean and variance of each Gaussian Relative weight of each Gaussian three labels

15. Optimization 1: parameter estimation – Given an MRI image to be segmented, what is the MAP parameter estimate ? – Parameter optimization with an Expectation Maximization (EM) algorithm current estimate –Repeatedly maximize a lower bound to the objective function –Iterative parameter optimizer using only closed- form parameter updates! Koen Van Leemput DTU Medical Visionday May 27, 2009

16. Optimization 1: parameter estimation

17. Koen Van Leemput DTU Medical Visionday May 27, 2009 Optimization 1: parameter estimation

18. Optimization 2: segmentation white matter gray matter CSF Koen Van Leemput DTU Medical Visionday May 27, 2009 Upon completion of the parameter estimation algorithm, assign each voxel to the MAP label

19. Overview Segmentation basics: modeling and inference Modeling MRI bias fields Mesh-based brain atlases Whole-brain segmentation Koen Van Leemput DTU Medical Visionday May 27, 2009

20. MRI bias field artifact Intensity inhomogeneities across the image area Imaging artifact in MRI equipment limitations patient-induced electrodynamic interactions Koen Van Leemput DTU Medical Visionday May 27, 2009 MRI data after intensity windowing…

21. MRI bias field artifact Causes segmentation errors with our segmentation procedure so far… Koen Van Leemput DTU Medical Visionday May 27, 2009

22. MRI bias field artifact Koen Van Leemput DTU Medical Visionday May 27, 2009 Causes segmentation errors with our segmentation procedure so far…

23. Improved imaging model “labeling model” “imaging model” Koen Van Leemput DTU Medical Visionday May 27, 2009 MRI image Label image

24. Improved imaging model “labeling model” “imaging model” old model Koen Van Leemput DTU Medical Visionday May 27, 2009 MRI image Label image

25. Improved imaging model “labeling model” “imaging model” + old model polynomial bias field model Koen Van Leemput DTU Medical Visionday May 27, 2009 MRI image Label image

26. Model parameter estimation – Polynomial coefficients are part of the model parameters – Parameter optimization with a Generalized Expectation Maximization (GEM) algorithm current estimate –Repeatedly improve a lower bound to the objective function –Iterative parameter optimizer using only closed- form parameter updates! [Van Leemput et al., IEEE TMI 1999] Koen Van Leemput DTU Medical Visionday May 27, 2009

27. Example Koen Van Leemput DTU Medical Visionday May 27, 2009 MRI data Estimated bias field Bias-corrected MRI data

28. Example MRI data White matter without bias field model White matter with bias field model Estimated bias field Koen Van Leemput DTU Medical Visionday May 27, 2009

29. Example MRI data White matter without bias field model White matter with bias field model Estimated bias field Koen Van Leemput DTU Medical Visionday May 27, 2009

30. Overview Segmentation basics: modeling and inference Modeling MRI bias fields Mesh-based brain atlases Whole-brain segmentation Koen Van Leemput DTU Medical Visionday May 27, 2009

31. Improving the labeling model “labeling model” “imaging model” Koen Van Leemput DTU Medical Visionday May 27, 2009 – So far our labeling model just expresses the relative frequency of occurrence of different labels – Too simplistic for segmenting the brain into 30+ subregions A more realistic labeling model is needed! MRI image Label image

32. Improving the labeling model Koen Van Leemput DTU Medical Visionday May 27, 2009

33. Improving the labeling model Try to find the underlying probability distribution Manual segmentations in N individuals (training data) Koen Van Leemput DTU Medical Visionday May 27, 2009

34. Modeling the training data (2-D) Triangular mesh representation Koen Van Leemput DTU Medical Visionday May 27, 2009

35. Modeling the training data (2-D) Assign label probabilities to each mesh node Flat prior Label probabilities are linearly interpolated over triangle areas “atlas” Koen Van Leemput DTU Medical Visionday May 27, 2009

36. Modeling the training data (2-D) Mesh node positions are sampled from a topology-preserving Markov random field prior “atlas” warped atlases Koen Van Leemput DTU Medical Visionday May 27, 2009 “knob” that controls the flexibility of the atlas warp

37. Modeling the training data (2-D) Example segmentations are sampled according to the deformed atlases atlas warped atlases example segmentations Koen Van Leemput DTU Medical Visionday May 27, 2009

38. Bayesian inference [Van Leemput, IEEE TMI 2009] Given a collection of manual segmentations – what is the most probable atlas? – what is the most likely value of the parameter controlling the flexibility of the deformations? – what is the most likely mesh representation? Good models explain regularities in the manual segmentations – Automatically yields sparse representations that explicitly avoid overfitting to the training data – cf. Minimum Description Length Koen Van Leemput DTU Medical Visionday May 27, 2009

39. Example atlas Koen Van Leemput DTU Medical Visionday May 27, 2009 Derived from manual segmentations of 36 brain substructures in 4 individuals Has average “shape”

40. Overview Segmentation basics: modeling and inference Modeling MRI bias fields Mesh-based brain atlases Whole-brain segmentation Koen Van Leemput DTU Medical Visionday May 27, 2009

41. Whole-brain segmentation – Tetrahedral mesh-based atlas – The labeling model parameters are the location of the mesh nodes – The prior is the topology-preserving MRF model (penalizes deformations) “labeling model” “imaging model” Koen Van Leemput DTU Medical Visionday May 27, 2009 MRI image Label image

42. Whole-brain segmentation “labeling model” “imaging model” + Gaussian mixture model polynomial bias field model Koen Van Leemput DTU Medical Visionday May 27, 2009 MRI image Label image

43. – Model parameter estimation: – Fully automated segmentation procedure No need for pre-processing (skull stripping, bias field corr., …) Automatically adapts to different scanners and acquisition sequences! Fast! Whole-brain segmentation Improve the imaging model parameters (Generalized Expectation-Maximization; closed-form expressions) Improve the atlas warp (registration; gradient in analytical form) Koen Van Leemput DTU Medical Visionday May 27, 2009

44. Examples (validation under way) Koen Van Leemput DTU Medical Visionday May 27, 2009

45. Examples (validation under way) Koen Van Leemput DTU Medical Visionday May 27, 2009

46. Examples (validation under way) Koen Van Leemput DTU Medical Visionday May 27, 2009

47. Examples (validation under way) Koen Van Leemput DTU Medical Visionday May 27, 2009

48. Examples (validation under way) Koen Van Leemput DTU Medical Visionday May 27, 2009

49. Examples (validation under way) Koen Van Leemput DTU Medical Visionday May 27, 2009

50. Thanks! Koen Van Leemput DTU Medical Visionday May 27, 2009