1. Automatic segmentation of brain structures
Alireza Akhondi-Asl
Boston Children’s Hospital, and Harvard Medical School
300 Longwood Ave. Boston MA USA

2. Table of Contents Introduction Methods Results Some Applications
Definition
Why we need segmentation
Problems of manual segmentation
Challenges of automatic segmentations
Literature review
Methods
Results
Some Applications
Conclusion

3. Introduction Segmentation Visual assessment of images
Identification of structure in images.
Visual assessment of images
Head position 2D
Longitudinal changes
Quantitative manual efforts
Time Consuming
Sometimes we do not have an expert for our application
Inter/Intra rater variability
Automated segmentation algorithms are thus of special interest.

4. Introduction Problems of Automatic Segmentation: Tissue Inhomogenity
Partial Volume Effect
Noise and Artifact
Various Imaging Modalities
Changes in the Structures Due to Diseases
Modeling Limitations
Number of parameters
Complexity of the algorithm

5. Introduction Applications: Quantitative image analysis
Abnormality Detection
Cortical gray matter thickness
Gray-matter/white-matter blurring
Lesion detection
MS, Tumor
Volumetry
Longitudinal Analysis
Image guided therapy
Visualization

6. Segmentation Parts
Many different algorithms and a wide range of principles upon which they are based. However, they all have these three key parts:
Energy Function
Shape Representation
Optimization

7. Segmentation Methods Shape Based Parametric Snake, Balloon,…
Non-Parametric
Level Set
Clustering
Probabilistic
Template Based
How they work:
Prior Knowledge
Intensity Statistics
Shape Statistics
Location
Relationship
Shape
Relative Position
Intensity
Intersection
Symmetry

8. Evaluation (Spatial overlap)

9. Evaluation

10. Multiple-Template-Based Segmentation

11. Fusion Why we need multiple templates?
Aligned label map of a template is not perfectly representing the anatomy of the target image:
Registration errors
Registration: Maximization of the local intensity similarity of the two images in the space of the topology preserving maps.
Topological differences between the anatomies of the template and the target image.
Sub-optimal selection of the registration parameters.
Template error
Template segmentation is the output of a manual or an automatic segmentation procedure:
Systematic and random errors
To get the perfect fusion result:
We need to have an appropriate template population.
We need to know the performance of the templates.
Since the performances are not a priori, we need to estimate the performance of the templates.

12. Registration Errors

13. Template Errors

14. Multiple Template Segmentation
Registration
Non-rigid registration
Template Selection
It reduces the computation time and also may increase the quality
Intensity Based
Transform Based
Fusion
Majority voting
Weighted majority voting
STAPLE
Post-Processing
Active Contours
Graph Cut

15. STAPLE STAPLE and its extensions are Bayesian algorithms.
STAPLE (Simultaneous Truth and Performance Level Estimation):
An algorithm for estimating performance and ground truth from a collection of independent segmentations.
Simultaneous estimation of hidden ``ground truth’’ and local or global expert performance using intensity and/or label information.
Enables comparison between and to experts.
Expectation-Maximization
General procedure for estimation problems that would be simplified if some missing data was available.
Other extensions
STAPLE MAP
Probabilistic STAPLE
LOP STAPLE
Empirical Bayesian Based STAPLE
LOCAL STAPLE MAP
STAPLE and its extensions are Bayesian algorithms.

16. Algorithm Description
True segmentation Ti for each voxel i
May be binary
May be categorical
Expert j makes segmentation decisions Dij
Expert performance s’s characterizes probability of deciding label s’ when true label is s.

17. STAPLE STAPLE Posterior probability of reference segmentation
Prior Probability distribution of the reference segmentation
Performance parameters of the raters

18. Intensity information
When there is an anatomical difference between the target image and a template, it is more likely to have:
A mismatch between the segmentation labels of the two images.
Some intensity dissimilarity between the two images.
It is sensible to use the intensity information of the templates and the target image as a source of information in the fusion process.
Intensity information can be used in a variety of ways as a source of information in the STAPLE formulation:
Probabilistic STAPLE: The segmentations can be modified based on intensity information.
LOP-STAPLE: The intensity similarity can be used as a reliability weight in the decision process.
MAP STAPLE: The intensity information can be used as the prior information.

19. Probabilistic STAPLE (PSTAPLE)
We propose modifying each one of the aligned template segmentations to correct errors due to the uncaptured anatomical differences between the target image and the templates.
Each one of the trained classifiers is used to segment the target image. The output of each one of the classifiers is a probabilistic segmentation of the target image.
Train
Segment
Probabilistic STAPLE
Train
Segment

20. Probabilistic STAPLE

21. LOP-STAPLE
In this model, we assume that the rater decision at each voxel is associated with a reliability weight where the higher weights indicate higher credibility of the decisions.
We use logarithmic opinion pool (LOP) to combine the weighted decisions.
LOP is a model where the linear weighted average of the logarithm of the rater probabilities is used as the aggregating method.
Local Intensity Similarity
LOP
STAPLE
Local Intensity Similarity

22. LOP-STAPLE

23. MAP STAPLE
A MAP formulation of the STAPLE algorithm which uses Dirichlet distribution to model the prior probability of the performance parameters of the templates.
This MAP formulation has a closed form solution. This significantly increases the speed of the algorithm compared to the previous approach.
We estimate the parameters of the prior distributions based on the intensity and label information of the templates and the target image.
Estimate MAP
Parameters
MAP
STAPLE
Estimate MAP
Parameters

24. Dirichlet Distribution
Dirichlet distribution is the multivariate generalization of the Beta distribution which can be used to model probabilities.
It has all benefits of the Beta distribution such as capability to model any prior on the parameters and easy computation of the logarithm and the derivatives.

25. MAP STAPLE

26. Empirical Bayesian Based STAPLE
E-STEP:
M-STEP:
The estimation of the prior probability of reference standard at each step is the estimated posterior probability of reference standard at the previous step.

27. Prior in Bayesian Analysis
In the standard Bayesian analysis the prior distribution is assumed to be known and the observations do not re-scales the prior distribution.
To improve the fusion performance, we use a novel framework to estimate the prior using a parametric empirical Bayesian procedure.
We assume that prior probability of reference standard is another set of unknown parameters and to solve the problem, we use the EM algorithm to iteratively estimate the ground truth, the performance parameters, and the prior distribution of the hidden ground truth.

28. Local STAPLE In this way expert’s performance is computed locally
If we apply STAPLE algorithm for each voxel independently, then the problem reduces to the majority voting.
We have two extreme approaches for performance evaluation:
- STAPLE
- Majority Voting
Is there any other way?
- LOCAL STAPLE
For each voxel in the image:
Use its neighbor and define an ROI
Consider the same ROI for all of experts segmentations
Apply STAPLE based on this ROI.
Combine the results.
In this way expert’s performance is computed locally

29. Results
We have evaluated our method and some of the state-of-the-art fusion algorithms for the automatic segmentation Two MRI datasets.
IBSR: 18 Subjects, 34 gray and white matter structures and 96 structures in Cerebral Cortex.
NMM: 15 Subjects, 53 gray and white matter structures and cortical gray matter lobes.
We have used the leave-one-out strategy for the evaluation.
ANTS was chosen for the alignment of the templates to the target images.
We have used the fusion algorithm for each structure independently and compared the fusion results.

30. RESULTS (PSTAPLE)
Quantitative Comparison of the IBSR Multi-Atlas Segmentation Results. Comparison of generalized Dice coefficient of M12 Local MAP STAPLE and M13 : Local MAP PSTAPLE for structures in 18 IBSR datasets. It indicates that Local MAP PSTAPLE is superior to the Local STAPLE. The horizontal axis represents each subject.

31. RESULTS (PSTAPLE)
Comparison of Dice coefficients for the proposed methods. Results of the segmentation of 16x2 +1=33 sample structures and average of the 128 brain structures of the IBSR dataset for M_0:Majority Voting, M_1: Artaechevarria et al. method, M_2: Sabuncu et al. algorithm, M_3: SIMPLE, M_4: STAPLER, M_5: COLLATE, M_6: STAPLE , M_7: Multi-STEPS, M_8: NLS, M_9: PICSL-MALF, M_10: Probabilistic Voting, M_11:PSTAPLE.

32. RESULTS (PSTAPLE)
Quantitative Comparison of the IBSR Multi-Atlas Segmentation Results. Comparison of generalized Dice coefficient of M13: Local MAP PSTAPLE, M1 : method of Artaechevarria et al., M2 : method of Sabuncu et al., M7 : Multi-STEPS, M8 : NLS, and M9 : PICSL-MALF for structures in 18 IBSR datasets. It indicates that Local MAP PSTAPLE is superior to the other methods. The horizontal axis represents each subject.

33. RESULTS (PSTAPLE) on IBSR Data

34. RESULTS (PSTAPLE)
Quantitative Comparison of the NMM Multi-Atlas Segmentation Results. Comparison of generalized Dice coefficient of M13: Local MAP PSTAPLE, M1 : method of Artaechevarria et al., M2 : method of Sabuncu et al., M7 : Multi-STEPS, M8 : NLS, and M9 : PICSL-MALF for structures in 15 NMM datasets. It indicates that Local MAP PSTAPLE is superior to the other methods. The horizontal axis represents each subject.

35. RESULTS (PSTAPLE) On NMM Data

36. Applications (Epilepsy Surgery)
Brain parcellation helps neurophysiologists recognize the functional/structural areas where the electrodes are placed

37. Applications (Fetal MRI)
Volumetric fetal brain MRI reconstruction and automatic segmentation allows 3D visualization of the ventricles which is quite useful in the analysis of ventriculomegaly. Marching cubes surface model rendering of fetal ventricles over a transparent surface model rendering of the intracranial volume (ICV) based on volumetric fetal brain MRIeas where the electrodes are placed.

38. Applications (Functional Connectivity)
Brain
Segmentation
Multiple-Atlas-Based Segmentation
Pre-Processing of rsfMRI
Building The Connectivity Matrix
Graph Analysis
rsfMRI Preprocessing
Connectivity Matrix
Graph
Analysis

39. Conclusions
Intensity information can help us to improve the segmentation accuracy.
Intensity information in the STAPLE framework:
Probabilistic STAPLE: The segmentations can be modified based on intensity information.
Weighted STAPLE: The intensity similarity can be used as a reliability weight in the decision process.
MAP STAPLE: The intensity information can be used as the prior information.
The prior has an important role in determining the local optimum to which the EM algorithm converges.
Our new algorithm is robust and has superior performance as compared to the other methods.

40. References
Akhondi-Asl A, Hoyte L, Lockhart ME, Warfield SK. A Logarithmic Opinion Pool Based STAPLE Algorithm for The Fusion of Segmentations With Associated Reliability Weights, IEEE Transactions on Medical Imaging. 2014, 33(10),
Siversson C, Akhondi-Asl A, Bixby S, Kim YJ, Warfield SK. Three-dimensional hip cartilage quality assessment of morphology and dGEMRIC by planar maps and automated segmentation, Osteoarthritis and Cartilage. 22(10), , 2014.
Akhondi-Asl A, Warfield SK. Simultaneous Truth and Performance Level Estimation through Fusion of Probabilistic Segmentations. IEEE Transactions on Medical Imaging, 2013.
Commowick O, Akhondi-Asl A, Warfield SK. Estimating A Reference Standard Segmentation with Spatially Varying Performance Parameters: Local MAP STAPLE, IEEE Transactions on Medical Imaging, 2012.
Gholipour A, Akhondi-Asl A, Estroff J A, Warfield SK. Multi-atlas multi-shape segmentation of fetal brain MRI for volumetric and morphometric analysis of ventriculomegaly. Neuroimage, 2012.
Akhondi-Asl A, Warfield SK. Estimation of the Prior Distribution of Ground Truth in the STAPLE Algorithm: An Empirical Bayesian Approach. MICCAI 2012.
Akhondi-Asl A, Jafari-Khouzani K, Elisevich K, Soltanian-Zadeh H. Hippocampal volumetry for lateralization of temporal lobe epilepsy: Automated versus manual methods. Neuroimage, 2011.

41. Thank you!