MultiModality Registration using Hilbert-Schmidt Estimators By: Srinivas Peddi Computer Integrated Surgery II Johns Hopkins University

Outline Brief description of my project One of the key problems present and how segmentation can help Detailed description of a Bayesian segmentation algorithm Conclusion

What is MultiModality Registration? Registration is simply the process of geomet-rically aligning images using translations and rotations. Multimodality simply means that the registration will be done over different imaging formats (MRI, CT, PET, SPECT, etc.)

Aim of My Project The goal is to be able to register 2D image slices from different modalities of MR (T1, T2 , proton density images) The problem is that white matter, gray matter, CSF, etc. show up very differently in the various scans.

Example T1 PD T2 The three images above are the same slice of the same person. However, as is obvious, WM, GM and CSF show up very differently in the various scans. White matter of the brain shows up as white on T1 scans, light gray on T2 scans and as dark gray on PD scans.

Solution to the problem Segmentation is the process by which we can classify complicated images. As discussed in CIS I, there are many types of segmentations, but the one we will focus on is Bayesian Segmentation.

Key Papers for this Presentation Brain Segmentation and the Generation of Cortical Surfaces by Joshi et al (NeuroImage, December 1998) Bayesian Construction of Geometrically Based Cortical Thickness Metrics by Miller et al (NeuroImage, November 2000) Validating Cortical Surface Analysis of Medial Prefrontal Cortex by Ratnanather et al (in review)

What is Bayesian Segmentation? The image on the left is a typical MR (mprage format) slice and the image on the right is after it has been segmented. As one can see, the image on the left has many different intensity values while the image on the right has only three unique values.

How does the algorithm work? For 8-bit images, each voxel can take on 255 values (x-axis). The program determines how many voxels there are of each intensity (blue). Next, the program tries to fit 3 Gaussian curves to the histogram (green line). The intersections of the green lines are called thresholds. Voxel Number Intensities

How does the algorithm work (cont’d) Once we have the thresholds (marked by white lines), we can then classify voxels of different intensities as white matter, gray matter or CSF. The important thing to note is that in this algorithm, the classification of regions is based solely on the intensities of the voxels and that the features of the geometry of the image are not taken advantage of. This is something that could possibly be improved. CSF GRAY WHITE

Example of Segmentations However, the above picture shows that the segmentation algorithm did fairly well in correctly labeling the WM area in MPRAGE images. (Everything surrounded by the red line is considered WM by the segmentation algorithm.)

Measuring Accuracy Rigorously: L1 Distances The gold standard in segmentation so far is what is known as hand segmentation where a trained technician actually goes out and draws the boundaries for the various regions. For a set of 10 brains, the hand segmentation was compared to the Bayesian segmentation using the equation below. Essentially, all that is being done is that we are counting the number of voxels where the segmentations disagree and normalizing that value by the total number of voxels.

Validation: L1 distance between Bayesian and hand segmentations 0.01 0.02 0.001 0.03 SD 0.10 0.09 0.12 0.11 0.07 Average 0.08 0.14 0.13 4 0.05 3 0.06 2 1 s2007 s2006 s2004 s2003 s2002 s1013 s1010 s1009 s1003 s1002 Rater The table shows that regardless of the min and max values and regardless of the initial conditions, the algorithm produces a segmentation that is fairly similar to the hand segmentation.

Why this is ground-breaking? Using a protocol such as this one, we are able to segment images very quickly. An average MR volume can have up to 270 slices and it could take a trained expert up to a day to segment it. Using an algorithm like this one, it takes less than a minute. The other advantage is consistency. Different people are at different skill levels and things like fatigue can also cause a person to segment something inaccurately. With this approach, not much technical knowledge is required by the end user and also very consistent results are produced.

How will I incorporate Bayesian segmentation into my project? The first step in my MultiModality registration protocol will be to segment slices into tertiary images (WM, GM, CSF) using known information of that modality. The next step would then be to register the segmentations with each other and apply our solution transformation matrix on the original images.

Conclusions Bayesian segmentation isn’t a perfect art and improvements can definitely be made. However, it has definite advantages in that the algorithm is fast and it offers us a highly consistent way of segmenting images. Without this high degree of consistency, segmentation could never be applied for registration problems.