MultiModality Registration Using Hilbert-Schmidt Estimators By: Srinivas Peddi Computer Integrated Surgery II April 6 th, 2001

Outline Brief description of my project Things that have been successful so far and some results of that Key problems left to overcome Discussion of deliverables & Conclusion

My Project T1 PD T2 I want to be able to register different modalities of MR images accurately. This means coming up with a new registration algorithm and getting around the intensity difference problem.

Original Deliverables Minimal goal was to get IntraModality Registration to work properly. Expected goal was to get MultiModality Registration working for at least two of the three modalities. Maximum goal was to get the algorithm to work for all three modalities and comparing its performance with the Mutual Information Algorithm (another MM algorithm).

Why is IntraModality Registration important? Since I am using a new method of registration, it needs to be validated first. The algorithm used here is called the Hilbert- Schmidt algorithm (duh!). It has been used before in Automatic Target Recognition but has never been applied in medical imaging.

What is Hilbert-Schmidt Registration? HS registration uses a minimum mean squared error (MMSE) estimator. The Hilbert-Schmidt norm is used as the error measure so the estimator is called the Hilbert-Schmidt estimator. Registration is achieved by finding the element of the special Euclidean group (SE n ) that minimizes the error. The estimator attains the lower bound of the registration error under this error metric.

IntraModality HS registration works! Using the Hilbert-Schmidt estimator, I produced code that registers images that look alike. I rotated the Proton Density image that I had by a given angle and then asked the algorithm to register the original image with the rotated image. It did! (Figure 1) Next, I added noise to both the images, and then asked for it to register them. It did! (Figure 2)

Figure 1 I asked the program to register these two images, and it came up with a transformation of 90 o which is correct.

Figure 2 The next step was to add noise to the image. I added Gaussian white noise (mean of 0) with a standard deviation of 20 to both the original images. The registration algorithm still gave a rotation of 90 o.

MultiModality Registration Now that we know that the HS registration algorithm works, we can move on to the real problem which is getting past the intensity difference problem. You may recall that in the first presentation I said that we would solve this problem using Bayesian Segmentation.

Brief Recap of Bayesian Segmentation CSF GRAY WHITE We create a histogram of number of voxels vs. intensity. We then fit the histogram with 3 Gaussians and the intersections of these Gaussians will be the thresholds. We can then relabel the image so that there are only 3 intensities (i.e. we get a segmentation).

Examples of the Segmentation BayesianSegmentation PD PD Segmentation The proton density image on the left was segmented using the Bayesian Segmentation algorithm described. As one can see, the segmented image has retained most of the geometrical features of the original image.

Examples of Segmentations Cont’d T1 T1 Segmentation As in the previous slide, the T1 image here was segmented using the Bayesian segmentation algorithm previously described. Again, we see that the segmentation preserved most of the geometrical features of this brain.

Switching the Intensity Values TissueT1T2PDCT White Matter Gray Matter CSF Skull These values are calculated on images that have 256 gray levels. The images were segmented manually and the average intensity value calculated for each tissue type. The tissue voxel intensities are highly dependent on the imaging parameters.

Changing the Segmentation T1 Segmentation Switched Original T1 Segmentation The switching was as follows: Original White  Black ; Original Black  White; Original Gray  Gray

Comparing the Segmentations Original PD Segmentation Transformed T1 Segmentation As one can see the two segmentations look fairly similar and it is now possible to register them. However, one may note the white skull area around the T1 segmentation, that is absent around the PD segmentation. This is because skull does not show up well in proton density images.

Recap The IntraModality registration has been successful and hence the minimal expectations of the project have been met. MultiModality Registration also works for at least registering T1 images with PD images (i.e. expected goal has been met). MultiModality Registration still needs to be done for the other two modality combinations and the Mutual Information algorithm still needs to be implemented (maximal goal).

Improvements to be made To handle the problem of the skull, try using a 4 peak segmentation. Perhaps use an algorithm to eliminate the partial volume area of the brain. Need to implement a ‘gradient descent’ algorithm to make the registration process more efficient.

Using Four-Peak Segmentation T1 If one uses a 4 peak segmentation, we can have different intensity values for four main things: CSF, Gray Matter, White Matter & Skull. Then we can remove the skull from the segmentation and then register the resultant image with the PD segmentation. This could possibly improve results.

Partial Volume Extraction What I have noticed is that most of the time, the segmentations differ most at the borders (partial volumes). Since we don’t really need all the voxels for the registration, we can throw out the voxels that are partial volume voxels. This could also possibly improve the registration process.

Gradient Descent Algorithm In the pdf above, we see that there is no probability that the rotation angle will be less than 150 degrees, or more than 210 degrees. In this case, it would be more beneficial to search the areas between 150 and 210 degrees more closely for the orientation that maximizes probability. We can do this using the gradient descent algorithm.

Gradient Descent Algorithm Cont’d Basically, we pick random points along the rotation axis and measure the probability that this is the correct rotation. Then we traverse the graph in the direction which increases the probability until we reach a maximum. The only thing that we need to ensure is that enough random points are picked so that we are not stuck in a local maxima.

Conclusion The minimum and expected goals of the project have basically been met. The maximum goal of this project still entails that the other two MM registrations need to work well. It also entails the coding of the Mutual Information algorithm. Even if this is achieved, the algorithm can still be enhanced in some ways to improve accuracy and efficiency further.