A Computationally Efficient Approach for 2D-3D Image Registration Juri Minxha Medical Image Analysis Professor Benjamin Kimia Spring 2011 Brown University

Review of Registration Similarity Metric Optimization 1. Similarity Metric Mutual Information, Cross-Correlation, Correlation Ratio, Cross Correlation Residual Entropy 2. Optimization, Non-gradient vs. Gradient Gauss-Newton, steepest descent, Levenberg-Larquardt, simplex method etc. The main challenge is: Minimize computation time Brown University

Review of the SoCV similarity metric 1.Similarity Measure: Sum of Conditional Variances 2.Optimization Algorithm: Gauss-Newton 1.Requires computation of gradient 2.Fast convergence Brown University

Similarity Metric: SoCV I0I0 RoRo R 0 =100·ln(256-I 0 ) Quantize images to 64 possible values 2.Each pixel in the image on the left corresponds to a bin in the histogram (64 x 64 bins) 3.Notice the non-linear relationship between I and R

Similarity Metric: SoCV What happens if I 0 is translated to the right? For each value of R, we have a range of values in I’

Similarity Metric: SoCV Compute the conditional expectation/mean of this distribution

Replace each value in R with the conditional mean Similarity Metric: SoCV

Optimization: Gauss-Newton Goal: Find values of 3D rigid-body transform that minimize S

Registration with CT, Fluoroscopy I) Last time: registration of MRI-MRI 2) This time: registration of CT and fluoroscopy 1. CT volume (512x512x369) of alligator bone 2. A fluroroscopy video (30 seconds, 30 FPS)

3D-visualization of CT data (Slicer) The big bone must be removed before projection!

Fluoroscopy video 1. ~30 seconds at 30 frames per second 2. The location of the bone is unknown throughout the video 3. Bone is being translated and Rotated 4. All artifacts must be removed before registration

Segmentation of smaller bone from CT Individual framesStacking the layers Thresholding to remove background and only Taking the frames that correspond to top bone

Ray-Casting: Generating the DRR Projecting the volume data onto the Three main axes.

Preparing the fluoroscopy image

Measure of Fit Optimization over three parameters: Rotation in x Rotation in y Rotation in z Manually adjusted three parameters: Translation in x Translation in y Translation in z

Shortcomings 1.There is no ground truth 2.Optimization was over a subset of the 6 parameters required for rigid body motion 3.The initial conditions are extremely important 1.The initial position has to be very close the final position 2.I manually translated the image, a procedure that should be automated 3.Speed optimization 4.Computational needs are excessive

References A computationally efficient approach for 2D-3D image registration Haque, M.N.; Pickering, M.R.; Biswas, M.; Frater, M.R.; Scarvell, J.M.; Smith, P.N.; 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Issue Date: Aug Sept , On page(s): 6268 – 6271 M. Pickering, A. Muhit, J. Scarvell, and P. Smith, "A new multimodal similarity measure for fast gradient-based 2D-3D image registration," in Proc. IEEE Int. Conf. on Engineering in Medicine and Biology (EMBC), Minneapolis, USA, 2009, pp Brown University