1. Medical Image Registration
Yujun Guo
Dept.of CS
Kent State University

2. Outline Why registration Registration basics Rigid registration
Non-rigid registration
Applications

3. Modalities in Medical Image
Computed Tomography (CT), Magnetic Resonance (MR) imaging, Ultrasound, and X-ray give anatomic information.
Positron Emission Tomography (PET) and Single Photon Emission CT (SPECT) give functional information.

4. Registration Monomodality: Multimodality:
A series of same modality images (CT/CT, MR/MR, Mammogram pairs,…).
Images may be acquired weeks or months apart; taken from different viewpoints.
Aligning images in order to detect subtle changes in intensity or shape
Multimodality:
Complementary anatomic and functional information from multiple modalities can be obtained for the precise diagnosis and treatment.
Examples:PET and SPECT (low resolution, functional information) need MR or CT (high resolution, anatomical information) to get structure information.

5. Registration Problem Definition
q = (912,632)
q = T(p;a)
p = (825,856)
Homologous pixel location in second image
Pixel location in first image
Pixel location mapping function

6. Example Mapping Function
q = (912,632)
p = (825,856)
Pixel scaling and translation

7. Image Registration
Define a transform T that will map one image onto another image of the same object such that some image quality criterion is maximized.
A mapping between two images both spatially and with respect to intensity
I2 = g (T(I1))

8. Registration Scheme

9. Components Feature Space Search Space or transformation
Similarity Metric
Search Strategy
There are plenty of publications dedicated to medical image registration. Various techniques, models, algorithms and clinical applications are described and reviewed. The aspects of image registration may differ greatly among various approaches, but an underlying framework is common to all techniques. Registration methods can be viewed as different combinations of choices for the following four components.

10. Feature Space Geometric landmarks: Points Edges Contours
Surfaces, etc.
Intensities:
Raw pixel values
Feature space extracts the information in the images to be used for the matching. It can be a set of matching targets, such as…
Feature-based
Intensity-based 35 56

11. Image transformations
Input
image
Output
Rigid transformation
Original
shape
Affine transformation
Homogeneous coordinates
Polynomial transformations
Rigid
Non-rigid

12. Similarity Metric Absolute difference SSD (Sum of Squared Difference)
Correlation Coefficient
Mutual Information / Normalized Mutual Information
Similarity metric determines the relative merit for each test.

13. Search Strategy Powell’s direction set method Downhill simplex method
Dynamic programming
Relaxation matching
Hierarchical techniques
Search Strategy, or optimization algorithm, decides how to choose the next transformation from this space, to be tested in the search for the optimal transformation. According to the similarity measure.
Powell and Downhill are used when local distortion is not present, and the global transformation can be put into one function.
Dynamic programming is used to register images where a local transformation is needed.
Relaxation matching is most often used when a global transformation is needed, but local distortion is present.
Hierarchical techniques can speed up different approaches by guiding search strategy through progressively finer solutions.

14. Multi-modality Brain image registration
Intensity-based
3D/3D Rigid transformation, DOF=6 (3 translations, 3 rotations)
Maximization of Normalized Mutual Information
Simplex Downhill
Multi-resolution
Dataset: Vanderbilt University

15. Mutual Information as Similarity Measure
Mutual information is applied to measure the statistic dependence between the image intensities of corresponding voxels in both images, which is assumed to be maximal if the images are geometrically aligned.

16. Normalized Mutual Information
Extension of Mutual Information
Maes et. al.:
Studholme et. Al.:
Compensate for the sensitivity of MI to changes in image overlap

17. Geometry Transformation
Image Coordinate transform:
The features (dimension, voxel size, slice spacing, gantry tilt, orientation) of images, which are acquired from different modalities, are not the same.
From voxel units (column, row, slice spacing) to millimeter units with its origin in the center of the image volume.

18. Target Image & Template Image
Target Image Grid j i
Template Image Grid j i y x
Target Image Physical Coordinates y’ x’
Template Image Physical Coordinates
Space Transform

19. Images from the same patient
256 x 256 pixels
MRI-T2
128 x 128 pixels
PET
Target Image ?
Template Image ?
Images provided as part of the project: “Retrospective Image Registration Evaluation”, NIH, Project No. 8R01EB , Principal Investigator, J. Michael Fitzpatrick, Vanderbilt University, Nashville, TN.

20. Interpolation Nearest Neighbor Tri-linear Interpolation
Partial-Volume Interpolation
Higher order partial-volume interpolation

21. Evaluating similarity measure for each transformationx x
Template Image
Target Image

22. Optimization
Powell’s Direction Set method
Downhill Simplex method

23. Multi-resolution Why Multi-resolution
Methods for detecting optimality can not guarantee that a global optimal value will be found.
Time to evaluate the registration criterion is proportional to the number of voxels.
The result at coarser level is used as the starting point for the finer level.
Currently multi-resolution approaches:
Sub-sampling
Averaging
Wavelet

24. Registration Result (I)
A typical superposition of CT-MR images.
Left : before registration Right: after registration.

25. Rigid transformation (II)
A typical superposition of MR-PET images.
Left : before registration Right: after registration.

26. Mammography
Breast cancer is the second leading cause of death among women in USA.
Detected in its early stage, breast cancer is most treatable.
Mammography is the main tool for detection and diagnosis of breast malignances.
It reduces breast cancer mortality by 25% to 30% for women in the 50 to 70 age group

27. Mammogram Registration
Temporal/bilateral mammograms vary
Breast compression
Breast position
Imaging Technique
Change in Breast

28. Mammogram registration techniques
Whole breast area vs. regional
Nipple location
Control-point location
Rigid & non-rigid registration

29. Non-rigid Mammogram Registration
Intensity-based
Elastic transformation
Multi-resolution
Demons algorithm (Thirion, 1996)

30. Demons
Transform
Scene (Target)
Model (Template)

31. Demons (Cont.)
Transform
Scene
Forces
Model

32. Demons (Cont.) Current Estimation Intensity Space Gradient Desired
Displacement
Scene

33. Demons From Optical Flow Scene: f, Model: g
Assumption: The intensity of a moving object is constant with time
(1)
One model image: G
One scene image: F
Fit model to the scene
One solution to the first equation is to consider that…
Leads to equation 2
(g-f)^2 is added to make the equation stable when ||->0
(2)

34. Description of the Approach
Select demon points.
Compute the force u on the model at each of the selected demons
Determine a global transformation based on the computed u and apply it to the model
If the model images is now registered to the scene image, stop. Else, go to Step 2.
The description of the demon-based approach is given here.

35. Registration Components
Image Intensities
Non-rigid transformation, one displacement vector for each pixel
Bilinear interpolation
Absolute difference as similarity metric
Multi-resolution
Dataset: MIAS,DDSM

36. Demons Results (I) Synthetic Images
Template: image T
Target: Image C
The second experiment is designed to test the performance of the multi-resolution techniques.
Level=2
Level=3
Level=5
Level=4

37. Demons Result (II) MIAS
Original images
Before registration
After rigid registration
After non-rigid registration

38. Ongoing registration topics
Trade-off of computation and accuracy
Evaluation of registration results
Visualization of registration

39. Applications: Change Detection
Images taken at different times
Following registration, the differences between the images may be indicative of change
Deciding if the change is really there may be quite difficult

40. Other Applications
Multi-subject registration to develop organ variation atlases.
Used as the basis for detecting abnormal variations
Object recognition - alignment of object model instance and image of unknown object (segmentation)

41. References
Maes F,Collignon A, et al. “Multimodality image registration by maximization of mutual information.” IEEE Trans. Med. Imaging. 1997, V16,pp
L.G.Brown, “A survey of image registration techniques,” ACM Computing Surveys, vol. 24, no. 4, pp. 325–376, 1992.
Jean-Philippe Thirion, “Non-Rigid Matching Using Demons,” IEEE Conference on Computer Vision and Pattern Recognition,1996