A New Method of Probability Density Estimation for Mutual Information Based Image Registration Ajit Rajwade, Arunava Banerjee, Anand Rangarajan. Dept. of Computer and Information Sciences & Engineering, University of Florida.

Image Registration: problem definition Given two images of an object, to find the geometric transformation that “best” aligns one with the other, w.r.t. some image similarity measure.

Mutual Information for Image Registration Mutual Information (MI) is a well known image similarity measure ([Viola95], [Maes97]). Insensitive to illumination changes; useful in multimodality image registration.

Marginal entropyJoint entropy Mathematical Definition for MI Conditional entropy

Calculation of MI Entropies calculated as follows: Joint Probability Marginal Probabilities

Joint Probability Functions of Geometric Transformation

Estimating probability distributions Histograms How do we select bin width? Too large bin width: Over-smooth distribution Too small bin width: Sparse, noisy distribution

Estimating probability distributions Parzen Windows Choice of kernel Choice of kernel width Too large: Over-smoothing Too small: Noisy, spiky

Estimating probability distributions Mixture Models [Leventon98] How many components? Difficult optimization in every step of registration. Local optima

Direct (Renyi) entropy estimation Minimal Spanning Trees, Entropic kNN Graphs [Ma00, Costa03] Requires creation of MST from complete graph of all samples

Cumulative Distributions Entropy defined on cumulatives [Wang03] Extremely Robust, Differentiable

A New Method What’s common to all previous approaches? Take samples Obtain approximation to the density More samples More accurate approximation

A New Method Assume uniform distribution on location Transformation Location Intensity Distribution on intensity Uncountable infinity of samples taken Each point in the continuum contributes to intensity distribution Image-Based

Other Previous Work A similar approach presented in [Kadir05]. Does not detail the case of joint density of multiple images. Does not detail the case of singularities in density estimates. Applied to segmentation and not registration.

A New Method Continuous image representation (use some interpolation scheme) No pixels! Trace out iso-intensity level curves of the image at several intensity values.

Analytical Formulation: Marginal Density Marginal density expression for image I(x,y) of area A: Relation between density and local image gradient (u is the direction tangent to the level curve):

Joint Probability

Analytical Formulation: Joint Density The joint density of images and with area of overlap A is related to the area of intersection of the regions between level curves at and of, and at and of as. Relation to local image gradients and the angle between them (and are the level curve tangent vectors in the two images):

Practical Issues Marginal density diverges to infinity, in areas of zero gradient (level curve does not exist!). Joint density diverges  in areas of zero gradient of either or both image(s).  in areas where gradient vectors of the two images are parallel.

Work-around Switch from densities (infinitesimal bin width) to distributions (finite bin width). That is, switch from an analytical to a computational procedure.

Binning without the binning problem! More bins = more (and closer) level curves. Choose as many bins as desired.

Standard histogramsOur Method 32 bins64 bins128 bins256 bins512 bins1024 bins

Pathological Case: regions in 2D space where both images have constant intensity

Pathological Case: regions in 2D space where only one image has constant intensity

Pathological Case: regions in 2D space where gradients from the two images run locally parallel

Registration Experiments: Single Rotation Registration between a face image and its 15 degree rotated version with noise of variance 0.1 (on a scale of 0 to 1). Optimal transformation obtained by a brute- force search for the maximum of MI. Tried on a varied number of histogram bins.

MI Trajectory versus rotation: noise variance 0.1 Standard Histograms Our Method 16 bins32 bins64 bins128 bins

MI Trajectory versus rotation: noise variance 0.8 Standard Histograms Our Method 16 bins32 bins64 bins128 bins

PD slice T2 slice Affine Image Registration BRAINWEB Warped T2 sliceWarped and Noisy T2 slice Brute force search for the maximum of MI

Affine Image Registration MI with standard histograms MI with our method

Directions for Future Work Our distribution estimates are not differentiable as we use a computational (not analytical) procedure. Differentiability required for non-rigid registration of images.

Directions for Future Work Simultaneous registration of multiple images: efficient high dimensional density estimation and entropy calculation. 3D Datasets.

References [Viola95] “Alignment by maximization of mutual information”, P. Viola and W. M. Wells III, IJCV [Maes97] “Multimodality image registration by maximization of mutual information”, F. Maes, A. Collignon et al, IEEE TMI, [Wang03] “A new & robust information theoretic measure and its application to image alignment”, F. Wang, B. Vemuri, M. Rao & Y. Chen, IPMI [BRAINWEB]

References [Ma00] “Image registration with minimum spanning tree algorithm”, B. Ma, A. Hero et al, ICIP [Costa03] “Entropic graphs for manifold learning”, J. Costa & A. Hero, IEEE Asilomar Conference on Signals, Systems and Computers [Leventon98] “Multi-modal volume registration using joint intensity distributions”, M. Leventon & E. Grimson, MICCAI 98. [Kadir05] “Estimating statistics in arbitrary regions of interest”, T. Kadir & M. Brady, BMVC 2005.

Acknowledgements NSF IIS NIH 2 R01 NS A2.

Questions??