Mutual Information Based Registration of Medical Images

Slides:



Advertisements
Similar presentations
Image Registration  Mapping of Evolution. Registration Goals Assume the correspondences are known Find such f() and g() such that the images are best.
Advertisements

Mutual Information Based Registration of Medical Images
Gordon Wright & Marie de Guzman 15 December 2010 Co-registration & Spatial Normalisation.
Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image.
Pattern Recognition and Machine Learning
Chapter 3 Image Enhancement in the Spatial Domain.
OverviewOverview Motion correction Smoothing kernel Spatial normalisation Standard template fMRI time-series Statistical Parametric Map General Linear.
Coregistration and Normalisation By Lieke de Boer & Julie Guerin.
Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović
Automatic Feature Extraction for Multi-view 3D Face Recognition
Instructor: Mircea Nicolescu Lecture 13 CS 485 / 685 Computer Vision.
Measures of Information Hartley defined the first information measure: –H = n log s –n is the length of the message and s is the number of possible values.
Surface Reconstruction from 3D Volume Data. Problem Definition Construct polyhedral surfaces from regularly-sampled 3D digital volumes.
1 Improving Entropy Registration Theodor D. Richardson.
Non-Rigid Registration. Why Non-Rigid Registration  In many applications a rigid transformation is sufficient. (Brain)  Other applications: Intra-subject:
Motion Analysis (contd.) Slides are from RPI Registration Class.
CSci 6971: Image Registration Lecture 4: First Examples January 23, 2004 Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware Prof. Chuck Stewart, RPI Dr.
Mutual Information for Image Registration and Feature Selection
Yujun Guo Kent State University August PRESENTATION A Binarization Approach for CT-MR Registration Using Normalized Mutual Information.
Image Registration Narendhran Vijayakumar (Naren) 12/17/2007 Department of Electrical and Computer Engineering 1.
Medical Image Registration
Preprocessing II: Between Subjects John Ashburner Wellcome Trust Centre for Neuroimaging, 12 Queen Square, London, UK.
FMRI Preprocessing John Ashburner. Contents *Preliminaries *Rigid-Body and Affine Transformations *Optimisation and Objective Functions *Transformations.
July 2010 Image Registration Techniques, Benchmarking, Strategy Surgical Planning Laboratory Center for Neurological Imaging July 2010 Lidwien Veugen Supervision.
Co-registration and Spatial Normalisation
National Alliance for Medical Image Computing Registration in Slicer3 Julien Jomier Kitware Inc.
Digital Image Processing Lecture 7: Geometric Transformation March 16, 2005 Prof. Charlene Tsai.
Texture. Texture is an innate property of all surfaces (clouds, trees, bricks, hair etc…). It refers to visual patterns of homogeneity and does not result.
Medical Image Analysis Image Reconstruction Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.
COLOR HISTOGRAM AND DISCRETE COSINE TRANSFORM FOR COLOR IMAGE RETRIEVAL Presented by 2006/8.
Coregistration and Spatial Normalisation
Jan Kamenický Mariánská  We deal with medical images ◦ Different viewpoints - multiview ◦ Different times - multitemporal ◦ Different sensors.
MEDICAL IMAGE REGISTRATION BY MAXIMIZATION OF MUTUAL INFORMATION Dissertation Defense by Chi-hsiang Lo June 27, 2003 PRESENTATION.
Chapter 10 Image Segmentation.
Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.
EECS 274 Computer Vision Segmentation by Clustering II.
Medical Image Analysis Image Registration Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.
Feature based deformable registration of neuroimages using interest point and feature selection Leonid Teverovskiy Center for Automated Learning and Discovery.
Spatio-Temporal Free-Form Registration of Cardiac MR Image Sequences Antonios Perperidis s /02/2006.
A New Method of Probability Density Estimation for Mutual Information Based Image Registration Ajit Rajwade, Arunava Banerjee, Anand Rangarajan. Dept.
A B C D E F A ABSTRACT A novel, efficient, robust, feature-based algorithm is presented for intramodality and multimodality medical image registration.
Statistical Parametric Mapping Lecture 11 - Chapter 13 Head motion and correction Textbook: Functional MRI an introduction to methods, Peter Jezzard, Paul.
Basic Concepts of Information Theory Entropy for Two-dimensional Discrete Finite Probability Schemes. Conditional Entropy. Communication Network. Noise.
SPM Pre-Processing Oli Gearing + Jack Kelly Methods for Dummies
Uses of Information Theory in Medical Imaging Wang Zhan, Ph.D. Center for Imaging of Neurodegenerative Diseases Tel: x2454,
Instructor: Mircea Nicolescu Lecture 5 CS 485 / 685 Computer Vision.
Introduction to Medical Imaging Regis Introduction to Medical Imaging Registration Alexandre Kassel Course
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 1: INTRODUCTION.
Basic Concepts of Information Theory Entropy for Two-dimensional Discrete Finite Probability Schemes. Conditional Entropy. Communication Network. Noise.
Image Subtraction Mask mode radiography h(x,y) is the mask.
University of Ioannina
Fereshteh S. Bashiri Advisors: Zeyun Yu, Roshan M. D’souza
Spatial Preprocessing John Ashburner
Lecture 12 Theory of Registration ch
Dynamical Statistical Shape Priors for Level Set Based Tracking
Multi-modality image registration using mutual information based on gradient vector flow Yujun Guo May 1,2006.
Learning with information of features
REMOTE SENSING Multispectral Image Classification
Computational Neuroanatomy for Dummies
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor
MPHY8149: Image Registration
2D transformations (a.k.a. warping)
Image Stitching Computer Vision CS 678
Image Registration 박성진.
Anatomical Measures John Ashburner
SimpleITK Fundamental Concepts
MultiModality Registration using Hilbert-Schmidt Estimators
Image Registration  Mapping of Evolution
Image Stitching Linda Shapiro ECE/CSE 576.
Image Stitching Linda Shapiro ECE P 596.
Presentation transcript:

Mutual Information Based Registration of Medical Images Pluim et al: Survey Mattes et al: CT/PET Registration

Background Mutual information-based registration was proposed by Viola and Wells (MIT) in 1994-5. It has become commonplace in many clinical applications. It comes from information theory: the Shannon entropy H =  pi log (1/pi) = -pi log pi The more rare an event, the more meaning is associated with its occurrence

ENTROPY Entropy comes from information theory. The higher the entropy the more the information content. Entropy = pi is the probability of event i Compute it as the proportion of event i in the set. 16/30 are green circles; 14/30 are pink crosses log2(16/30) = -.9; log2(14/30) = -1.1 Entropy = -(16/30)(-.9) –(14/30)(-1.1) = .99

2-Class Case: minimum entropy What is the entropy of a group in which all examples belong to the same class? entropy = - 1 log21 = 0 What is the entropy of a group with 50% in either class? entropy = -0.5 log20.5 – 0.5 log20.5 =1 maximum entropy

Entropy for Images Shannon entropy can be computed for a gray-tone image. It then focuses on the distribution of the gray tones. An image consisting of almost a single intensity will have low entropy. An image with roughly equal quantities of different gray tones will have high entropy.

Histograms of Image Intensity

Mutual Information Woods introduced a registration measure for multimodality images in 1992. The measure was based on the assumption that regions of similar tissue (and similar gray tones) in one image would correspond to regions in the other image that also consist of similar gray values (but not the same as in the first image). Instead of defining regions of similar tissue in the image, they defined the regions in a feature space. When the images are correctly registered, the joint histogram of the two images will show certain clusters for gray tones of matching structures.

CT/MRI Example For each pair of corresponding points (x,y) CT gray tones MRI gray tones CT Image MR Image Joint Histogram For each pair of corresponding points (x,y) with x in the CT image and y in the MR image, there is a gray tone correspondence (gx,gy). The joint histogram counts how many times each gray tone correspondence occurs.

Joint Gray-tone Histograms of an MR Image with itself at Different Rotations joint entropy 0 degrees 2 degrees 5 degrees 10 degrees Because the images are identical, all gray-tone correspondences lie on the diagonal of the histogram matrix.

Measures of Mutual Information H = -pi,j log pi,j is the Shannon entropy for a joint distribution; pij is probability of co-occurence of i and j. Def. 1: I(A,B) = H(B) – H(B|A) Def. 2: I(A,B) = H(A) + H(B) – H(A,B) Def. 3: Kullback-Leibler distance joint gray values joint in case of independent images

Different Aspects of Mutual Information Procedures Preprocessing (ie. filtering) Measures (entropy measure, normalization measures) Spatial Information (not just gray tones, but where) Transformation (applied to register images) rigid affine deformable Implementation interpolation probability distribution estimation optimization

Modalities CT with PET, SPECT, other (video, fluoroscopy) MR with CT, PET, SPECT, US CT with PET, SPECT, other (video, fluoroscopy) Microscopy with other

Anatomical Entities thorax/lungs spine heart breast abdomen/liver brain thorax/lungs spine heart breast abdomen/liver pelvis tissue other

PET-CT Image Registration in the Chest Using Free-Form Deformations David Mattes, David Haynor, et al UW Medical Center Popular implementation of mutual information registration Available in ITK package We use it in our research.

Application PET-to-CT image registration in the chest Fuse images from a modality with high anatomic detail (CT) With images from a modality delineating biological function (PET) Producing a nonparametric deformation that registers them.

Overall Method The PET image has a corresponding transmission image (TR) The TR image is similar to a CT attenuation map with a higher energy radiation beam, resulting in less soft-tissue detail and limited resolution Once the TR and CT images are registered, the resulting transformation can be applied to the emission image for improved PET image interpretation. GOAL: find a deformation map that aligns the TR image with the CT image and evaluate the accuracy.

Axial Slice Coronal Slice CT Image TR Image

Methodology Notation fT(x) is a test image over domain VT fR(x) is a reference image over domain VR g(x | ) is a deformation from VT to VR  is the set of parameters of the transformation We want to find the set of parameters  that minimizes an image discrepancy function S They hypothesize that the set of transformation parameters that minimizes S brings the transformed test image into best registration with the reference image.  = arg min S(fR , fT  g( | ))

Image Representation Optimizing a function requires taking derivatives. Thus it is easier if the function can be represented in a form that is explicitly differentiable. This means that both the deformations and the similarity criterion must be differentiable. So images are represented using a B-Spline basis. Parzen windows are used instead of simple, bilinear interpolation.

SOME of the Math An image f(x), coming in as a set of sampled values, is represented by a cubic spline function that can be interpolated at any between-pixel position. The spline function is differentiable. The smoothed joint histogram of (fR , fT  g( | )) is defined as a cross product of the two spline functions. Computation of mutual information requires the smoothed joint histogram the marginal smoothed histogram for the test image the marginal probability for the reference image, which is independent of the transformation parameters

The image discrepancy measure is the negative of mutual information S between the reference image and the transformed test image expressed as a function of the transformation parameters . where p, pT, and pR are the joint, marginal test, and marginal reference probability distributions, respectively. The variables  and  are the histogram bin indexes for the reference and test images, respectively.

Deformations Deformations are also modeled as cubic B-splines. They are defined on a much coarser grid. A deformation is defined on a sparse, regular grid of control points placed over the test image. A deformation is varied by defining the motion g(j) at each control point j.

Transformation The transformation of the test image is specified by mapping reference image coordinates according to a locally perturbed rigid body transformation. The parameters of the transformation are:  = { , , , tx, ty, tz; j } {, , } are the roll-pitch-yaw Euler angles, [tx, ty, tz] is the translation vector, and j is the set of deformation coefficients (2200 of them)

Multiresolution Optimization Strategy The registration process is automated by varying the deformation in the test image until the discrepancy between the two images is minimized. The alignment process is divided into two registrations: one for the rigid body part and one for the deformation A limited-memory, quasi-Newton minimization package is used. To avoid local minima and decrease computation time, a hierarchical multiresolution optimization scheme is used.

Results 28 patients, 27 successful registrations 205 slices per image average of 100 minutes per registration 10 minutes for the rigid body part 90 minutes for the deformable part error index of .54, which is in the 0 to 6mm error range

axial coronal CT registered TR unregistered TR

Sample Images from 7 Anatomic Locations