Elastic Registration in the Presence of Intensity Variations Source: IEEE Transactions on Medical Imaging, Vol. 22, No. 7, July 2003 Authors: Senthil Periaswamy, Hany Farid 實驗室：孫永年老師 視覺系統實驗室 Speaker: 醫資所 Q 陳儀津, 資工所 P 翟靖華 Date: January 07, 2009

2 Outline Introduction Methods Results Discussion

3 Introduction Methods 翟靖華

4 Introduction Image registration is the process of finding a transformation that aligns one image to another. This paper has presented a general purpose elastic registration algorithm that contends with both large overall transformations, and with more localized nonrigid transformations.

5 Introduction In estimating the transformation between two images we must choose  to estimate the transformation between a small number of extracted landmarks or features, or between the unprocessed intensity images  geometric transformation model  how to explicitly model intensity changes  An error metric that incorporates the previous three choices  minimize the error metric

6 Introduction Feature-based approaches can be sensitive to the accuracy of the feature extraction. Intensity-based approaches are more computationally demanding, but avoids the difficulties of a feature extraction stage. Our registration model incorporates both a geometric and intensity transformation.

7 Introduction The geometric model assumes a locally affine and a global smoothness constraint. The intensity model accounts for local changes in brightness and contrast and a global smoothness constraint. Employ a standard MSE metric on the intensity values.

8 Introduction An error function linear in the model parameters is minimized using least- squares, then augmented with a nonlinear smoothness constraint. This entire procedure is built on a differential multiscale framework.

9 Methods A. Local Affine Model Denote and as the source and target images.

10 Methods Quadratic error function

11 Methods Using a first-order truncated Taylor series expansion

12 Methods where Note that this quadratic error function is now linear in its unknowns This error function may be expressed more compactly in vector form as

13 Methods This error function can now be minimized analytically by differentiating with respect to the unknowns. Setting this result equal to zero, and solving for yields

14 Methods B. Intensity Variations Inherent to the model outlined in the previous section is the assumption that the image intensities between the source and target are unchanged. To account for intensity variations, we incorporate into our model an explicit change of local contrast and brightness. our initial model, (1), now takes the form

15 Methods where Minimization of this error function is accomplished as before by differentiating, setting the result equal to zero, and solving for

16 Methods C. Smoothness Until now, we have assumed that the local affine and contrast brightness parameters are constant within a small spatial neighborhood. A smoothness constraint on the contrast/brightness parameters has the added benefit of avoiding a degenerate solution where a pure brightness modulation is used to describe the mapping between images.

17 Methods

18 This error function is again minimized by differentiating with respect to the model parameters, setting the result equal to zero and solving Methods is computed by convolving with the 3×3 kernel (1 4 1 ; ; 1 4 1) /20

19 On each iteration j, is estimated from the current. The initial estimate is estimated from the closed-form solution of Method B. Methods

20 Method-Implementation Details Method-Extensions to Three Dimensions Result Demo Discuss 陳儀津

21 Methods D. Implementation Details First, in order to simplify the minimization, the error function of (13) was derived through a Taylor-series expansion. In particular, on each iteration, the estimated transformation is applied to the source, and a new transformation is estimated between the source and target image. As few as five iterations greatly improve the final estimate.

22 Source image Target image Warped image Source image Warped image

23 Methods Second, the required spatial/temporal derivatives have finite support thus fundamentally limiting the amount of motion that can be estimated. A coarse-to-fine scheme is adopted in order to contend with larger motions. A Gaussian pyramid is built for both source and target images, and the local affine and contrast/brightness parameters estimated at the coarsest level. These parameters are used to warp the source image in the next level of the pyramid.

24 Methods Finally, the calculation of the spatial/temporal derivatives is a crucial step. Spatial/temporal derivatives of discretely sampled images are often computed as differences between neighboring sample values. Such differences are typically poor approximation to derivatives and lead to substantial errors. In computing derivatives we employ a set of derivative filters specifically designed for multidimensional differentiation. These filters significantly improve the result registration.

25 Methods H. Farid and E. P. Simoncelli, “Optimally rotation-equivariant directional derivative kernels,” in Proc. Int. Conf. Computer Analysis of Images and Patterns, Sept. 1997, pp

26 Methods E. Extensions to Three Dimensions Denote and as the source and target volumes, respectively.

27 Methods As before, we define an error function, approximate it with a first-order truncated Taylor series expansion, differentiate with respect to the unknowns, set the result equal to zero and solve to obtain

28 Methods The contrast and brightness terms( and ) are derived as in the 2-D case, see(9) and (10). The resulting solution is once again in the same form as in(19), with now being a vector with fourteen elements, and and defined as shown in (22) and (23).

29 Methods Smoothness is incorporated as before by augmenting the error function of the previous section with a smooth term, where is the error function from above given by:, where and as in (22) and (23), and is

30 Methods An estimate of is obtained once again by differentiating, setting result equal to zero, and solving, giving the same iterative solution as in (17) with and given by (22) and (23). is now a diagonal matrix with diagonal elements,and zero off the diagonal.

31 Result Gaussian Pyramid Global Registration Local Registration Outer Loop: (5) Inner Loop: (40) every pixel Image

32 Result Fig. 1 are results from four synthetically transformed images. In each case, a different random geometric and contrast/brightness transformation was applied to the source image. In each case, the registered source image is in good agreement with target image.

33 Result Shown in Fig. 2 are results from four clinical cases. In each case, the source and target images are either from different subjects or from subjects at different times.

34 Result Shown in Fig. 3 are results from four more clinical cases. In each case, the source and target images are from different modalities. MRIT1/ T2 MRI-T1/MRI-Proton Density photograph/MRI-T2 CT/photograph

35 Result Fig. 4. Results from a synthetic 3- D transform. Shown in each row is the source, target, and registered source for several z slices.

36 Result Fig. 5. Results from a clinical (different subjects) 3-D case. Shown in each row is the source, target, and registered source for several z slices.

37 Level 0 Global Local Source Target

38 Level 1 Global Local

39 Level 2 Global

40 Local

41 Level 3 Global

42 Global

43 Local Source Target

44 Discussion We have presented a general purpose elastic registration algorithm. Our registration model incorporates both a geometric and intensity transformation. The geometric model assumes a locally affine and globally smooth transformation. The intensity model accounts for local differences in contrast and brightness while imposing a global smoothness on the overall intensity differences. In this paper we have not explicitly dealt with the problem of missing data, such as structures present in one image but absent in the other.

45 Thank you