1. Consistent Linear-Elastic Transformations for Image Matching Gary E. Christensen Department of Electrical & Computer Engineering The University of Iowa This work was supported by NIH grant NS35368 and a grant from the Whitaker Foundation.

2. Introduction Uses of image registration image segmentation/deformable atlas characterization of normal vs. abnormal shape/variation multi-modality fusion functional brain mapping/removing shape variation surgical planning and evaluation image guided surgery template constrained reconstruction Image registration methods landmark, contour, surface, volume

3. Introduction Landmarks specify correspondence. Transformation interpolated between landmarks. Ideally, forward and reverse transforms are inverses of each other.

4. Introduction Limitations –Landmark manual identification, low-dimensional –Contour manual/semi-automatic, correspondence ambiguity –Surface semi-automatic/automatic, correspondence ambiguity –Volume automatic, correspondence ambiguity

5. Introduction Woods et al., Automated Image Registration: II. Intersubject Validation of Linear and Nonlinear Models, Journal of Computer Assisted Tomography, 22(1), 1998 Pairwise consistency Compute all pairwise registrations of a population using the affine transformation model. Average the transformation from A to B with all the transformations from A to X to B. Replace the original transformation from A to B with average transformation. Repeat for all until convergence.

6. Introduction Woods et al., Automated Image Registration: II. Intersubject Validation of Linear and Nonlinear Models, Journal of Computer Assisted Tomography, 22(1), 1998 Limitations Does not apply for a population of two data sets. There is no guarantee that the generated set of consistent transformations are valid. –ex. A poorly registered pair of images can adversely effect all of the pairwise transformations.

7. Introduction Consistent Transformation Estimation –Jointly estimate the forward and reverse transformation between two image volumes –Constrain the forward and reverse transformations to be inverses –Constrain the transformations to preserve topology

8. Problem Statement Jointly estimate the transformations h and g such that h maps T to S and g maps S to T subject to the constraint that h = g -1

9. Transformation Properties From a biological standpoint, it is desirable that image registration algorithms produce transformations with the properties: 1.The transformation from image A to B is unique, i.e., the forward h ab and reverse h ba transformations are inverses of one another. 2.The transformations have the transitive property, i.e., h ab (h bc (x)) = h ac (x). Most image registration algorithms do not produce transformations with these properties.

10. Sources of Error: Inverse Consistency Error (E ICC ) y=g(x) x’ =h(y) y x x’ Inverse Consistency Error = ||x-x’|| where x’=h(g(x))

11. 2D Landmark Experiment Compare thin-plate spline algorithms –Unidirectional vs. consistent registration* *Consistent Landmark Registration: 2000 iterations, X harmonics = 50, Y harmonics = 50 ForwardReverse

12. Inverse Consistency Error (Cyclic Boundary Conditions) 5.0 0.00 A—B—AB—A—BA—B—A TPS 0.00 5.0 Consistent TPS

13. Inverse Consistency Error (Cyclic Boundary Conditions) 5.0 0.00 A—B—AB—A—BA—B—A TPS 0.00 0.01 Consistent TPS

14. Inverse Consistency Error (Cyclic Boundary Conditions) 5.0 0.00 0.01 A—B—AB—A—B TPS Consistent TPS LabelPixel Err. A5.0 B0.008 B’0.27 C3.9 D0.008 D’0.33 LabelPixel Err. A0.003 B B’0.014 C0.005 D0.001 D’0.018 B A D C B A D C

15. Notation Image volumes: –T(x) = TemplateS(x) = Target Coordinate system: Transformations:

16. Symmetric Similarity Function Jointly estimate transformations from T to S and from S to T Minimize cost w.r.t. h and g Works with any similarity function –mutual information

17. Inverse Transformation Consistency Symmetric similarity functions do not guarantee g and h are inverses of each other. Impose constraint that g and h are inverses.

18. Diffeomorphic Transformations h:   Onto Globally One-to-One Continuous –Compact sets are mapped to compact sets –Connected sets are mapped to connected sets –A composition of continuous transformations is continuous Differentiable

19. Diffeomorphic Constraint The inverse consistency constraint only guarantees h and g are diffeomorphic transformations when To constrain h and g to be diffeomorphic, we use continuum mechanical models –linear elasticity –viscous fluid

20. Diffeomorphic Constraint Linear Elasticity

21. 1D Example Complex exponentials are eigenfunctions of constant coefficient difference equations

22. Transformation Parameterization Displacement fields (cyclic boundary conditions) coefficients –(3x1) complex-valued vectors –complex conjugate symmetry

23. Diffeomorphic Constraint Combining Gives

24. Diffeomorphic Constraint Linear Elasticity constraint

25. Minimization Problem Find h and g that satisfy: ^ ^ and  are Lagrange multipliers

26. Consistent Landmark Consistent Landmark Cost Minimization

27. Minimization Algorithm Gradient descent is used to solve for new basis coefficients at each iteration. Coarse to fine registration –Start algorithm with 0 and 1st harmonics. –Increase the number of harmonics by one after every N iterations. The reverse basis coefficients are fixed while estimating the forward basis coefficients and visa versa.

28. Gradient Descent Solution exists and is unique if h is a monotonic function of x h is diffeomorphic => h is monotonic in x Inverse Transformation Computation

29. 3D CT Inverse Consistency Experiment Use 3D CT data of infant heads Transform data volume A to B, and vice versa –Traditional linear-elasticity model –Consistent linear-elasticity model Combine the forward & reverse transformations Compare the composite transformation to Identity

30. 3D CT Inverse Consistency Experiment X-Dev.Y-Dev.Mag. Dev. Error of composite mapping h ab (h ba (x)) using the linear elastic model without inverse consistency constraint. Z-Dev. Axial Sagittal Coronal -0.94 1.2

31. 3D CT Inverse Consistency Experiment Error of composite mapping h ab (h ba (x)) with inverse consistency constraint using the linear elastic model. X-Dev.Y-Dev.Mag. Dev.Z-Dev. Axial Sagittal Coronal -0.1 0.1

32. 3D CT Inverse Consistency Experiment Error of composite mapping h ab (h ba (x)) using the linear elastic model with & without inverse consistency constraint. X-Dev.Y-Dev.Mag. Dev.Z-Dev. -0.1 0.1 -0.94 1.2 Without inverse consistency With inverse consistency

33. 3D CT Inverse Consistency Experiment w/ Inverse Consistency Constraint w/o Inverse Consistency Constraint Ratio Max. error whole volume0.111.2311 Ave. error whole volume0.00200.1680 Max. error in head above FH0.0781.2316 Ave. error in head above FH0.00730.4865 Error of composite mapping h ab (h ba (x)) using the linear elastic model with and without inverse consistency constraint. FH-Frankfurt Horizontal Plane

34. Experiments Eight experiments: MRI1: no constraints MRI2: linear elasticity MRI3: inverse consistency MRI4: lin. elast. and inv. consist. CT1: no constraints CT2: linear elasticity CT3: inverse consistency CT4: lin. elast. and inv. consist.

35. MRI4 Experiment T S T(h) S(g) u1 u2 u3 w1 w2 w3 Christensen, IPMI’99

36. MRI4 Experiment Christensen, IPMI’99

37. MRI4 Experiment Christensen, IPMI’99

38. MRI4 Experiment Christensen, IPMI’99

39. MRI4 Experiment Christensen, IPMI’99

40. Transformation Measurements ~~ Christensen, IPMI’99

41. CT4 Experiment T S T(h) S(g) u1 u2 u3 w1 w2 w3 Christensen, IPMI’99

42. CT4 Experiment Christensen, IPMI’99

43. CT4 Experiment Christensen, IPMI’99

44. CT4 Experiment Christensen, IPMI’99

45. CT4 Experiment Christensen, IPMI’99

46. Transformation Measurements ~~ Christensen, IPMI’99

47. Transformation Measurements ~~ Christensen, IPMI’99

48. Computational Costs Computational efficiency was achieved by using FFTs. Transforming one 64 3 voxel volume into another using 300 iterations takes approximately 25 minutes on a 180 MHz, R10000 processor. Computational time can be reduced by –reducing the number of iterations –using a more efficient optimization algorithm such as conjugate gradient, etc.

49. Anatomical Variation Goal is to quantify the average shape & variability of anatomical populations.

50. Literature Joshi et al., Gaussian Random Fields on Sub-manifolds for Characterizing Brain Surfaces, XVth International Conference on Information Processing in Medical Imaging, eds. Duncan and Gindi, Poultney, VT, June,1997 Miller et al., Statistical Methods in Computational Anatomy, Statistical Methods in Medical Research, vol. 6, 1997 Woods et al., Automated Image Registration: II. Intersubject Validation of Linear and Nonlinear Models, Journal of Computer Assisted Tomography, 22(1), 1998

51. Synthesizing the Average Within a given population –Determine the “average” shape –Determine the variability i i

52. Methods 1.Select one image volume from the population as the template 2.Estimate transformations by registering the template to all of the population images 3.Compute average and variance transformation from the estimated transformations 4.Compute synthesized average by applying the average transformation to the template image

53. Synthesizing the Average Shape Template Population

54. Synthesizing the Average Shape Template Population Average

55. Average and Variance Calculations Average: Variance:

56. Results Christensen et al., SPIE Medical Imaging 1999

57. Average and Original Data Avg. 1 2 3 4 5 6 128 146 163 Christensen et al., Synthesizing average 3D anatomical shapes using deformable templates, SPIE Medical Imaging 1999: Image Processing, ed. K.M. Hanson, SPIE vol. 3661.

58. Results Avg. 1 2 3

59. Skull Population 12345

60. Synthesized Average Skulls 12345

61. Skull Shape Variability Population Synthesized Averages The variability in skull shape for the 5 population skulls is greater than the skull shape variability for the 5 synthesized average skulls. Transverse Slice 67 Transverse Slice 108 Sagittal Slice 96

62. Skull Shape Differences Data Set Average displacement of skull voxels (mm) Variance of skull voxel displacements (mm 2 ) 4.609.92 3.237.30 3.148.33 3.107.50 2.995.56 2.742.63 2.572.72 2.222.79 2.562.64 2.592.70 Average chamfer distance of skull voxels (mm) Variance of skull voxel chamfer distances (mm 2 ) 2.517.08 1.424.80 1.394.96 1.264.38 1.364.07 1.112.44 0.9522.24 1.082.51 1.022.42 1.042.58

63. 20 Normal Adult Brains (Tns140hnnl) PopulationSynthesized Averages

64. 20 Normal Adult Brain Tracing (Tns140Avg) PopulationSynthesized Averages

65. 20 Normal Adult Brain Tracings (Tns140hnnl) Brain1 Brain2 Brain1 Brain2 Brain3 Brain1 to Brain9 Brain1 to Brain20 Synthesized Averages Population

66. 20 Brain Contours Population Synthesized Averages TransverseCoronalSagittalSagittal Projection

67. Avg. Displacement Distance Projections

68. Var. Displacement Distance Projections

69. Brain Volume and Chamfer Distance Measures

70. Bayesian Hypothesis Testing Empirically estimate a shape probability density –normal population p 0 (u) –abnormal population p 1 (u) Use Bayesian hypothesis testing to determine if a test transformation is closer to hypothesis 0 or 1 p 1 (u) p 0 (u) > < H0H0 H1H1 

71. Summary and Conclusions A new technique was presented for jointly estimating a consistent set of forward and reverse transformations. A new transformation model based on the Fourier series was presented and was used to simplify the discretized linear-elasticity constraint. The algorithm was efficiently implemented using FFTs.

72. Summary and Conclusions Unconstrained estimation leads to singular or near singular transformations. The linear-elastic constraint alone does not guarantee inverse consistency. The inverse consistency constraint alone does not guarantee nonsingular transformations during the iterative estimation procedure. The best results were generated using both the inverse consistency and linear-elastic constraints.

73. Summary and Conclusions A technique was presented for computing the average shape and variation of a population of data sets. Statistical shape models estimated in this fashion may be used to discriminate between normal and abnormal populations.

74. Acknowledgements This work was supported by NIH grant NS35368 and a grant from the Whitaker Foundation. We would also like to thank Richard Robb of the Mayo Clinic for his support in providing Analyze TM