1. CSci 6971: Image Registration Lecture 27: FEM-Based Methods April 23, 2004 Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware

2. Image RegistrationLecture 27 2 Open Source Disclaimer Many of the slides in this talk were taken from the ITK course presented at MICCAI 2003 by Dr. James Gee (U. Penn) Brian Avants (U. Penn) Tessa Sundaram (U. Penn) Dr. Lydia Ng (Insightful Corp.) Of course, any errors… are mine.

3. Image RegistrationLecture 27 3 Deformable Registration Finite Element Methods for Deformable Registration

4. Image RegistrationLecture 27 4 Deformable Transformation y Fixed Image Transform x y Moving Image x

5. Image RegistrationLecture 27 5 Deformable Transformation y Fixed Image Transform x y Moving Image x

6. Image RegistrationLecture 27 6 Deformable Transformation y x

7. Image RegistrationLecture 27 7 Deformable Transformation y x

8. Image RegistrationLecture 27 8 FEM Grid y x FEM Grid Resampling Grid

9. Image RegistrationLecture 27 9 FEM Grid y x FEM Grid

10. Image RegistrationLecture 27 10 FEM Grid y x FEM Grid Computed Deformation

11. Image RegistrationLecture 27 11 FEM Grid y x FEM Grid Forces Displacements

12. Image RegistrationLecture 27 12 FEM Iterative Linear System Finite Element Methods Forces Displacements Regularization F U K Vector N Matrix NxN

13. Image RegistrationLecture 27 13 FEM Iterative Linear System Finite Element Methods F U K FUK ● =

14. Image RegistrationLecture 27 14 FEM Iterative Linear System N = Number of Nodes FU K N x NNN ● =

15. Image RegistrationLecture 27 15 FEM Iterative Linear System Iteratively Solving a Linear System FUK ● = Image based forces Linearization of a Physical Model Node Displacements

16. Image RegistrationLecture 27 16 FEM Energy Formulation Find registration transformation that maximizes

17. Image RegistrationLecture 27 17 FEM Energy Formulation Benefits  Intuitive; easier to express constraints  Powerful numerical solutions available  Optimality of solutions; easier to debug Limitations  Difficult / overhead to implement

18. Image RegistrationLecture 27 18 Deformable Registration To solve the deformation, consider only displacements of the form

19. Image RegistrationLecture 27 19 Deformable Registration Linear Elements

20. Image RegistrationLecture 27 20 Deformable Registration φ1φ1 Element α1α1 Shape Function

21. Image RegistrationLecture 27 21 Deformable Registration φ2φ2 Element α2α2 Shape Function

22. Image RegistrationLecture 27 22 Deformable Registration φ3φ3 Element α3α3 Shape Function

23. Image RegistrationLecture 27 23 Deformable Registration φ3φ3 Element φ1φ1 φ2φ2 u α2α2 α3α3 Shape Functions α1α1

24. Image RegistrationLecture 27 24 Deformable Registration Higher Order Elements

25. Image RegistrationLecture 27 25 Deformable Registration φ1φ1 Element α1α1 Shape Function

26. Image RegistrationLecture 27 26 Deformable Registration φ4φ4 Element α4α4 Shape Function

27. Image RegistrationLecture 27 27 Deformable Registration φ2φ2 Element α2α2 Shape Function

28. Image RegistrationLecture 27 28 Deformable Registration φ5φ5 Element α5α5 Shape Function

29. Image RegistrationLecture 27 29 Deformable Registration φ3φ3 Element α3α3 Shape Function

30. Image RegistrationLecture 27 30 Deformable Registration φ6φ6 Element α6α6 Shape Function

31. Image RegistrationLecture 27 31 Deformable Registration φ3φ3 Element φ1φ1 φ2φ2 u α2α2 α3α3 Shape Functions α1α1 α6α6 α5α5 α4α4 φ5φ5 φ6φ6 φ4φ4

32. Image RegistrationLecture 27 32 Deformable Registration Substitute u h into E, then minimizing with respect to α i :

33. Image RegistrationLecture 27 33 BSplines Grid & Image Grid Calculation are made in an Element by Element basis

34. Image RegistrationLecture 27 34 BSplines Grid & Image Grid Elements are connected at Nodes at which the displacement is solved

35. Image RegistrationLecture 27 35 BSplines Grid & Image Grid Efficiency is gained by elemental computation

36. Image RegistrationLecture 27 36 BSplines Grid & Image Grid Domain subdivision (Mesh) can be tailored to the underlying geometry of the image.

37. Image RegistrationLecture 27 37 FEM Solver Image Metric Derivative Physical Assumptions Solve New Solution Start Iteration Loop End Iteration Loop  Begin Loop by making physical assumptions and then taking the derivative of the similarity metric.  End loop when the solution stabilizes.

38. Image RegistrationLecture 27 38 FEM Solver Image Metric Derivative Physical Assumptions Solve New Solution Start Iteration Loop End Iteration Loop F U K

39. Image RegistrationLecture 27 39 FEM Solver F K U U new Start Iteration Loop If ( U new – U old ) < ε then Stop KFU ● =

40. Image RegistrationLecture 27 40 KU=F in Code FEMSolver::AssembleF() calls FEMImageMetricLoad::Fe() itk::FEMSolver::AssembleK() FEMSolver:: Solve() FEMSolver :: AddSolution() itk::FEMRegistrationFilter::IterativeSolve()

41. Image RegistrationLecture 27 41 FEM-Based Registration Options Element Type  Triangles  Quadrilaterals  Hexahedra  Tetrahedra

42. Image RegistrationLecture 27 42 FEM-Based Registration Options Continuum / Physical Model  Linear elasticity  Membrane  Other specialized

43. Image RegistrationLecture 27 43 FEM-Based Registration Options Mesh geometry  Uniform grid vs. adaptive  Anatomy-specific mesh

44. Image RegistrationLecture 27 44 FEM-Based Registration Options Metric  Mean square  Normalized cross-correlation  Mutual information  Pattern intensity

45. Image RegistrationLecture 27 45 ITK FEM Library Introduction to the ITK Finite Element Library

46. Image RegistrationLecture 27 46 ITK FEM Library Library for solving general FEM problems  Object oriented  C++ classes are used to  specify the geometry and behavior of the elements  apply external forces and boundary conditions  solve problem and post-process the results

47. Image RegistrationLecture 27 47 ITK FEM Library Applications  Mechanical modeling  Image registration

48. Image RegistrationLecture 27 48 FEM Basics  Mesh  Nodes  Points in space where solutions are obtained  Elements  e.g., 2-D triangular elements  Loads  e.g., gravity (body) load  Boundary conditions  e.g., nodes fixed in space

49. Image RegistrationLecture 27 49 ITK FEM Elements  Core of the library is the Element class  Code is in two functionally independent parts  Geometry and Physics  Arbitrarily combined to create new elements  Problem domain is specified by a mesh GeometryPhysics

50. Image RegistrationLecture 27 50 Loads Classes that apply external forces (loads) to elements  Various types  Easily extensible

51. Image RegistrationLecture 27 51 Solvers  Provide functionality to obtain and process the solution  Different solution methods  different solver classes  Static problems  Time dependent - dynamic problems

52. Image RegistrationLecture 27 52 Solvers Use linear system wrappers to link FEM classes to an external numeric library  Any numeric library can be used to solve the systems of linear equations in FEM problems  VNL and ITPACK currently supported

53. Image RegistrationLecture 27 53 Setting Up a FEM Problem  Four-step process  Select element classes  Discretize problem domain  Specify boundary conditions  Specify/Apply external loads  Two options  Directly  create proper objects in code  Indirectly  read object definitions from a file

54. Image RegistrationLecture 27 54 Deformable Registration FEM-Base Registration Parameters

55. Image RegistrationLecture 27 55 Parameter File : Part 1 % --------------------------------------------------------- % Parameters for the single- or multi-resolution techniques % --------------------------------------------------------- 1% Number of levels in the multi-resolution pyramid (1 = single-res) 1% Highest level to use in the pyramid 1 1 % Scaling at lowest level for each image dimension 8 % Number of pixels per element 1.e5 % Elasticity (E) 1.e4 % Density (RhoC) 1. % Image energy scaling 4 % NumberOfIntegrationPoints 1 % WidthOfMetricRegion 25 % MaximumIterations % ------------------------------- % Parameters for the registration % ------------------------------- 0 1.0% Similarity metric (0=mean sq, 1=ncc, 2=pattern int, 3=MI) 1.0 % Alpha 1 % DescentDirection 2 % DoLineSearch (0=never, 1=always, 2=if needed) 1.e1 % TimeStep 1.e-15 % Energy Reduction Factor

56. Image RegistrationLecture 27 56 Parameter File : Part 2 % ---------------------------------- % Information about the image inputs % ---------------------------------- 2 % ImageDimension 256 % Nx (image x dimension) 256 % Ny (image y dimension) 128 % Nz (image z dimension - not used if 2D) brain_slice1.mhd % ReferenceFileName brain_slice1warp.mhd % TargetFileName % ------------------------------------------------------------------- % The actions below depend on the values of the flags preceding them. % For example, to write out the displacement fields, you have to set % the value of WriteDisplacementField to 1. % ------------------------------------------------------------------- 0 % UseLandmarks? - % LandmarkFileName brain_result % ResultsFileName (prefix only) 1 % WriteDisplacementField? brain_disp % DisplacementsFileName (prefix only) 1 % ReadMeshFile? brain_mesh.fem % MeshFileName END

57. Image RegistrationLecture 27 57 Configuring Parameters #1 this->DoMultiRes(true); this->m_NumLevels = nlev; this->m_MaxLevel = mlev; for (jj=0; jj < ImageDimension; jj++) { m_ImageScaling[jj] = dim; } for (jj=0; jj m_NumLevels; jj++) { this->m_MeshPixelsPerElementAtEachResolution(jj) = p; this->SetElasticity(e, jj); this->SetRho(p, jj); this->SetGamma(g, jj); this->SetNumberOfIntegrationPoints(ip, jj); this->SetWidthOfMetricRegion(w, jj); this->SetMaximumIterations(mit, jj); }

58. Image RegistrationLecture 27 58 Configuring Parameters #2 this->SetDescentDirectionMinimize(); or this->SetDescentDirectionMaximize(); this->DoLineSearch( n ); // n = 0, 1, 2 this->SetTimeStep( t ); this->SetEnergyReductionFactor( fbuf );

59. Image RegistrationLecture 27 59 Configuring Parameters #3 this->m_ImageSize[0] = xdim; this->m_ImageSize[1] = ydim; if (dim == 3) this->m_ImageSize[2] = zdim; this->SetReferenceFile( imgfile1 ); this->SetTargetFile( imgfile2 ); this->UseLandmarks( true ); this->SetLandmarkFile( lmfile ); this->SetResultsFile( resfile ); this->SetWriteDisplacements( true ); this->SetDisplacementsFile( dispfile ); this->m_ReadMeshFile = true; this->m_MeshFileName = meshfile;

60. Image RegistrationLecture 27 60 Deformable Registration FEM-Based Registration: Writing the Code../ Insight / Examples / Registration / DeformableRegistration1.cxx

61. Image RegistrationLecture 27 61 Header Declarations #include "itkImageFileReader.h" #include "itkImageFileWriter.h“ #include "itkFEM.h" #include “itkFEMRegistrationFilter.h"

62. Image RegistrationLecture 27 62 Type Definitions typedef itk::Image fileImageType; typedef itk::Image ImageType; typedef itk::fem::Element2DC0LinearQuadrilateralMembrane ElementType; typedef itk::fem::Element2DC0LinearTriangularMembrane ElementType2; typedef itk::fem::FEMRegistrationFilter RegistrationType;

63. Image RegistrationLecture 27 63 Registering Objects ElementType::LoadImplementationFunctionPointer fp1 = & itk::fem::ImageMetricLoadImplementation ::ImplementImageMetricLoad; DispatcherType::RegisterVisitor( (ImageLoadType*)0, fp1 ); ElementType2::LoadImplementationFunctionPointer fp2 = & itk::fem::ImageMetricLoadImplementation ::ImplementImageMetricLoad; DispatcherType2::RegisterVisitor( (ImageLoadType*)0, fp2 );

64. Image RegistrationLecture 27 64 Input / Output RegistrationType::Pointer registration = RegistrationType::New(); registration->SetConfigFileName( paramname ); registration->ReadConfigFile();

65. Image RegistrationLecture 27 65 Material and Element Setup // Create the material properties itk::fem::MaterialLinearElasticity::Pointer m; m = itk::fem::MaterialLinearElasticity::New(); m->GN = 0; m->E = registration->GetElasticity(); m->A = 1.0; // Cross-sectional area m->h = 1.0; // Thickness m->I = 1.0; // Moment of inertia m->nu = 0.; // Poisson's ratio m->RhoC = 1.0; // Density // Create the element type ElementType::Pointer e1 = ElementType::New(); e1->m_mat= dynamic_cast ( m ); registration->SetElement( e1 ); registration->SetMaterial( m );

66. Image RegistrationLecture 27 66 Running the Registration registration->RunRegistration(); registration->WriteWarpedImage(); if ( registration->GetWriteDisplacements() ) { registration->WriteDisplacementField( 0 ); // x registration->WriteDisplacementField( 1 ); // y registration->WriteDisplacementFieldMultiComponent(); }

67. Image RegistrationLecture 27 67 FEM - Deformable Registration Example #1

68. Image RegistrationLecture 27 68 Fixed Image

69. Image RegistrationLecture 27 69 Moving Image

70. Image RegistrationLecture 27 70 Registered Image

71. Image RegistrationLecture 27 71 Registered Image

72. Image RegistrationLecture 27 72 FEM - Deformable Registration Example #2

73. Image RegistrationLecture 27 73 Fixed Image

74. Image RegistrationLecture 27 74 Moving Image

75. Image RegistrationLecture 27 75 Registered Image

76. Image RegistrationLecture 27 76 Registered Image

77. Image RegistrationLecture 27 77 FEM - Deformable Registration Example #3

78. Image RegistrationLecture 27 78 Fixed Image

79. Image RegistrationLecture 27 79 Moving Image

80. Image RegistrationLecture 27 80 Registered Image

81. Image RegistrationLecture 27 81 Registered Image

82. Image RegistrationLecture 27 82 FEM - Deformable Registration Example #4 Elasticity value was doubled

83. Image RegistrationLecture 27 83 Fixed Image

84. Image RegistrationLecture 27 84 Moving Image

85. Image RegistrationLecture 27 85 Registered Image

86. Image RegistrationLecture 27 86 Registered Image

87. Image RegistrationLecture 27 87 End Enjoy ITK !