ITK Deformable Registration Finite Elements Methods

Open Source Disclaimer Many of the slides in this talk were taken from the ITK course presented at MICCAI 2003 by Dr. James Gee (U. Penn) Brian Avants (U. Penn) Tessa Sundaram (U. Penn) Dr. Lydia Ng (AllenInstitue) Of course, any errors… are mine.

Deformable Registration Finite Element Methods for Deformable Registration

Deformable Transformation y y Transform x x Fixed Image Moving Image

Deformable Transformation y y Transform x x Fixed Image Moving Image

Deformable Transformation y x

Deformable Transformation y x

FEM Grid y FEM Grid Resampling Grid x

FEM Grid y FEM Grid x

FEM Grid y FEM Grid Computed Deformation x

FEM Grid y FEM Grid Displacements Forces x

FEM Iterative Linear System Finite Element Methods F Forces Vector N U Vector N Displacements K Regularization Matrix NxN

FEM Iterative Linear System Finite Element Methods F K U F U = ● K

FEM Iterative Linear System N = Number of Nodes N x N N N = ● K U F

FEM Iterative Linear System Iteratively Solving a Linear System K U F = ● Linearization of a Physical Model Image based forces Node Displacements

FEM Energy Formulation Find registration transformation that maximizes

FEM Energy Formulation Benefits Intuitive; easier to express constraints Powerful numerical solutions available Optimality of solutions; easier to debug Limitations Difficult / overhead to implement

Deformable Registration To solve the deformation, consider only displacements of the form

Deformable Registration Linear Elements

Deformable Registration α1 φ1 Element Shape Function

Deformable Registration Element α2 φ2 Shape Function

Deformable Registration α3 φ3 Element Shape Function

Deformable Registration u α3 φ3 α1 φ1 Element α2 φ2 Shape Functions

Deformable Registration Higher Order Elements

Deformable Registration α1 φ1 Element Shape Function

Deformable Registration α4 φ4 Element Shape Function

Deformable Registration Element α2 φ2 Shape Function

Deformable Registration φ5 α5 Element Shape Function

Deformable Registration α3 φ3 Element Shape Function

Deformable Registration φ6 α6 Element Shape Function

Deformable Registration α4 φ4 u α3 φ3 α1 φ1 α6 φ6 Element α5 φ5 α2 φ2 Shape Functions

Deformable Registration Substitute uh into E, then minimizing with respect to αi:

BSplines Grid & Image Grid Calculation are made in an Element by Element basis

BSplines Grid & Image Grid Elements are connected at Nodes at which the displacement is solved

BSplines Grid & Image Grid Efficiency is gained by elemental computation

BSplines Grid & Image Grid Domain subdivision (Mesh) can be tailored to the underlying geometry of the image.

FEM Solver Start Iteration Loop Begin Loop by making physical assumptions and then taking the derivative of the similarity metric. End loop when the solution stabilizes. Physical Assumptions New Solution Solve Image Metric Derivative End Iteration Loop

K U F FEM Solver Start Iteration Loop Physical Assumptions Solve New Solution Solve U Image Metric Derivative F End Iteration Loop

If ( Unew – Uold) < ε then Stop FEM Solver Start Iteration Loop K Unew U F K F U ● = If ( Unew – Uold) < ε then Stop

KU=F in Code itk::FEMRegistrationFilter::IterativeSolve() itk::FEMSolver::AssembleK() FEMSolver :: AddSolution() FEMSolver:: Solve() FEMSolver::AssembleF() calls FEMImageMetricLoad::Fe()

FEM-Based Registration Options Element Type Triangles Quadrilaterals Hexahedra Tetrahedra

FEM-Based Registration Options Continuum / Physical Model Linear elasticity Membrane Other specialized

FEM-Based Registration Options Mesh geometry Uniform grid vs. adaptive Anatomy-specific mesh

FEM-Based Registration Options Metric Mean square Normalized cross-correlation Mutual information Pattern intensity

Introduction to the ITK Finite Element Library ITK FEM Library Introduction to the ITK Finite Element Library

ITK FEM Library Library for solving general FEM problems Object oriented C++ classes are used to specify the geometry and behavior of the elements apply external forces and boundary conditions solve problem and post-process the results

ITK FEM Library Applications Mechanical modeling Image registration

FEM Basics Mesh Loads Boundary conditions Nodes Elements Points in space where solutions are obtained Elements e.g., 2-D triangular elements Loads e.g., gravity (body) load Boundary conditions e.g., nodes fixed in space

ITK FEM Elements Core of the library is the Element class Code is in two functionally independent parts Geometry and Physics Arbitrarily combined to create new elements Problem domain is specified by a mesh Geometry Physics

Loads Classes that apply external forces (loads) to elements Various types Easily extensible

Solvers Provide functionality to obtain and process the solution Different solution methods  different solver classes Static problems Time dependent - dynamic problems

Solvers Use linear system wrappers to link FEM classes to an external numeric library Any numeric library can be used to solve the systems of linear equations in FEM problems VNL and ITPACK currently supported

Setting Up a FEM Problem Four-step process Select element classes Discretize problem domain Specify boundary conditions Specify/Apply external loads Two options Directly  create proper objects in code Indirectly  read object definitions from a file

Deformable Registration FEM-Base Registration Parameters

Parameter File : Part 1 % --------------------------------------------------------- % Parameters for the single- or multi-resolution techniques 1 % Number of levels in the multi-resolution pyramid (1 = single-res) 1 % Highest level to use in the pyramid 1 1 % Scaling at lowest level for each image dimension 8 % Number of pixels per element 1.e5 % Elasticity (E) 1.e4 % Density (RhoC) 1. % Image energy scaling 4 % NumberOfIntegrationPoints 1 % WidthOfMetricRegion 25 % MaximumIterations % ------------------------------- % Parameters for the registration 0 1.0 % Similarity metric (0=mean sq, 1=ncc, 2=pattern int, 3=MI) 1.0 % Alpha 1 % DescentDirection 2 % DoLineSearch (0=never, 1=always, 2=if needed) 1.e1 % TimeStep 1.e-15 % Energy Reduction Factor

Parameter File : Part 2 % ---------------------------------- % Information about the image inputs 2 % ImageDimension 256 % Nx (image x dimension) 256 % Ny (image y dimension) 128 % Nz (image z dimension - not used if 2D) brain_slice1.mhd % ReferenceFileName brain_slice1warp.mhd % TargetFileName % ------------------------------------------------------------------- % The actions below depend on the values of the flags preceding them. % For example, to write out the displacement fields, you have to set % the value of WriteDisplacementField to 1. 0 % UseLandmarks? - % LandmarkFileName brain_result % ResultsFileName (prefix only) 1 % WriteDisplacementField? brain_disp % DisplacementsFileName (prefix only) 1 % ReadMeshFile? brain_mesh.fem % MeshFileName END

Configuring Parameters #1 this->DoMultiRes(true); this->m_NumLevels = nlev; this->m_MaxLevel = mlev; for (jj=0; jj < ImageDimension; jj++) { m_ImageScaling[jj] = dim; } for (jj=0; jj < this->m_NumLevels; jj++) { this->m_MeshPixelsPerElementAtEachResolution(jj) = p; this->SetElasticity(e, jj); this->SetRho(p, jj); this->SetGamma(g, jj); this->SetNumberOfIntegrationPoints(ip, jj); this->SetWidthOfMetricRegion(w, jj); this->SetMaximumIterations(mit, jj);

Configuring Parameters #2 this->SetDescentDirectionMinimize(); or this->SetDescentDirectionMaximize(); this->DoLineSearch( n ); // n = 0, 1, 2 this->SetTimeStep( t ); this->SetEnergyReductionFactor( fbuf );

Configuring Parameters #3 this->m_ImageSize[0] = xdim; this->m_ImageSize[1] = ydim; if (dim == 3) this->m_ImageSize[2] = zdim; this->SetReferenceFile( imgfile1 ); this->SetTargetFile( imgfile2 ); this->UseLandmarks( true ); this->SetLandmarkFile( lmfile ); this->SetResultsFile( resfile ); this->SetWriteDisplacements( true ); this->SetDisplacementsFile( dispfile ); this->m_ReadMeshFile = true; this->m_MeshFileName = meshfile;

Deformable Registration FEM-Based Registration: Writing the Code ../ Insight / Examples / Registration / DeformableRegistration1.cxx

Header Declarations #include "itkImageFileReader.h" #include "itkImageFileWriter.h“ #include "itkFEM.h" #include “itkFEMRegistrationFilter.h"

Type Definitions typedef itk::Image< unsigned char, 2 > fileImageType; typedef itk::Image< float, 2 > ImageType; typedef itk::fem::Element2DC0LinearQuadrilateralMembrane ElementType; typedef itk::fem::Element2DC0LinearTriangularMembrane ElementType2; typedef itk::fem::FEMRegistrationFilter< ImageType, ImageType > RegistrationType;

Registering Objects ElementType::LoadImplementationFunctionPointer fp1 = & itk::fem::ImageMetricLoadImplementation< ImageLoadType >::ImplementImageMetricLoad; DispatcherType::RegisterVisitor( (ImageLoadType*)0 , fp1 ); ElementType2::LoadImplementationFunctionPointer fp2 = & itk::fem::ImageMetricLoadImplementation< ImageLoadType >::ImplementImageMetricLoad; DispatcherType2::RegisterVisitor( (ImageLoadType*)0 , fp2 );

Input / Output RegistrationType::Pointer registration = RegistrationType::New(); registration->SetConfigFileName( paramname ); registration->ReadConfigFile();

Material and Element Setup // Create the material properties itk::fem::MaterialLinearElasticity::Pointer m; m = itk::fem::MaterialLinearElasticity::New(); m->GN = 0; m->E = registration->GetElasticity(); m->A = 1.0; // Cross-sectional area m->h = 1.0; // Thickness m->I = 1.0; // Moment of inertia m->nu = 0.; // Poisson's ratio m->RhoC = 1.0; // Density // Create the element type ElementType::Pointer e1 = ElementType::New(); e1->m_mat= dynamic_cast< itk::fem::MaterialLinearElasticity* >( m ); registration->SetElement( e1 ); registration->SetMaterial( m );

Running the Registration registration->RunRegistration(); registration->WriteWarpedImage(); if ( registration->GetWriteDisplacements() ) { registration->WriteDisplacementField( 0 ); // x registration->WriteDisplacementField( 1 ); // y registration->WriteDisplacementFieldMultiComponent(); }

FEM - Deformable Registration Example #1

Fixed Image

Moving Image

Registered Image

Registered Image

FEM - Deformable Registration Example #2

Fixed Image

Moving Image

Registered Image

Registered Image

FEM - Deformable Registration Example #3

Fixed Image

Moving Image

Registered Image

Registered Image

FEM - Deformable Registration Example #4 Elasticity value was doubled

Fixed Image

Moving Image

Registered Image

Registered Image

Enjoy ITK !