1. Jan Kamenický Mariánská 2008

2. 2

3.  We deal with medical images ◦ Different viewpoints - multiview ◦ Different times - multitemporal ◦ Different sensors – multimodal  Area-based methods (no features)  Transformation model  Cost function minimization 3

4.  Transformation model ◦ Displacement field u(x) 4

5.  Transformation model ◦ Displacement field u(x)  Cost function ◦ Similarity measure (external forces) ◦ Smoothing (penalization) term (internal forces) ◦ Additional constraints (landmarks, volume preservation) 5

6.  Transformation model ◦ Displacement field u(x)  Cost function ◦ Similarity measure (external forces) ◦ Smoothing (penalization) term (internal forces) ◦ Additional constraints (landmarks, volume preservation)  Minimization 6

7.  Main problems ◦ Computationally intensive ◦ Sensitive to initial positioning 7 reference image sensed imagepyramid sampler interpolator optimizer transform pyramid metric multi-resolution resolution level

8.  Translation  Rigid (Euler) ◦ Translation, rotation  Similarity ◦ Translation, rotation, scaling  Affine  B-splines ◦ Control points - regular grid on reference image 8

9. 9

10.  Sum of Squared Differences  Normalized Correlation Coefficients  Mutual Information  Normalized Gradient Field 10

11.  Sum of Squared Differences (SSD) ◦ Equal intensity distribution (same modality)  Normalized Correlation Coefficients  Mutual Information  Normalized Gradient Field 11

12.  Sum of Squared Differences  Normalized Correlation Coefficients (NCC) ◦ Linear relation between intensity values (but still same modality)  Mutual Information  Normalized Gradient Field 12

13.  Sum of Squared Differences  Normalized Correlation Coefficients  Mutual Information ◦ Any statistical dependence  Normalized Gradient Field 13

14.  Mutual Information (MI) ◦ From entropy 14

15.  Mutual Information (MI) ◦ From Kullback-Leibler distance 15

16.  Mutual Information (MI) ◦ For images  p(x) … normalized image histogram ◦ Normalized Mutual Information (NMI) 16

17.  Mutual Information (MI) ◦ Joint probability estimation  Using B-spline Parzen windows  and are defined by the histogram bins widths 17

18.  Sum of Squared Differences  Normalized Correlation Coefficients  Mutual Information  Normalized Gradient Field (NGF) ◦ Based on edges 18

19.  Elastic ◦ Elastic potential (motivated by material properties)  Fluid ◦ Viscous fluid model (based on Navier-Stokes)  Diffusion ◦ Much faster 19

20.  Curvature ◦ Doesn’t penalize affine transformation  Bending energy (Thin plate splines) 20

21. 21 curvaturediffusion elasticfluid

22.  Landmarks (fiducial markers) ◦ “Hard” constraint ◦ “Soft” constraint  Volume preservation 22

23.  Full  Grid ◦ Used with multi-resolution  Random ◦ Random subset of voxels is selected ◦ Improved speed 23

24.  Nearest Neighbour (NN)  Linear ◦ Usually sufficient during optimization  N -th order B-spline ◦ Useful for the final image (usually 3 rd order) 24

25. 25

26.  Gradient Descent (GD) ◦ Linear rate of convergence  Quasi-Newton  Nonlinear Conjugate Gradient  Stochastic Gradient Descent  Evolution Strategy 26

27.  Gradient Descent  Quasi-Newton (QN) ◦ Can be superlinearly convergent  Nonlinear Conjugate Gradient  Stochastic Gradient Descent  Evolution Strategy 27

28.  Gradient Descent  Quasi-Newton  Nonlinear Conjugate Gradient (NCG) ◦ Superlinear rate of convergence can be achieved  Stochastic Gradient Descent  Evolution Strategy 28

29.  Gradient Descent  Quasi-Newton  Nonlinear Conjugate Gradient  Stochastic Gradient Descent (SGD) ◦ Similar to GD, but uses approximation of the gradient (Kiefer-Wolfowitz, Simultaneous Perturbation, Robbins-Monro)  Evolution Strategy 29

30.  Gradient Descent  Quasi-Newton  Nonlinear Conjugate Gradient  Stochastic Gradient Descent  Evolution Strategy (ES) ◦ Covariance matrix adaptation ◦ Tries several possible directions (randomly according to the covariance matrix of the cost function), the best are chosen and their weighted average is used 30

31.  Data complexity ◦ Gaussian pyramid ◦ Laplacian pyramid ◦ Wavelet pyramid  Transformation complexity ◦ Transformation superposition ◦ Different B-spline grid density 31

32.  Registration toolkit based on ITK  Handles many methods ◦ Similarity measures (SSD, NCC, MI, NMI) ◦ Transformations (rigid, affine, B-splines) ◦ Optimizers (GD, SGD-RM) ◦ Samplers, Interpolators, Multi-resolution, … http://elastix.isi.uu.nl 32