Jan Kamenický Mariánská 2008

2

 We deal with medical images ◦ Different viewpoints - multiview ◦ Different times - multitemporal ◦ Different sensors – multimodal  Area-based methods (no features)  Transformation model  Cost function minimization 3

 Transformation model ◦ Displacement field u(x) 4

 Transformation model ◦ Displacement field u(x)  Cost function ◦ Similarity measure (external forces) ◦ Smoothing (penalization) term (internal forces) ◦ Additional constraints (landmarks, volume preservation) 5

 Transformation model ◦ Displacement field u(x)  Cost function ◦ Similarity measure (external forces) ◦ Smoothing (penalization) term (internal forces) ◦ Additional constraints (landmarks, volume preservation)  Minimization 6

 Main problems ◦ Computationally intensive ◦ Sensitive to initial positioning 7 reference image sensed imagepyramid sampler interpolator optimizer transform pyramid metric multi-resolution resolution level

 Translation  Rigid (Euler) ◦ Translation, rotation  Similarity ◦ Translation, rotation, scaling  Affine  B-splines ◦ Control points - regular grid on reference image 8

9

 Sum of Squared Differences  Normalized Correlation Coefficients  Mutual Information  Normalized Gradient Field 10

 Sum of Squared Differences (SSD) ◦ Equal intensity distribution (same modality)  Normalized Correlation Coefficients  Mutual Information  Normalized Gradient Field 11

 Sum of Squared Differences  Normalized Correlation Coefficients (NCC) ◦ Linear relation between intensity values (but still same modality)  Mutual Information  Normalized Gradient Field 12

 Sum of Squared Differences  Normalized Correlation Coefficients  Mutual Information ◦ Any statistical dependence  Normalized Gradient Field 13

 Mutual Information (MI) ◦ From entropy 14

 Mutual Information (MI) ◦ From Kullback-Leibler distance 15

 Mutual Information (MI) ◦ For images  p(x) … normalized image histogram ◦ Normalized Mutual Information (NMI) 16

 Mutual Information (MI) ◦ Joint probability estimation  Using B-spline Parzen windows  and are defined by the histogram bins widths 17

 Sum of Squared Differences  Normalized Correlation Coefficients  Mutual Information  Normalized Gradient Field (NGF) ◦ Based on edges 18

 Elastic ◦ Elastic potential (motivated by material properties)  Fluid ◦ Viscous fluid model (based on Navier-Stokes)  Diffusion ◦ Much faster 19

 Curvature ◦ Doesn’t penalize affine transformation  Bending energy (Thin plate splines) 20

21 curvaturediffusion elasticfluid

 Landmarks (fiducial markers) ◦ “Hard” constraint ◦ “Soft” constraint  Volume preservation 22

 Full  Grid ◦ Used with multi-resolution  Random ◦ Random subset of voxels is selected ◦ Improved speed 23

 Nearest Neighbour (NN)  Linear ◦ Usually sufficient during optimization  N -th order B-spline ◦ Useful for the final image (usually 3 rd order) 24

25

 Gradient Descent (GD) ◦ Linear rate of convergence  Quasi-Newton  Nonlinear Conjugate Gradient  Stochastic Gradient Descent  Evolution Strategy 26

 Gradient Descent  Quasi-Newton (QN) ◦ Can be superlinearly convergent  Nonlinear Conjugate Gradient  Stochastic Gradient Descent  Evolution Strategy 27

 Gradient Descent  Quasi-Newton  Nonlinear Conjugate Gradient (NCG) ◦ Superlinear rate of convergence can be achieved  Stochastic Gradient Descent  Evolution Strategy 28

 Gradient Descent  Quasi-Newton  Nonlinear Conjugate Gradient  Stochastic Gradient Descent (SGD) ◦ Similar to GD, but uses approximation of the gradient (Kiefer-Wolfowitz, Simultaneous Perturbation, Robbins-Monro)  Evolution Strategy 29

 Gradient Descent  Quasi-Newton  Nonlinear Conjugate Gradient  Stochastic Gradient Descent  Evolution Strategy (ES) ◦ Covariance matrix adaptation ◦ Tries several possible directions (randomly according to the covariance matrix of the cost function), the best are chosen and their weighted average is used 30

 Data complexity ◦ Gaussian pyramid ◦ Laplacian pyramid ◦ Wavelet pyramid  Transformation complexity ◦ Transformation superposition ◦ Different B-spline grid density 31

 Registration toolkit based on ITK  Handles many methods ◦ Similarity measures (SSD, NCC, MI, NMI) ◦ Transformations (rigid, affine, B-splines) ◦ Optimizers (GD, SGD-RM) ◦ Samplers, Interpolators, Multi-resolution, … 32