Image Registration Lecture 9: Registration Components March 22, 2005 Prof. Charlene Tsai

Image RegistrationLecture 8 2 Image Metrics How similar is image A to image B ?

Image RegistrationLecture 8 3 Image Metrics Does Image B matches Image A better than Image C ?

Image RegistrationLecture 8 4 Image Metrics Image AImage CImage B Match( A, B )Match( A, C ) ><><

Image RegistrationLecture 8 5 Image Metrics Match( A, B ) Simplest Metric Mean Squared Differences

Image RegistrationLecture 8 6 Mean Squared Differences Image A Image B For each pixel in A Difference( index ) = A( index ) – B( index ) Sum += Difference( index ) 2 Match( A, B ) = Sum / numberOfPixels

Image RegistrationLecture 8 7 Mean Squared Differences #include "itkImage.h" #include "itkMeanSquaresImageToImageMetric.h" #include "itkLinearInterpolateImageFunction.h" #include "itkTranslationTransform.h" typedef itk::Image ImageType; ImageType::ConstPointer fixedImage = GetFixedImage(); ImageType::ConstPointer movingImage = GetMovingImage(); typedef itk::LinearInterpolateImageFunction InterpolatorType; InterpolatorType::Pointer interpolator = InterpolatorType::New(); typedef itk::TranslationTransform TransformType; TransformType::Pointer transform = TransformType::New();

Image RegistrationLecture 8 8 Mean Squared Differences typedef itk::MeanSquaresImageToImageMetric MetricType; MetricType::Pointer metric = MetricType::New(); metric->SetInterpolator( interpolator ); metric->SetTransform( transform ); metric->SetFixedImage( fixedImage ); metric->SetMovingImage( movingImage ); MetricType::TransformParametersType translation( Dimension ); translation[0] = 12; translation[1] = 27; double value = metric->GetValue( translation );

Image RegistrationLecture 8 9 Evaluating many matches MetricType::TransformParametersType translation( Dimension ); double value[21][21]; for( int dx = 0; dx <= 20; dx++) { for( int dy = 0; dy <= 20; dy++) { translation[0] = dx; translation[1] = dy; value[dx][dy] = metric->GetValue( translation ); } }

Image RegistrationLecture 8 10 Evaluating many matches y Fixed Image Transform x y Moving Image x (0,0) mm

Image RegistrationLecture 8 11 Plotting the Metric Mean Squared Differences Transform Parametric Space

Image RegistrationLecture 8 12 Plotting the Metric Mean Squared Differences Transform Parametric Space

Image RegistrationLecture 8 13 Evaluating many matches y Fixed Image Transform x y Moving Image x (-15,-25) mm

Image RegistrationLecture 8 14 Plotting the Metric Mean Squared Differences Transform Parametric Space

Image RegistrationLecture 8 15 Plotting the Metric Mean Squared Differences Transform Parametric Space

Image RegistrationLecture 8 16 The Best Transform Parameters  Evaluation of the full parameter space is equivalent to perform optimization by exhaustive search  Very safe but very slow!  Many better optimization methods, for example gradient descent  Evaluation of the full parameter space is equivalent to perform optimization by exhaustive search  Very safe but very slow!  Many better optimization methods, for example gradient descent

Image RegistrationLecture 8 17 Image Registration Framework Fixed Image Moving Image Metric Transform Interpolator Optimizer Parameters

Image RegistrationLecture 8 18 Gradient Descent Optimizer f( x, y ) S = L ∙ G( x, y ) f( x, y ) ∆ G( x, y ) = S = Step L = Learning Rate

Image RegistrationLecture 8 19 Gradient Descent Optimizer f( x, y ) S = L ∙ G( x, y ) f( x, y ) ∆ G( x, y ) =

Image RegistrationLecture 8 20 Gradient Descent Optimizer f( x, y ) S = L ∙ G( x, y ) f( x, y ) ∆ G( x, y ) = L too large

Image RegistrationLecture 8 21 Gradient Descent Optimizer f( x, y ) S = L ∙ G( x, y ) f( x, y ) ∆ G( x, y ) = L too small

Image RegistrationLecture 8 22 Gradient Descent Optimizer What’s wrong with this algorithm ? S Units ? = millimeters f(x,y) Units ? = intensity G(x,y) Units ? = intensity / millimeters S = L ∙ G( x, y ) S is proportional to intensity change

Image RegistrationLecture 8 23 Gradient Descent Optimizer f( x ) S = L ∙ G( x ) 1 1

Image RegistrationLecture 8 24 Gradient Descent Optimizer f( x ) 1 1 S = L ∙ G( x ) S = Large in high gradients S = Small in low gradients

Image RegistrationLecture 8 25 Gradient Descent Optimizer f( x ) S = L ∙ G( x ) Works great with this function Works badly with this function

Image RegistrationLecture 8 26 Gradient Descent Variant Driving Safe !

Image RegistrationLecture 8 27 Regular Step Gradient Descent f( x ) S = D ∙ G( x ) ^ D i = D i-1 / 2.0 D1D1 If G changes direction D1D1 D1D1 D2D2

Image RegistrationLecture 8 28 Regular Step Gradient Descent f( x ) S = D ∙ G( x ) ^ D1D1 User Selects D 1 D1D1 D1D1 D2D2 User Selects D stop User Selects G stop

Image RegistrationLecture 8 29 Optimizers are like a car Watch while you are driving !

Image RegistrationLecture 8 30 Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15mm, -25mm)

Image RegistrationLecture 8 31 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 1.0 mm

Image RegistrationLecture 8 32 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 2.0 mm

Image RegistrationLecture 8 33 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 5.0 mm

Image RegistrationLecture 8 34 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 10.0 mm

Image RegistrationLecture 8 35 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 20.0 mm

Image RegistrationLecture 8 36 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 40.0 mm

Image RegistrationLecture 8 37 Watch over your optimizer Example: Optimizer registering an image shifted by (-15mm, -25mm) The optimizer starts at (0mm,0mm)

Image RegistrationLecture 8 38 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 1.0 mm

Image RegistrationLecture 8 39 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 2.0 mm

Image RegistrationLecture 8 40 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 5.0 mm

Image RegistrationLecture 8 41 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 10.0 mm

Image RegistrationLecture 8 42 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 20.0 mm

Image RegistrationLecture 8 43 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 40.0 mm

Image RegistrationLecture 8 44 Other Image Metrics Normalized Correlation itkNormalizedCorrelationImageToImageMetric.h

Image RegistrationLecture 8 45 Normalized Correlation Image A Image B For each pixel in A SumAB += A( index ) ∙ B( index ) Match( A, B ) = SumAB / √ ( SumAA ∙ SumBB ) SumAA += A( index ) ∙ A( index ) SumBB += B( index ) ∙ B( index )

Image RegistrationLecture 8 46 Evaluating many matches y Fixed Image Transform x y Moving Image x

Image RegistrationLecture 8 47 Plotting the Metric Normalized Correlation Metric Transform Parametric Space

Image RegistrationLecture 8 48 Plotting the Metric Normalized Correlation Metric Transform Parametric Space

Image RegistrationLecture 8 49 Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15mm, -25mm)

Image RegistrationLecture 8 50 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 1.0 mm

Image RegistrationLecture 8 51 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 2.0 mm

Image RegistrationLecture 8 52 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 5.0 mm

Image RegistrationLecture 8 53 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 10.0 mm

Image RegistrationLecture 8 54 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 20.0 mm

Image RegistrationLecture 8 55 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 40.0 mm

Image RegistrationLecture 8 56 Evaluating many matches y Fixed Image Transform x y Moving Image x (-15,-25) mm

Image RegistrationLecture 8 57 Plotting the Metric Normalized Correlation Metric Transform Parametric Space

Image RegistrationLecture 8 58 Plotting the Metric Normalized Correlation Metric Transform Parametric Space

Image RegistrationLecture 8 59 Watch over your optimizer Example: Optimizer registering an image shifted by (-15mm, -25mm) The optimizer starts at (0mm,0mm)

Image RegistrationLecture 8 60 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 1.0 mm

Image RegistrationLecture 8 61 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 2.0 mm

Image RegistrationLecture 8 62 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 5.0 mm

Image RegistrationLecture 8 63 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 10.0 mm

Image RegistrationLecture 8 64 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 20.0 mm

Image RegistrationLecture 8 65 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 40.0 mm

Image RegistrationLecture 8 66 Other Image Metrics Mean Reciprocal Square Differences itkMeanReciprocalSquareDifferenceImageToImageMetric.h

Image RegistrationLecture 8 67 Mean Reciprocal Squared Differences Image A Image B For each pixel in A Difference( index ) = A( index ) – B( index ) 1 Match( A, B ) += ( 1 + λ ∙ Difference( index ) 2 )

Image RegistrationLecture 8 68 Evaluating many matches y Fixed Image Transform x y Moving Image x

Image RegistrationLecture 8 69 Plotting the Metric Mean Reciprocal Squared Differences Transform Parametric Space

Image RegistrationLecture 8 70 Plotting the Metric Mean Reciprocal Squared Differences Transform Parametric Space

Image RegistrationLecture 8 71 Evaluating many matches y Fixed Image Transform x y Moving Image x (-15,-25) mm

Image RegistrationLecture 8 72 Plotting the Metric Mean Reciprocal Squared Differences Transform Parametric Space

Image RegistrationLecture 8 73 Plotting the Metric Mean Reciprocal Squared Differences Transform Parametric Space

Image RegistrationLecture 8 74 Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15mm, -25mm)

Image RegistrationLecture 8 75 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 1.0 mm

Image RegistrationLecture 8 76 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 2.0 mm

Image RegistrationLecture 8 77 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 5.0 mm

Image RegistrationLecture 8 78 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 10.0 mm

Image RegistrationLecture 8 79 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 20.0 mm

Image RegistrationLecture 8 80 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 40.0 mm

Image RegistrationLecture 8 81 Quiz #1 If the Metric is Noisy Where is the noise coming from ?

Image RegistrationLecture 8 82 Smoothing the Image

Image RegistrationLecture 8 83 Evaluating many matches y Fixed Image Transform x y Moving Image x

Image RegistrationLecture 8 84 Plotting the Smoothed Metric Mean Reciprocal Squared Differences Transform Parametric Space

Image RegistrationLecture 8 85 Plotting the Smoothed Metric Mean Reciprocal Squared Differences Transform Parametric Space

Image RegistrationLecture 8 86 Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15mm, -25mm)

Image RegistrationLecture 8 87 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 1.0 mm

Image RegistrationLecture 8 88 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 2.0 mm

Image RegistrationLecture 8 89 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 5.0 mm

Image RegistrationLecture 8 90 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 10.0 mm

Image RegistrationLecture 8 91 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 20.0 mm

Image RegistrationLecture 8 92 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 40.0 mm

Image RegistrationLecture 8 93 Watch over your optimizer How much can you blindly trust your optimizer…

Image RegistrationLecture 8 94 I left my Optimizer working…

Image RegistrationLecture 8 95 I think it’s going to finish soon…

Image RegistrationLecture 8 96 Summary  Two Optimizers:  Gradient descent  Regular-step gradient descent  3 metrics  Mean squared differences  Normalized correlation metric  Mean reciprocal square differences  Watch where you’re driving!!!  Two Optimizers:  Gradient descent  Regular-step gradient descent  3 metrics  Mean squared differences  Normalized correlation metric  Mean reciprocal square differences  Watch where you’re driving!!!