1. CSci 6971: Image Registration Lecture 8: Registration Components February 6, 2004 Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware

2. Image RegistrationLecture 8 2 Image Metrics How similar is image A to image B ?

3. Image RegistrationLecture 8 3 Image Metrics Does Image B matches Image A better than Image C ?

4. Image RegistrationLecture 8 4 Image Metrics Image AImage CImage B Match( A, B )Match( A, C ) ><><

5. Image RegistrationLecture 8 5 Image Metrics Match( A, B ) Simplest Metric Mean Squared Differences

6. Image RegistrationLecture 8 6 Mean Squared Differences Image A Image B For each pixel in A Difference( index ) = A( index ) – B( index ) Sum += Difference( index ) 2 Match( A, B ) = Sum / numberOfPixels

7. Image RegistrationLecture 8 7 For each pixel in the Fixed Image Fixed Image Grid j i y x Fixed Image Physical Coordinates y’ x’ Moving Image Physical Coordinates Moving Image Grid j i Space Transform

8. Image RegistrationLecture 8 8 Image Metrics Fixed Image Moving Image Metric Transform Interpolator Valu e Parameters

9. Image RegistrationLecture 8 9 Mean Squared Differences #include "itkImage.h" #include "itkMeanSquaresImageToImageMetric.h" #include "itkLinearInterpolateImageFunction.h" #include "itkTranslationTransform.h" typedef itk::Image ImageType; ImageType::ConstPointer fixedImage = GetFixedImage(); ImageType::ConstPointer movingImage = GetMovingImage(); typedef itk::LinearInterpolateImageFunction InterpolatorType; InterpolatorType::Pointer interpolator = InterpolatorType::New(); typedef itk::TranslationTransform TransformType; TransformType::Pointer transform = TransformType::New();

10. Image RegistrationLecture 8 10 Mean Squared Differences typedef itk::MeanSquaresImageToImageMetric MetricType; MetricType::Pointer metric = MetricType::New(); metric->SetInterpolator( interpolator ); metric->SetTransform( transform ); metric->SetFixedImage( fixedImage ); metric->SetMovingImage( movingImage ); MetricType::TransformParametersType translation( Dimension ); translation[0] = 12; translation[1] = 27; double value = metric->GetValue( translation );

11. Image RegistrationLecture 8 11 Mean Squared Differences MetricType::TransformParametersType translation( Dimension ); double value[21][21]; for( int dx = 0; dx <= 20; dx++) { for( int dy = 0; dy <= 20; dy++) { translation[0] = dx; translation[1] = dy; value[dx][dy] = metric->GetValue( translation ); } }

12. Image RegistrationLecture 8 12 Evaluating many matches y Fixed Image Transform x y Moving Image x

13. Image RegistrationLecture 8 13 Plotting the Metric Mean Squared Differences Transform Parametric Space

14. Image RegistrationLecture 8 14 Plotting the Metric Mean Squared Differences Transform Parametric Space

15. Image RegistrationLecture 8 15 Evaluating many matches y Fixed Image Transform x y Moving Image x (-15,-25) mm

16. Image RegistrationLecture 8 16 Plotting the Metric Mean Squared Differences Transform Parametric Space

17. Image RegistrationLecture 8 17 Plotting the Metric Mean Squared Differences Transform Parametric Space

18. Image RegistrationLecture 8 18 The Best Transform Parameters Evaluation of the full parameter space is equivalent to perform optimization by exhaustive search

19. Image RegistrationLecture 8 19 The Best Transform Parameters Very Safe but Very Slow

20. Image RegistrationLecture 8 20 The Best Transform Parameters Better Optimization Methods for example Gradient Descent

21. Image RegistrationLecture 8 21 Image Registration Framework Fixed Image Moving Image Metric Transform Interpolator Optimizer Parameters

22. Image RegistrationLecture 8 22 Gradient Descent Optimizer f( x, y ) S = L ∙ G( x, y ) f( x, y ) ∆ G( x, y ) = S = Step L = Learning Rate

23. Image RegistrationLecture 8 23 Gradient Descent Optimizer f( x, y ) S = L ∙ G( x, y ) f( x, y ) ∆ G( x, y ) =

24. Image RegistrationLecture 8 24 Gradient Descent Optimizer f( x, y ) S = L ∙ G( x, y ) f( x, y ) ∆ G( x, y ) = L too large

25. Image RegistrationLecture 8 25 Gradient Descent Optimizer f( x, y ) S = L ∙ G( x, y ) f( x, y ) ∆ G( x, y ) = L too small

26. Image RegistrationLecture 8 26 Gradient Descent Optimizer What’s wrong with this algorithm ?

27. Image RegistrationLecture 8 27 Gradient Descent Optimizer S Units ? = millimeters f(x,y) Units ? = intensity G(x,y) Units ? = intensity / millimeters L Units ? = millimeters 2 / intensity S = L ∙ G( x, y )

28. Image RegistrationLecture 8 28 Gradient Descent Optimizer f( x ) S = L ∙ G( x ) 1 1

29. Image RegistrationLecture 8 29 Gradient Descent Optimizer f( x ) 1 1 S = L ∙ G( x ) S = Large in high gradients S = Small in low gradients

30. Image RegistrationLecture 8 30 Gradient Descent Optimizer f( x ) S = L ∙ G( x ) Works great with this function Works badly with this function

31. Image RegistrationLecture 8 31 Gradient Descent Variant Driving Safe !

32. Image RegistrationLecture 8 32 Regular Step Gradient Descent f( x ) S = D ∙ G( x ) ^ D i = D i-1 / 2.0 D1D1 If G changes direction D1D1 D1D1 D2D2

33. Image RegistrationLecture 8 33 Regular Step Gradient Descent f( x ) S = D ∙ G( x ) ^ D1D1 User Selects D 1 D1D1 D1D1 D2D2 User Selects D stop User Selects G stop

34. Image RegistrationLecture 8 34 Optimizers are like a car Watch while you are driving !

35. Image RegistrationLecture 8 35 Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15mm, -25mm)

36. Image RegistrationLecture 8 36 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 1.0 mm

37. Image RegistrationLecture 8 37 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 2.0 mm

38. Image RegistrationLecture 8 38 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 5.0 mm

39. Image RegistrationLecture 8 39 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 10.0 mm

40. Image RegistrationLecture 8 40 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 20.0 mm

41. Image RegistrationLecture 8 41 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 40.0 mm

42. Image RegistrationLecture 8 42 Watch over your optimizer Example: Optimizer registering an image shifted by (-15mm, -25mm) The optimizer starts at (0mm,0mm)

43. Image RegistrationLecture 8 43 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 1.0 mm

44. Image RegistrationLecture 8 44 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 2.0 mm

45. Image RegistrationLecture 8 45 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 5.0 mm

46. Image RegistrationLecture 8 46 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 10.0 mm

47. Image RegistrationLecture 8 47 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 20.0 mm

48. Image RegistrationLecture 8 48 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 40.0 mm

49. Image RegistrationLecture 8 49 Other Image Metrics Normalized Correlation

50. Image RegistrationLecture 8 50 Normalized Correlation Image A Image B For each pixel in A SumAB += A( index ) ∙ B( index ) Match( A, B ) = SumAB / √ ( SumAA ∙ SumBB ) SumAA += A( index ) ∙ A( index ) SumBB += B( index ) ∙ B( index )

51. Image RegistrationLecture 8 51 Evaluating many matches y Fixed Image Transform x y Moving Image x

52. Image RegistrationLecture 8 52 Plotting the Metric Normalized Correlation Metric Transform Parametric Space

53. Image RegistrationLecture 8 53 Plotting the Metric Normalized Correlation Metric Transform Parametric Space

54. Image RegistrationLecture 8 54 Normalized Correlation Metric #include "itkImage.h" #include "itkNormalizedCorrelationImageToImageMetric.h" #include "itkLinearInterpolateImageFunction.h" #include "itkTranslationTransform.h" typedef itk::Image ImageType; ImageType::ConstPointer fixedImage = GetFixedImage(); ImageType::ConstPointer movingImage = GetMovingImage(); typedef itk::LinearInterpolateImageFunction InterpolatorType; InterpolatorType::Pointer interpolator = InterpolatorType::New(); typedef itk::TranslationTransform TransformType; TransformType::Pointer transform = TransformType::New();

55. Image RegistrationLecture 8 55 Normalized Correlation Metric typedef itk::NormalizedCorrelationImageToImageMetric MetricType; MetricType::Pointer metric = MetricType::New(); metric->SetInterpolator( interpolator ); metric->SetTransform( transform ); metric->SetFixedImage( fixedImage ); metric->SetMovingImage( movingImage ); MetricType::TransformParametersType translation( Dimension ); translation[0] = 12; translation[1] = 27; double value = metric->GetValue( translation );

56. Image RegistrationLecture 8 56 Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15mm, -25mm)

57. Image RegistrationLecture 8 57 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 1.0 mm

58. Image RegistrationLecture 8 58 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 2.0 mm

59. Image RegistrationLecture 8 59 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 5.0 mm

60. Image RegistrationLecture 8 60 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 10.0 mm

61. Image RegistrationLecture 8 61 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 20.0 mm

62. Image RegistrationLecture 8 62 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 40.0 mm

63. Image RegistrationLecture 8 63 Evaluating many matches y Fixed Image Transform x y Moving Image x (-15,-25) mm

64. Image RegistrationLecture 8 64 Plotting the Metric Normalized Correlation Metric Transform Parametric Space

65. Image RegistrationLecture 8 65 Plotting the Metric Normalized Correlation Metric Transform Parametric Space

66. Image RegistrationLecture 8 66 Watch over your optimizer Example: Optimizer registering an image shifted by (-15mm, -25mm) The optimizer starts at (0mm,0mm)

67. Image RegistrationLecture 8 67 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 1.0 mm

68. Image RegistrationLecture 8 68 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 2.0 mm

69. Image RegistrationLecture 8 69 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 5.0 mm

70. Image RegistrationLecture 8 70 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 10.0 mm

71. Image RegistrationLecture 8 71 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 20.0 mm

72. Image RegistrationLecture 8 72 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 40.0 mm

73. Image RegistrationLecture 8 73 Watch over your optimizer How much can you blindly trust your optimizer…

74. Image RegistrationLecture 8 74 I left my Optimizer working…

75. Image RegistrationLecture 8 75 I think it’s going to finish soon…

76. Image RegistrationLecture 8 76 End Enjoy ITK !