1. CSci 6971: Image Registration Lecture 9: Registration Components February 10, 2004 Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware

2. Image RegistrationLecture 9 2 Registration Components Basic Registration Framework

3. Image RegistrationLecture 9 3 Image Registration Framework Fixed Image Moving Image Metric Transform Interpolator Optimizer Parameters

4. Image RegistrationLecture 9 4 Other Image Metrics Mean Reciprocal Square Differences

5. Image RegistrationLecture 9 5 Mean Reciprocal Squared Differences Image A Image B For each pixel in A Difference( index ) = A( index ) – B( index ) 1 Match( A, B ) += ( 1 + λ ∙ Difference( index ) 2 )

6. Image RegistrationLecture 9 6 For each pixel in the Fixed Image Fixed Image Grid j i y x Fixed Image Physical Coordinates y’ x’ Moving Image Physical Coordinates Moving Image Grid j i Space Transform

7. Image RegistrationLecture 9 7 Mean Reciprocal Squared Differences #include "itkImage.h" #include "itkMeanReciprocalSquareDifferenceImageToImageMetric.h" #include "itkLinearInterpolateImageFunction.h" #include "itkTranslationTransform.h" typedef itk::Image ImageType; ImageType::ConstPointer fixedImage = GetFixedImage(); ImageType::ConstPointer movingImage = GetMovingImage(); typedef itk::LinearInterpolateImageFunction InterpolatorType; InterpolatorType::Pointer interpolator = InterpolatorType::New(); typedef itk::TranslationTransform TransformType; TransformType::Pointer transform = TransformType::New();

8. Image RegistrationLecture 9 8 typedef itk::MeanReciprocalSquareDifferenceImageToImageMetric MetricType; MetricType::Pointer metric = MetricType::New(); metric->SetInterpolator( interpolator ); metric->SetTransform( transform ); metric->SetFixedImage( fixedImage ); metric->SetMovingImage( movingImage ); MetricType::TransformParametersType translation( Dimension ); translation[0] = 12; translation[1] = 27; double value = metric->GetValue( translation ); Mean Reciprocal Squared Differences

9. Image RegistrationLecture 9 9 Evaluating many matches y Fixed Image Transform x y Moving Image x

10. Image RegistrationLecture 9 10 Plotting the Metric Mean Reciprocal Squared Differences Transform Parametric Space

11. Image RegistrationLecture 9 11 Plotting the Metric Mean Reciprocal Squared Differences Transform Parametric Space

12. Image RegistrationLecture 9 12 Evaluating many matches y Fixed Image Transform x y Moving Image x (-15,-25) mm

13. Image RegistrationLecture 9 13 Plotting the Metric Mean Reciprocal Squared Differences Transform Parametric Space

14. Image RegistrationLecture 9 14 Plotting the Metric Mean Reciprocal Squared Differences Transform Parametric Space

15. Image RegistrationLecture 9 15 Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15mm, -25mm)

16. Image RegistrationLecture 9 16 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 1.0 mm

17. Image RegistrationLecture 9 17 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 2.0 mm

18. Image RegistrationLecture 9 18 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 5.0 mm

19. Image RegistrationLecture 9 19 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 10.0 mm

20. Image RegistrationLecture 9 20 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 20.0 mm

21. Image RegistrationLecture 9 21 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 40.0 mm

22. Image RegistrationLecture 9 22 Quiz #1 If the Metric is Noisy Where is the noise coming from ?

23. Image RegistrationLecture 9 23 Smoothing the Image

24. Image RegistrationLecture 9 24 Evaluating many matches y Fixed Image Transform x y Moving Image x

25. Image RegistrationLecture 9 25 Plotting the Smoothed Metric Mean Reciprocal Squared Differences Transform Parametric Space

26. Image RegistrationLecture 9 26 Plotting the Smoothed Metric Mean Reciprocal Squared Differences Transform Parametric Space

27. Image RegistrationLecture 9 27 Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15mm, -25mm)

28. Image RegistrationLecture 9 28 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 1.0 mm

29. Image RegistrationLecture 9 29 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 2.0 mm

30. Image RegistrationLecture 9 30 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 5.0 mm

31. Image RegistrationLecture 9 31 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 10.0 mm

32. Image RegistrationLecture 9 32 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 20.0 mm

33. Image RegistrationLecture 9 33 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 40.0 mm

34. Image RegistrationLecture 9 34 Evaluating many matches y Fixed Image Transform x y Moving Image x (-15,-25) mm

35. Image RegistrationLecture 9 35 Plotting the Smoothed Metric Mean Reciprocal Squared Differences Transform Parametric Space

36. Image RegistrationLecture 9 36 Plotting the Smoothed Metric Mean Reciprocal Squared Differences Transform Parametric Space

37. Image RegistrationLecture 9 37 Watch over your optimizer Example: Optimizer registering an image shifted by (-15mm, -25mm) The optimizer starts at (0mm,0mm)

38. Image RegistrationLecture 9 38 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 1.0 mm

39. Image RegistrationLecture 9 39 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 2.0 mm

40. Image RegistrationLecture 9 40 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 5.0 mm

41. Image RegistrationLecture 9 41 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 10.0 mm

42. Image RegistrationLecture 9 42 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 20.0 mm

43. Image RegistrationLecture 9 43 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 40.0 mm

44. Image RegistrationLecture 9 44 Other Image Metrics Multi – Modality Registration

45. Image RegistrationLecture 9 45 Multiple Image Modalities Number of pairs

46. Image RegistrationLecture 9 46 Multiple Image Modalities More possible pairs

47. Image RegistrationLecture 9 47 Intuitive Notion of Joint Entropy The More Pairs Exist The Larger the Joint Entropy

48. Image RegistrationLecture 9 48 Mutual Information Reduction of Number of Pairs Reduction of Joint Entropy

49. Image RegistrationLecture 9 49 Mutual Information Mutual Information = Joint Entropy ( Image A, Image B ) - Entropy Image A - Entropy Image B

50. Image RegistrationLecture 9 50 Mattes Mutual Information #include "itkImage.h" #include "itkMattesMutualInformationImageToImageMetric.h" #include "itkLinearInterpolateImageFunction.h" #include "itkTranslationTransform.h" typedef itk::Image ImageType; ImageType::ConstPointer fixedImage = GetFixedImage(); ImageType::ConstPointer movingImage = GetMovingImage(); typedef itk::LinearInterpolateImageFunction InterpolatorType; InterpolatorType::Pointer interpolator = InterpolatorType::New(); typedef itk::TranslationTransform TransformType; TransformType::Pointer transform = TransformType::New();

51. Image RegistrationLecture 9 51 typedef itk::MattesMutualInformationImageToImageMetric MetricType; MetricType::Pointer metric = MetricType::New(); metric->SetNumberOfHistogramBins( 20 ); metric->SetNumberOfSpatialSamples( 10000 ); metric->SetInterpolator( interpolator ); metric->SetTransform( transform ); metric->SetFixedImage( fixedImage ); metric->SetMovingImage( movingImage ); MetricType::TransformParametersType translation( Dimension ); translation[0] = 12; translation[1] = 27; double value = metric->GetValue( translation ); Mattes Mutual Information

52. Image RegistrationLecture 9 52 Evaluating many matches y Fixed Image Transform x y Moving Image x

53. Image RegistrationLecture 9 53 Plotting the Same Modality Metric Mattes Mutual Information Transform Parametric Space

54. Image RegistrationLecture 9 54 Plotting the Same Modality Metric Mattes Mutual Information Transform Parametric Space

55. Image RegistrationLecture 9 55 Evaluating many matches y Fixed Image Transform x y Moving Image x (-15,-25) mm

56. Image RegistrationLecture 9 56 Plotting the Same Modality Metric Mattes Mutual Information Transform Parametric Space

57. Image RegistrationLecture 9 57 Plotting the Same Modality Metric Mattes Mutual Information Transform Parametric Space

58. Image RegistrationLecture 9 58 Evaluating many matches y Fixed Image Transform x y Moving Image x

59. Image RegistrationLecture 9 59 Plotting the Multi-Modality Metric Mattes Mutual Information Transform Parametric Space

60. Image RegistrationLecture 9 60 Plotting the Multi-Modality Metric Mattes Mutual Information Transform Parametric Space

61. Image RegistrationLecture 9 61 Evaluating many matches y Fixed Image Transform x y Moving Image x

62. Image RegistrationLecture 9 62 Plotting the Multi-Modality Metric Mattes Mutual Information Transform Parametric Space

63. Image RegistrationLecture 9 63 Plotting the Multi-Modality Metric Mattes Mutual Information Transform Parametric Space

64. Image RegistrationLecture 9 64 Watch over your optimizer Example: Optimizer registering two aligned multi-modality images starting at (-15mm, -25mm)

65. Image RegistrationLecture 9 65 Plotting the Optimizer’s Path Mattes Mutual Information Step Length = 1.0 mm

66. Image RegistrationLecture 9 66 Plotting the Optimizer’s Path Mattes Mutual Information Step Length = 2.0 mm

67. Image RegistrationLecture 9 67 Plotting the Optimizer’s Path Mattes Mutual Information Step Length = 5.0 mm

68. Image RegistrationLecture 9 68 Plotting the Optimizer’s Path Mattes Mutual Information Step Length = 10.0 mm

69. Image RegistrationLecture 9 69 Plotting the Optimizer’s Path Mattes Mutual Information Step Length = 20.0 mm

70. Image RegistrationLecture 9 70 Plotting the Optimizer’s Path Mattes Mutual Information Step Length = 40.0 mm

71. Image RegistrationLecture 9 71 Watch over your optimizer Example: Optimizer registering two multi-modality images shifted by (-15mm, -25mm) The optimizer starts at (0mm,0mm)

72. Image RegistrationLecture 9 72 Plotting the Optimizer’s Path Mattes Mutual Information Step Length = 1.0 mm

73. Image RegistrationLecture 9 73 Plotting the Optimizer’s Path Mattes Mutual Information Step Length = 2.0 mm

74. Image RegistrationLecture 9 74 Plotting the Optimizer’s Path Mattes Mutual Information Step Length = 5.0 mm

75. Image RegistrationLecture 9 75 Plotting the Optimizer’s Path Mattes Mutual Information Step Length = 10.0 mm

76. Image RegistrationLecture 9 76 Plotting the Optimizer’s Path Mattes Mutual Information Step Length = 20.0 mm

77. Image RegistrationLecture 9 77 Plotting the Optimizer’s Path Mattes Mutual Information Step Length = 40.0 mm

78. Image RegistrationLecture 9 78 Mutual Information Variants in ITK Viola – Wells Mattes Mutual Information Histogram

79. Image RegistrationLecture 9 79 End Enjoy ITK !