1. Image Registration Lecture 9: Registration Components March 22, 2005 Prof. Charlene Tsai

2. Image RegistrationLecture 8 2 Image Metrics How similar is image A to image B ?

3. Image RegistrationLecture 8 3 Image Metrics Does Image B matches Image A better than Image C ?

4. Image RegistrationLecture 8 4 Image Metrics Image AImage CImage B Match( A, B )Match( A, C ) ><><

5. Image RegistrationLecture 8 5 Image Metrics Match( A, B ) Simplest Metric Mean Squared Differences

6. Image RegistrationLecture 8 6 Mean Squared Differences Image A Image B For each pixel in A Difference( index ) = A( index ) – B( index ) Sum += Difference( index ) 2 Match( A, B ) = Sum / numberOfPixels

7. Image RegistrationLecture 8 7 Mean Squared Differences #include "itkImage.h" #include "itkMeanSquaresImageToImageMetric.h" #include "itkLinearInterpolateImageFunction.h" #include "itkTranslationTransform.h" typedef itk::Image ImageType; ImageType::ConstPointer fixedImage = GetFixedImage(); ImageType::ConstPointer movingImage = GetMovingImage(); typedef itk::LinearInterpolateImageFunction InterpolatorType; InterpolatorType::Pointer interpolator = InterpolatorType::New(); typedef itk::TranslationTransform TransformType; TransformType::Pointer transform = TransformType::New();

8. Image RegistrationLecture 8 8 Mean Squared Differences typedef itk::MeanSquaresImageToImageMetric MetricType; MetricType::Pointer metric = MetricType::New(); metric->SetInterpolator( interpolator ); metric->SetTransform( transform ); metric->SetFixedImage( fixedImage ); metric->SetMovingImage( movingImage ); MetricType::TransformParametersType translation( Dimension ); translation[0] = 12; translation[1] = 27; double value = metric->GetValue( translation );

9. Image RegistrationLecture 8 9 Evaluating many matches MetricType::TransformParametersType translation( Dimension ); double value[21][21]; for( int dx = 0; dx <= 20; dx++) { for( int dy = 0; dy <= 20; dy++) { translation[0] = dx; translation[1] = dy; value[dx][dy] = metric->GetValue( translation ); } }

10. Image RegistrationLecture 8 10 Evaluating many matches y Fixed Image Transform x y Moving Image x (0,0) mm

11. Image RegistrationLecture 8 11 Plotting the Metric Mean Squared Differences Transform Parametric Space

12. Image RegistrationLecture 8 12 Plotting the Metric Mean Squared Differences Transform Parametric Space

13. Image RegistrationLecture 8 13 Evaluating many matches y Fixed Image Transform x y Moving Image x (-15,-25) mm

14. Image RegistrationLecture 8 14 Plotting the Metric Mean Squared Differences Transform Parametric Space

15. Image RegistrationLecture 8 15 Plotting the Metric Mean Squared Differences Transform Parametric Space

16. Image RegistrationLecture 8 16 The Best Transform Parameters  Evaluation of the full parameter space is equivalent to perform optimization by exhaustive search  Very safe but very slow!  Many better optimization methods, for example gradient descent  Evaluation of the full parameter space is equivalent to perform optimization by exhaustive search  Very safe but very slow!  Many better optimization methods, for example gradient descent

17. Image RegistrationLecture 8 17 Image Registration Framework Fixed Image Moving Image Metric Transform Interpolator Optimizer Parameters

18. Image RegistrationLecture 8 18 Gradient Descent Optimizer f( x, y ) S = L ∙ G( x, y ) f( x, y ) ∆ G( x, y ) = S = Step L = Learning Rate

19. Image RegistrationLecture 8 19 Gradient Descent Optimizer f( x, y ) S = L ∙ G( x, y ) f( x, y ) ∆ G( x, y ) =

20. Image RegistrationLecture 8 20 Gradient Descent Optimizer f( x, y ) S = L ∙ G( x, y ) f( x, y ) ∆ G( x, y ) = L too large

21. Image RegistrationLecture 8 21 Gradient Descent Optimizer f( x, y ) S = L ∙ G( x, y ) f( x, y ) ∆ G( x, y ) = L too small

22. Image RegistrationLecture 8 22 Gradient Descent Optimizer What’s wrong with this algorithm ? S Units ? = millimeters f(x,y) Units ? = intensity G(x,y) Units ? = intensity / millimeters S = L ∙ G( x, y ) S is proportional to intensity change

23. Image RegistrationLecture 8 23 Gradient Descent Optimizer f( x ) S = L ∙ G( x ) 1 1

24. Image RegistrationLecture 8 24 Gradient Descent Optimizer f( x ) 1 1 S = L ∙ G( x ) S = Large in high gradients S = Small in low gradients

25. Image RegistrationLecture 8 25 Gradient Descent Optimizer f( x ) S = L ∙ G( x ) Works great with this function Works badly with this function

26. Image RegistrationLecture 8 26 Gradient Descent Variant Driving Safe !

27. Image RegistrationLecture 8 27 Regular Step Gradient Descent f( x ) S = D ∙ G( x ) ^ D i = D i-1 / 2.0 D1D1 If G changes direction D1D1 D1D1 D2D2

28. Image RegistrationLecture 8 28 Regular Step Gradient Descent f( x ) S = D ∙ G( x ) ^ D1D1 User Selects D 1 D1D1 D1D1 D2D2 User Selects D stop User Selects G stop

29. Image RegistrationLecture 8 29 Optimizers are like a car Watch while you are driving !

30. Image RegistrationLecture 8 30 Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15mm, -25mm)

31. Image RegistrationLecture 8 31 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 1.0 mm

32. Image RegistrationLecture 8 32 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 2.0 mm

33. Image RegistrationLecture 8 33 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 5.0 mm

34. Image RegistrationLecture 8 34 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 10.0 mm

35. Image RegistrationLecture 8 35 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 20.0 mm

36. Image RegistrationLecture 8 36 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 40.0 mm

37. Image RegistrationLecture 8 37 Watch over your optimizer Example: Optimizer registering an image shifted by (-15mm, -25mm) The optimizer starts at (0mm,0mm)

38. Image RegistrationLecture 8 38 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 1.0 mm

39. Image RegistrationLecture 8 39 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 2.0 mm

40. Image RegistrationLecture 8 40 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 5.0 mm

41. Image RegistrationLecture 8 41 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 10.0 mm

42. Image RegistrationLecture 8 42 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 20.0 mm

43. Image RegistrationLecture 8 43 Plotting the Optimizer’s Path Mean Squared Differences Step Length = 40.0 mm

44. Image RegistrationLecture 8 44 Other Image Metrics Normalized Correlation itkNormalizedCorrelationImageToImageMetric.h

45. Image RegistrationLecture 8 45 Normalized Correlation Image A Image B For each pixel in A SumAB += A( index ) ∙ B( index ) Match( A, B ) = SumAB / √ ( SumAA ∙ SumBB ) SumAA += A( index ) ∙ A( index ) SumBB += B( index ) ∙ B( index )

46. Image RegistrationLecture 8 46 Evaluating many matches y Fixed Image Transform x y Moving Image x

47. Image RegistrationLecture 8 47 Plotting the Metric Normalized Correlation Metric Transform Parametric Space

48. Image RegistrationLecture 8 48 Plotting the Metric Normalized Correlation Metric Transform Parametric Space

49. Image RegistrationLecture 8 49 Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15mm, -25mm)

50. Image RegistrationLecture 8 50 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 1.0 mm

51. Image RegistrationLecture 8 51 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 2.0 mm

52. Image RegistrationLecture 8 52 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 5.0 mm

53. Image RegistrationLecture 8 53 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 10.0 mm

54. Image RegistrationLecture 8 54 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 20.0 mm

55. Image RegistrationLecture 8 55 Plotting the Optimizer’s Path Normalized Correlation Metric Step Length = 40.0 mm

56. Image RegistrationLecture 8 56 Evaluating many matches y Fixed Image Transform x y Moving Image x (-15,-25) mm

57. Image RegistrationLecture 8 57 Plotting the Metric Normalized Correlation Metric Transform Parametric Space

58. Image RegistrationLecture 8 58 Plotting the Metric Normalized Correlation Metric Transform Parametric Space

59. Image RegistrationLecture 8 59 Watch over your optimizer Example: Optimizer registering an image shifted by (-15mm, -25mm) The optimizer starts at (0mm,0mm)

60. Image RegistrationLecture 8 60 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 1.0 mm

61. Image RegistrationLecture 8 61 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 2.0 mm

62. Image RegistrationLecture 8 62 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 5.0 mm

63. Image RegistrationLecture 8 63 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 10.0 mm

64. Image RegistrationLecture 8 64 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 20.0 mm

65. Image RegistrationLecture 8 65 Plotting the Optimizer Path Normalized Correlation Metric Step Length = 40.0 mm

66. Image RegistrationLecture 8 66 Other Image Metrics Mean Reciprocal Square Differences itkMeanReciprocalSquareDifferenceImageToImageMetric.h

67. Image RegistrationLecture 8 67 Mean Reciprocal Squared Differences Image A Image B For each pixel in A Difference( index ) = A( index ) – B( index ) 1 Match( A, B ) += ( 1 + λ ∙ Difference( index ) 2 )

68. Image RegistrationLecture 8 68 Evaluating many matches y Fixed Image Transform x y Moving Image x

69. Image RegistrationLecture 8 69 Plotting the Metric Mean Reciprocal Squared Differences Transform Parametric Space

70. Image RegistrationLecture 8 70 Plotting the Metric Mean Reciprocal Squared Differences Transform Parametric Space

71. Image RegistrationLecture 8 71 Evaluating many matches y Fixed Image Transform x y Moving Image x (-15,-25) mm

72. Image RegistrationLecture 8 72 Plotting the Metric Mean Reciprocal Squared Differences Transform Parametric Space

73. Image RegistrationLecture 8 73 Plotting the Metric Mean Reciprocal Squared Differences Transform Parametric Space

74. Image RegistrationLecture 8 74 Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15mm, -25mm)

75. Image RegistrationLecture 8 75 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 1.0 mm

76. Image RegistrationLecture 8 76 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 2.0 mm

77. Image RegistrationLecture 8 77 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 5.0 mm

78. Image RegistrationLecture 8 78 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 10.0 mm

79. Image RegistrationLecture 8 79 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 20.0 mm

80. Image RegistrationLecture 8 80 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 40.0 mm

81. Image RegistrationLecture 8 81 Quiz #1 If the Metric is Noisy Where is the noise coming from ?

82. Image RegistrationLecture 8 82 Smoothing the Image

83. Image RegistrationLecture 8 83 Evaluating many matches y Fixed Image Transform x y Moving Image x

84. Image RegistrationLecture 8 84 Plotting the Smoothed Metric Mean Reciprocal Squared Differences Transform Parametric Space

85. Image RegistrationLecture 8 85 Plotting the Smoothed Metric Mean Reciprocal Squared Differences Transform Parametric Space

86. Image RegistrationLecture 8 86 Watch over your optimizer Example: Optimizer registering an image with itself starting at (-15mm, -25mm)

87. Image RegistrationLecture 8 87 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 1.0 mm

88. Image RegistrationLecture 8 88 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 2.0 mm

89. Image RegistrationLecture 8 89 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 5.0 mm

90. Image RegistrationLecture 8 90 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 10.0 mm

91. Image RegistrationLecture 8 91 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 20.0 mm

92. Image RegistrationLecture 8 92 Plotting the Optimizer’s Path Mean Reciprocal Squared Differences Step Length = 40.0 mm

93. Image RegistrationLecture 8 93 Watch over your optimizer How much can you blindly trust your optimizer…

94. Image RegistrationLecture 8 94 I left my Optimizer working…

95. Image RegistrationLecture 8 95 I think it’s going to finish soon…

96. Image RegistrationLecture 8 96 Summary  Two Optimizers:  Gradient descent  Regular-step gradient descent  3 metrics  Mean squared differences  Normalized correlation metric  Mean reciprocal square differences  Watch where you’re driving!!!  Two Optimizers:  Gradient descent  Regular-step gradient descent  3 metrics  Mean squared differences  Normalized correlation metric  Mean reciprocal square differences  Watch where you’re driving!!!