Perceptual Grouping: The Closure of Gaps within Elongated Structures in Medical Images Renske de Boer March 23 rd, /mhj Committee: prof. dr. ir. B.M. ter Haar Romeny prof. dr. ir. F.N. van de Vosse dr. L.M.J. Florack dr. ir. R. Duits ir. E.M. Franken

Contents 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n 1.Introduction 2.Orientation scores Cake kernels 3.Linear operations G-Convolution Stochastic completion kernel 4.Non-linear operations Probability density function 5 non-linear operations 5.Experiments Measures for gap filling Non-linear operations Noise robustness Probability density function Examples artificial images Examples medical images 6.Conclusion Conclusions Recommendations

Introduction IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n

Introduction 1212 Bosking et al., 1997 Ts’o et al., 1990 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n

Orientation scores D image f(x,y)  ‘2D + orientation’ score U f (x,y,  ) with position (x,y) and local orientation . IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n

Orientation scores 1212 with and the rotation matrix: Orientation score is obtained by wavelet transformation of image: IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Reconstruction of image is possible.

Orientation scores 1212 Advantage: easy reconstruction of image. IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Different possibilities for kernel For example cake kernels

Linear operations 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Normal convolution G-convolution, where G is the Euclidean motion group

Linear operations 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Stochastic completion kernel creates probability density field for line continuation.

Linear operations 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Filling gaps in line structures

Non-linear operations 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Probability density function (PDF) Stochastic completion kernel should be applied to an orientation score containing the probability density of lines in the image. Probability is obtained by creating 2D-histogram of 2 features: gray values of the image and orientation score values. Only for line structures. The Bayesian theorem is used to calculate the desired probability. PDF is estimated by kernel density estimation of the histogram.

Non-linear operations 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n 1 Orientation score thinning Thinning with a certain number of pixels of the orientation score in both the spatial dimensions and the orientation dimension. 2 Angular thinning The two orientations that give maximum orientation score responses get values, all other orientations are put to zero.

Non-linear operations 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n 3 Pyramid thinning

Non-linear operations 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n 4 Normal power enhancement 5 Power enhancement

Experiments 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Measures for gap filling: mean filling Ratio of mean gap filling and mean of line structure. min filling Ratio of minimal gap filling and mean line structure. background ratio Ratio of mean background and mean of line structure and gap. fill back Ratio of mean gap filling and mean background.

Experiments 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Evaluation of non-linear operations 1 Orientation score thinning 4 Normal power enhancement 5 Power enhancement

Experiments 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Noise robustness NSR = 0.5NSR = 2.5NSR = 5.0  N = 0.5  N = 1.5  N = 2.5

Experiments 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Noise robustness NSR is noise to signal ratio  N is scale of Gaussian correlation of noise Filling measure Correlation of noise

Experiments 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Obtained PDF Used ground truth Probability density function

Experiments 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Examples of artificial images

Experiments 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Examples of artificial images

Experiments 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n – Examples of medical images OriginalAfter preprocessingResult [1] [2]

Experiments 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n – Examples of medical images OriginalAfter preprocessingResult [3]

Conclusion 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Conclusions New method is successfully generated. Method performs reasonable on lower noise levels. PDF detects line structures but filling of gap is weak due to enlargement of the gap. Larger gaps or high curvature result in weaker filling. Some undesired filling might be caused by line structures that are close together and are not part of the same line. Medical images need a lot of preprocessing to prevent background artifacts. Overall the method gives reasonable results for filling gaps and enhancing line structures.

Conclusion 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Recommendations Deblurring afterwards is necessary! Include curvature in stochastic completion kernel to fill gaps in lines with high curvature. Adjust width of cake kernels to detect lines at different scales. Optimize parameters of stochastic completion kernel. Find a better non-linear enhancement operation. Improve PDF results by creating extra PDF for line endings. Preprocess (medical) images!

Acknowledgment 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n Thanks to: Committee, especially Erik, Remco and Markus. Family, friends and housemates. Questions? [1] [2] J.J. Staal, M.D. Abramoff, M. Niemeijer, M.A. Viergever, and B. van Ginneken. Ridge based vessel segmentation in color images of the retina. IEEE Transactions on Medical Imaging, 23(4): 501–509, [3] Philips StentBoost

Timing 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n ParameterTest 1 Test 2 Test 3 Number of samples in x and y in orientation score Number of orientation layers in orientation score Number of samples in x and y in kernel41 Number of orientation layers in kernel9179 Number of m-components of steerable kernel11 Time in seconds of normal G-convolution Time in seconds of steerable G-convolution

G-correlation 1212 IntroductionOrientation scores Linear operations Non-linear operations Experiment s Conclusio n G-correlation Result of G-correlation with delta function is not the kernel, therefore G-convolution is used. Relation to G-convolution with