1. ter Haar Romeny, Computer Vision 2014 Prof. Bart M. ter Haar Romeny, PhD Biomedical Image Analysis & Interpretation TU/e, Department of Biomedical Engineering B.M.terHaarRomeny@tue.nl

2. ter Haar Romeny, Computer Vision 2014 MR slice hartcoronair  scale toppoints graph theory Edge focusing

3. ter Haar Romeny, Computer Vision 2014 Structures exist at their own scale: Original  = e 0 px  = e 1 px  = e 2 px  = e 3 px Noise edges

4. ter Haar Romeny, Computer Vision 2014 The graph of the sign-change of the first derivative of a signal as a function of scale is denoted the scale-space signature of the signal. Zero-crossings of the second order derivative = max of first order derivative, as a function of scale

5. ter Haar Romeny, Computer Vision 2014 The notion of longevity can be viewed of a measure of importance for singularities [Witkin83]. The semantical notions of prominence and conspicuity now get a clear meaning in scale-space theory. In a scale-space we see the emergence of the hierarchy of structures. Positive and negative edges come together and annihilate in singularity points.

6. ter Haar Romeny, Computer Vision 2014 Example: Lysosome segmentation in noisy 2-photon microscopy 3D images of macrophages.

7. ter Haar Romeny, Computer Vision 2014 Marching-cubes isophote surface of the macrophage. Preprocessing: - Blur with  = 3 px - Detect N strongest maxima

8. ter Haar Romeny, Computer Vision 2014 We interpolate with cubic splines interpolation 35 radial tracks in 35 3D orientations

9. ter Haar Romeny, Computer Vision 2014 The profiles are extremely noisy: Observation: visually we can reasonably point the steepest edgepoints.

10. ter Haar Romeny, Computer Vision 2014 Edge focusing over all profiles. Choose a start level based on the task, i.e. find a single edge.

11. ter Haar Romeny, Computer Vision 2014 Detected 3D points per maximum. We need a 3D shape fit function.

12. ter Haar Romeny, Computer Vision 2014 The 3D points are least square fit with 3D spherical harmonics:

13. ter Haar Romeny, Computer Vision 2014 Resulting detection:

14. ter Haar Romeny, Computer Vision 2014 An efficient way to detect maxima and saddlepoints is found in the theory of vector field analysis (Stoke’s theorem)

15. ter Haar Romeny, Computer Vision 2014 Topological winding numbers N-D 2-D  is the wedge product (outer product for functionals)

16. ter Haar Romeny, Computer Vision 2014 In 2D: the surrounding of the point P is a closed path around P. The winding number of a point P is defined as the number of times the image gradient vector rotates over 2  when we walk over a closed path around P. maximum: = 1 minumum: = 1 regular point: = 0 saddle point: = -1 monkey saddle: = -2

17. ter Haar Romeny, Computer Vision 2014 The notion of scale appears in the size of the path. Winding number = +1  extremum Winding number = -1  saddle

18. ter Haar Romeny, Computer Vision 2014 Generalised saddle point (5 th order): (x+i y) 5 The winding numbers add within a closed contour, e.g. A saddle point (-1) and an extremum (+1) cancel, i.e. annihilate. Catastrophe theory Winding number = - 4  monkey saddle

19. ter Haar Romeny, Computer Vision 2014

20. The number of extrema and saddlepoints decrease as e -N over scale Decrease of structure over scale scales with the dimensionality.

21. ter Haar Romeny, Computer Vision 2014 Fertility Prospects In most developed countries a postponement of childbearing is seen. E.g. in the Netherlands: Average age of bearing first child is 30 years. Computer-Assisted Human Follicle Analysis for Fertility Prospects with 3D Ultrasound ter Haar Romeny et al., IPMI 1999 Application:

22. ter Haar Romeny, Computer Vision 2014 pelvis oviduct ovary uterus rectum vagina anus bladder vulva ureter clitoris Female reproductive anatomy

23. ter Haar Romeny, Computer Vision 2014 Ovary Oviduct Uterus wall Uterus Endometrium Uterus neck

24. ter Haar Romeny, Computer Vision 2014 The number of follicles decreases during lifetime

25. ter Haar Romeny, Computer Vision 2014 1. As female fecundicity decreases with advancing age, an increasing number of couples is faced with unexpected difficulties in conceiving. Approx. 15000 couples visit fertility clinics annually In 70% of these cases age-related fecundicity decline may play a role A further increase is expected 2. In our emancipated society a tension between family planning and career exists. Being young, till what age can I safely postpone childbearing? Getting older, at what age am I still likely to be able to conceive spontaneously? A further increase is expected Menopausal age

26. ter Haar Romeny, Computer Vision 2014 Resting0.03 mminitiation of growth > 120 days? Early growing0.03 - 0.1 mm Preantral0.1 - 0.2 mmbasal growth ~ 65 days Antral0.2 - 2 mm Selectable2 - 5 mmrescued by FSH window ~ 5 days Selected5 - 10 mm Dominance10 - 20 mmmaturation ~ 15 days Ovulation A follicle’s life

27. ter Haar Romeny, Computer Vision 2014 3D Ultrasound is a safe, cheap and versatile appropriate modality Kretz Medicor 530D

28. ter Haar Romeny, Computer Vision 2014 Two 3D acquisition strategies: 1. Position tracker on regular probe 2. Sweep of 2D array in transducer Trans-vaginal probe Regular sampling from irregularly space slices

29. ter Haar Romeny, Computer Vision 2014

30. Manual counting is very cumbersome  Automated follicle assessment 2-5 mm hypodense structures structured noise vessels may look like follicles ovary boundary sometimes missing

31. ter Haar Romeny, Computer Vision 2014 Detection of a singularity (i.e. a minimum) From theory of vector fields several important theorems (Stokes, Gauss) exist that relate something happening in a volume with just its surface. We can detect singularities by measurements around the singularity. P 1-D: difference of signs of the gradient  i  zero crossing or extremum ii  i i The surrounding of the point P are just 2 points left and right of P  1D sphere.

32. ter Haar Romeny, Computer Vision 2014 In subscript notation: where  ij is the antisymmetric tensor.

33. ter Haar Romeny, Computer Vision 2014 Example of a result: 1 cm Dataset 256 3, radius Stokes’ sphere 1 pixel, blurring scale 3 pixels

34. ter Haar Romeny, Computer Vision 2014 Detection of follicle boundaries: generation of 200 - 500 rays in a homogeneous orientation distribution determine most pronounced edge along ray by winding number focusing fit spherical harmonics to get an analytical description of the shape calculate volume and statistics on shape Distance along ray Scale  US intensity Scale  Distance along ray

35. ter Haar Romeny, Computer Vision 2014 3D scatterplot of detected endpoints 3D visualisation of fitted spherical harmonics function

36. ter Haar Romeny, Computer Vision 2014 Validation with 2 bovine ovaria anatomincal coupes high resolution MR 3D ultrasound

37. ter Haar Romeny, Computer Vision 2014 Conclusions: 3D ultrasound is a feasible modality for follicle-based fertilitiy state estimation automated CAD is feasible, more clinical validation needed winding numbers are robust (scaled) singularity detectors a robust class of topological properties emerges

38. ter Haar Romeny, Computer Vision 2014 Multi-scale watershed segmentation Watershed are the boundaries of merging water basins, when the image landscape is immersed by punching the minima. At larger scale the boundaries get blurred, rounded and dislocated.

39. ter Haar Romeny, Computer Vision 2014 Regions of different scales can be linked by calculating the largest overlap with the region in the scales just above.

40. ter Haar Romeny, Computer Vision 2014 The method is often combined with nonlinear diffusion schemes E. Dam, ITU

41. ter Haar Romeny, Computer Vision 2014 Nabla VisionNabla Vision is an interactive 3D watershed segmentation tool developed by the University of Copenhagen. Sculpture the 3D shape by successively clicking precalculated finer scale watershed details.