1. ter Haar Romeny, FEV MR slice hartcoronair  scale toppoints graph theory Edge focusing

2. ter Haar Romeny, FEV Structures exist at their own scale: Original  = e 0 px  = e 1 px  = e 2 px  = e 3 px Noise edges

3. ter Haar Romeny, FEV The graph of the sign-change of the first derivative of a signal as a function of scale is denoted the scale-space signature of the signal. Zero-crossings of the second order derivative = max of first order derivative, as a function of scale

4. ter Haar Romeny, FEV The notion of longevity can be viewed of a measure of importance for singularities [Witkin83]. The semantical notions of prominence and conspicuity now get a clear meaning in scale-space theory. In a scale-space we see the emergence of the hierarchy of structures. Positive and negative edges come together and annihilate in singularity points.

5. ter Haar Romeny, FEV Example: Lysosome segmentation in noisy 2-photon microscopy 3D images of macrophages.

6. ter Haar Romeny, FEV Marching-cubes isophote surface of the macrophage. Preprocessing: - Blur with  = 3 px - Detect N strongest maxima

7. ter Haar Romeny, FEV We interpolate with cubic splines interpolation 35 radial tracks in 35 3D orientations

8. ter Haar Romeny, FEV The profiles are extremely noisy: Observation: visually we can reasonably point the steepest edgepoints.

9. ter Haar Romeny, FEV Edge focusing over all profiles. Choose a start level based on the task, i.e. find a single edge.

10. ter Haar Romeny, FEV Detected 3D points per maximum. We need a 3D shape fit function.

11. ter Haar Romeny, FEV The 3D points are least square fit with 3D spherical harmonics:

12. ter Haar Romeny, FEV Resulting detection:

13. ter Haar Romeny, FEV An efficient way to detect maxima and saddlepoints is found in the theory of vector field analysis (Stoke’s theorem)

14. ter Haar Romeny, FEV Topological winding numbers N-D 2-D  is the wedge product (outer product for functionals)

15. ter Haar Romeny, FEV In 2D: the surrounding of the point P is a closed path around P. The winding number of a point P is defined as the number of times the image gradient vector rotates over 2  when we walk over a closed path around P. maximum: = 1 minumum: = 1 regular point: = 0 saddle point: = -1 monkey saddle: = -2

16. ter Haar Romeny, FEV The notion of scale appears in the size of the path. Winding number = +1  extremum Winding number = -1  saddle

17. ter Haar Romeny, FEV Generalised saddle point (5 th order): (x+i y) 5 The winding numbers add within a closed contour, e.g. A saddle point (-1) and an extremum (+1) cancel, i.e. annihilate. Catastrophe theory Winding number = - 4  monkey saddle

18. ter Haar Romeny, FEV

19. The number of extrema and saddlepoints decrease as e -N over scale Decrease of structure over scale scales with the dimensionality.

20. ter Haar Romeny, FEV Fertility Prospects In most developed countries a postponement of childbearing is seen. E.g. in the Netherlands: Average age of bearing first child is 30 years. Computer-Assisted Human Follicle Analysis for Fertility Prospects with 3D Ultrasound ter Haar Romeny et al., IPMI 1999 Application:

21. ter Haar Romeny, FEV pelvis oviduct ovary uterus rectum vagina anus bladder vulva ureter clitoris Female reproductive anatomy

22. ter Haar Romeny, FEV Ovary Oviduct Uterus wall Uterus Endometrium Uterus neck

23. ter Haar Romeny, FEV The number of follicles decreases during lifetime

24. ter Haar Romeny, FEV 1. As female fecundicity decreases with advancing age, an increasing number of couples is faced with unexpected difficulties in conceiving. Approx. 15000 couples visit fertility clinics annually In 70% of these cases age-related fecundicity decline may play a role A further increase is expected 2. In our emancipated society a tension between family planning and career exists. Being young, till what age can I safely postpone childbearing? Getting older, at what age am I still likely to be able to conceive spontaneously? A further increase is expected Menopausal age

25. ter Haar Romeny, FEV Resting0.03 mminitiation of growth > 120 days? Early growing0.03 - 0.1 mm Preantral0.1 - 0.2 mmbasal growth ~ 65 days Antral0.2 - 2 mm Selectable2 - 5 mmrescued by FSH window ~ 5 days Selected5 - 10 mm Dominance10 - 20 mmmaturation ~ 15 days Ovulation A follicle’s life

26. ter Haar Romeny, FEV 3D Ultrasound is a safe, cheap and versatile appropriate modality Kretz Medicor 530D

27. ter Haar Romeny, FEV Two 3D acquisition strategies: 1. Position tracker on regular probe 2. Sweep of 2D array in transducer Trans-vaginal probe Regular sampling from irregularly space slices

28. ter Haar Romeny, FEV

30. Manual counting is very cumbersome  Automated follicle assessment 2-5 mm hypodense structures structured noise vessels may look like follicles ovary boundary sometimes missing

31. ter Haar Romeny, FEV Automated method: 1. Detection of intensity minima by 3D ‘winding numbers’ 2. Isotropic ray tracing (500 directions) from detected centra 3. Edge detection by 1D winding numbers 4. Edge focusing to detect most prominent edge 5. Fit spherical harmonics to edgepoints 6. Calculate follicle shape/size parameters and visualize

32. ter Haar Romeny, FEV Detection of a singularity (i.e. a minimum) From theory of vector fields several important theorems (Stokes, Gauss) exist that relate something happening in a volume with just its surface. We can detect singularities by measurements around the singularity. P 1-D: difference of signs of the gradient  i  zero crossing or extremum ii  i i The surrounding of the point P are just 2 points left and right of P  1D sphere.

33. ter Haar Romeny, FEV We consider a unit gradient vector, so  1 2 +  2 2 =1. In subscript notation: where  ij is the antisymmetric tensor.

34. ter Haar Romeny, FEV For regular points, i.e. when no singularity is present in W, the winding number is zero, as we see from the Stokes’ theorem: where the fact that the (d-1)-form  is a closed form was used. So, as most of our datapoints are regular, we detect singularities very robustly as integer values embedded in a space of zero's.

35. ter Haar Romeny, FEV Example of a result: 1 cm Dataset 256 3, radius Stokes’ sphere 1 pixel, blurring scale 3 pixels

36. ter Haar Romeny, FEV a conservation of winding number within the closed contour. We measure the sum of the winding numbers. E.g. enclosing a saddlepoint and a minimum adds up to zero. the winding number is independent of the shape of  W. It is a topological entity. the winding number only takes integer values. Multiples of the full rotation angle. Eeven when the numerical addition of angles does not sum up to precisely an integer value, we may rightly round off to the nearest integer. The winding number has nice properties:

37. ter Haar Romeny, FEV the winding number is a scaled notion The neighbourhood defines the scale. the behaviour over scale generates a tree-like structure Typical annihilations, creations and collisions, from which much can be learned about the ‘deep structure’ of images. the winding number is easy to compute, in any dimension. the WN is a robust characterisation of the singular points in the image: small deformations have a small effect.

38. ter Haar Romeny, FEV Detection of follicle boundaries: generation of 200 - 500 rays in a homogeneous orientation distribution determine most pronounced edge along ray by winding number focusing fit spherical harmonics to get an analytical description of the shape calculate volume and statistics on shape Distance along ray Scale  US intensity Scale  Distance along ray

39. ter Haar Romeny, FEV 3D scatterplot of detected endpoints 3D visualisation of fitted spherical harmonics function

40. ter Haar Romeny, FEV

41. Validation with 2 bovine ovaria anatomincal coupes high resolution MR 3D ultrasound

42. ter Haar Romeny, FEV Patient studies: Performance of the algorithm compared with a human expert. Number of follicles found. Data for 6 patients. The datasets are cut off to contain only the ovary. Scales used:  = 3.6, 4.8, 7.2 and 12 pixels.

43. ter Haar Romeny, FEV Conclusions: 3D ultrasound is a feasible modality for follicle-based fertilitiy state estimation automated CAD is feasible, more clinical validation needed winding numbers are robust (scaled) singularity detectors a robust class of topological properties emerges

44. ter Haar Romeny, FEV Multi-scale watershed segmentation Watershed are the boundaries of merging water basins, when the image landscape is immersed by punching the minima. At larger scale the boundaries get blurred, rounded and dislocated.

45. ter Haar Romeny, FEV Regions of different scales can be linked by calculating the largest overlap with the region in the scales just above.

46. ter Haar Romeny, FEV The method is often combined with nonlinear diffusion schemes E. Dam, ITU

47. ter Haar Romeny, FEV Nabla VisionNabla Vision is an interactive 3D watershed segmentation tool developed by the University of Copenhagen. Sculpture the 3D shape by successively clicking precalculated finer scale watershed details.

48. ter Haar Romeny, FEV

49. We expand the left and right hand side of the last equation in a Taylor series up to first order in  and  1 respectively. For the left hand side we obtain 3D winding number And for the righthand side

50. ter Haar Romeny, FEV In n-D: In 3-D: This expression has to be evaluated for all voxels of our closed surface. We can do this e.g. for the 6 planes of the surrounding cube. On the surface z = constant the previous equation reduces to Contraction of indices:

51. ter Haar Romeny, FEV Performing the contraction on the indices i, j and k gives Calculation of the gradient vector elements  i = {  x,  y,  z } the derivatives of the gradient field, e.g.  x  y =   y /  x is done by neighbour subtraction. The single pixel steps dx and dy are unity.