1. ter Haar Romeny, TU/e Mathematical Models of Contextual Operators Eindhoven University of Technology Department of Biomedical Engineering Markus van Almsick, Remco Duits, Erik Franken Bart ter Haar Romeny

2. ter Haar Romeny, TU/e Context: the Idea What a local filter sees:What a context filter sees:

3. ter Haar Romeny, TU/e Perceptual grouping (Gestalt) from orientations: robust detection Gestalt laws

4. ter Haar Romeny, TU/e Introduction Problem: segmentation of curves, contours, surfaces, etc. Methods can be distinguished by (spatial) ‘locality’ LocalGlobal Pixelwise Local filters /derivatives Context operators Active contours, ASM, etc. E.g. threshold on pixel values Pro: computationally efficient Con: only applicable on very ‘clean’ images E.g. Gaussian derivatives+threshold/local max Pro: pretty efficient Con: sensitive to noise or inconsistent data if features “live” at low scale in scale-space Optimization of global cost functional based on smoothness constraints (+ shape/texture knowledge) Pro: effective and stable on specific class of objects Con: needs initial estimate, (prior shape knowledge) Operators that take a “larger context” into account, by enhancing local features using context model. Pro: noise-robust, limited amount of prior knowledge Con: computational expensive

5. ter Haar Romeny, TU/e Context: the Empirics Angular specifity in the striate cortex: voltage sensitive dye recording of cortical colums. Similar orientations are connected (even over great distances) – “probability voting”. “Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex” W.H.Bosking, Y Zhang, Y.Schofield, D.Fitzpatrick (1997) J. Neuroscience 17:2112-2127

6. ter Haar Romeny, TU/e Goal: Extracting Edges, Lines and Surfaces from noisy, low dose, or fastly acquired medical images

7. ter Haar Romeny, TU/e Overview Invertible Orientation Bundle Transformation The output of the oriented filters spans a new transformed space, like the Fourier transform. An inverse transform can be found! Tensor Voting

8. ter Haar Romeny, TU/e Template Matching imagekernelresponse Classical filters

9. ter Haar Romeny, TU/e G-Convolution symmetry transformation g g dependence Classical filters

10. ter Haar Romeny, TU/e Linear Convolution Filter translation by b Classical filters

11. ter Haar Romeny, TU/e Wavelet Transform dilation atranslation b Classical filters

12. ter Haar Romeny, TU/e Orientation Bundle Transform rotation αtranslation b New filter family

13. ter Haar Romeny, TU/e Orientation Bundle Transform

14. ter Haar Romeny, TU/e Measures L 2 inner product by Euclidean measure L 2 inner product by Haar measure imageresponse

15. ter Haar Romeny, TU/e Inverse Transformation Kernel Constraint

16. ter Haar Romeny, TU/e Gaussian Orientation Bundle Harmonic amplitudes are constructed from the local Gaussian derivative jet

17. ter Haar Romeny, TU/e RemcoDuits: Invertible Orientation Wavelet Transform [Siam2004] Best paper award at PRIA 2004

18. ter Haar Romeny, TU/e Strong non-linear filtering in orientation space gives a much better detection of very dim lines in noise {x,y}  OS OS  OS 6 OS 6  {x,y}

19. ter Haar Romeny, TU/e Finding the very thin Adamkiewicz vessel in aorta reconstructive surgery: Not reconnecting may give spinal lesion. 3D wavelet for invertible orientation transform Noisy original Denoised vessel

20. ter Haar Romeny, TU/e Orientation Bundle Transform invertible isometric variety of admissible kernels This gives a new ‘space’ for geometric reasoning

21. ter Haar Romeny, TU/e Context: Autocorrelation of Luminosity

22. ter Haar Romeny, TU/e Autocorrelation of Edges

23. ter Haar Romeny, TU/e Autocorrelation of Lines

24. ter Haar Romeny, TU/e Autocorrelation of Lines

25. ter Haar Romeny, TU/e Tensor voting Voting kernel

26. ter Haar Romeny, TU/e Steerable Tensor Voting

27. ter Haar Romeny, TU/e Context filters for dim and broken contour detection Ultrasound kidney Context-enhanced Contour extraction Local Contour extraction

28. ter Haar Romeny, TU/e Vessel detection for Computer Aided Diagnosis in mammography E. Franken, M. van Almsick

29. ter Haar Romeny, TU/e Application: Cardiac Electrophysiology Treatment of heart rhythm disorders 1.Insertion of EP catheters 2.Recording of intracardiac electrograms 3.Ablation of problematic spot, or blocking undesired conduction path Erik Franken, 2006

30. ter Haar Romeny, TU/e Example - input Source imageLocal ridgeness   Erik Franken, 2006

31. ter Haar Romeny, TU/e Example - result  Context enhanced ridgeness * * * * * + + + + U 2 (x,y)= |U 2 |  Erik Franken, 2006

32. ter Haar Romeny, TU/e Repeated tensor voting Tensor voting  thinning  tensor voting Result after first stepResult after second step  Erik Franken, 2006

33. ter Haar Romeny, TU/e Fluoroscopy at 1/50 of the regular dose

34. ter Haar Romeny, TU/e

39. Extracted most salient paths Extraction of paths Extracted catheter tips Erik Franken, 2006

40. ter Haar Romeny, TU/e Extension of catheter tips Selection of the best extension candidate for each tip. Result: Erik Franken, 2006

41. ter Haar Romeny, TU/e Evaluation of extraction results Erik Franken, 2006

42. ter Haar Romeny, TU/e Sarcomers – bands of overlapping actine – myosine molecules in muscle fibres Orientation score - nonlinar diffusion