ter Haar Romeny, TU/e Mathematical Models of Contextual Operators Eindhoven University of Technology Department of Biomedical Engineering Markus van Almsick, Remco Duits, Erik Franken Bart ter Haar Romeny

ter Haar Romeny, TU/e Context: the Idea What a local filter sees:What a context filter sees:

ter Haar Romeny, TU/e Perceptual grouping (Gestalt) from orientations: robust detection Gestalt laws

ter Haar Romeny, TU/e Introduction Problem: segmentation of curves, contours, surfaces, etc. Methods can be distinguished by (spatial) ‘locality’ LocalGlobal Pixelwise Local filters /derivatives Context operators Active contours, ASM, etc. E.g. threshold on pixel values Pro: computationally efficient Con: only applicable on very ‘clean’ images E.g. Gaussian derivatives+threshold/local max Pro: pretty efficient Con: sensitive to noise or inconsistent data if features “live” at low scale in scale-space Optimization of global cost functional based on smoothness constraints (+ shape/texture knowledge) Pro: effective and stable on specific class of objects Con: needs initial estimate, (prior shape knowledge) Operators that take a “larger context” into account, by enhancing local features using context model. Pro: noise-robust, limited amount of prior knowledge Con: computational expensive

ter Haar Romeny, TU/e Context: the Empirics Angular specifity in the striate cortex: voltage sensitive dye recording of cortical colums. Similar orientations are connected (even over great distances) – “probability voting”. “Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex” W.H.Bosking, Y Zhang, Y.Schofield, D.Fitzpatrick (1997) J. Neuroscience 17:

ter Haar Romeny, TU/e Goal: Extracting Edges, Lines and Surfaces from noisy, low dose, or fastly acquired medical images

ter Haar Romeny, TU/e Overview Invertible Orientation Bundle Transformation The output of the oriented filters spans a new transformed space, like the Fourier transform. An inverse transform can be found! Tensor Voting

ter Haar Romeny, TU/e Template Matching imagekernelresponse Classical filters

ter Haar Romeny, TU/e G-Convolution symmetry transformation g g dependence Classical filters

ter Haar Romeny, TU/e Linear Convolution Filter translation by b Classical filters

ter Haar Romeny, TU/e Wavelet Transform dilation atranslation b Classical filters

ter Haar Romeny, TU/e Orientation Bundle Transform rotation αtranslation b New filter family

ter Haar Romeny, TU/e Orientation Bundle Transform

ter Haar Romeny, TU/e Measures L 2 inner product by Euclidean measure L 2 inner product by Haar measure imageresponse

ter Haar Romeny, TU/e Inverse Transformation Kernel Constraint

ter Haar Romeny, TU/e Gaussian Orientation Bundle Harmonic amplitudes are constructed from the local Gaussian derivative jet

ter Haar Romeny, TU/e RemcoDuits: Invertible Orientation Wavelet Transform [Siam2004] Best paper award at PRIA 2004

ter Haar Romeny, TU/e Strong non-linear filtering in orientation space gives a much better detection of very dim lines in noise {x,y}  OS OS  OS 6 OS 6  {x,y}

ter Haar Romeny, TU/e Finding the very thin Adamkiewicz vessel in aorta reconstructive surgery: Not reconnecting may give spinal lesion. 3D wavelet for invertible orientation transform Noisy original Denoised vessel

ter Haar Romeny, TU/e Orientation Bundle Transform invertible isometric variety of admissible kernels This gives a new ‘space’ for geometric reasoning

ter Haar Romeny, TU/e Context: Autocorrelation of Luminosity

ter Haar Romeny, TU/e Autocorrelation of Edges

ter Haar Romeny, TU/e Autocorrelation of Lines

ter Haar Romeny, TU/e Autocorrelation of Lines

ter Haar Romeny, TU/e Tensor voting Voting kernel

ter Haar Romeny, TU/e Steerable Tensor Voting

ter Haar Romeny, TU/e Context filters for dim and broken contour detection Ultrasound kidney Context-enhanced Contour extraction Local Contour extraction

ter Haar Romeny, TU/e Vessel detection for Computer Aided Diagnosis in mammography E. Franken, M. van Almsick

ter Haar Romeny, TU/e Application: Cardiac Electrophysiology Treatment of heart rhythm disorders 1.Insertion of EP catheters 2.Recording of intracardiac electrograms 3.Ablation of problematic spot, or blocking undesired conduction path Erik Franken, 2006

ter Haar Romeny, TU/e Example - input Source imageLocal ridgeness   Erik Franken, 2006

ter Haar Romeny, TU/e Example - result  Context enhanced ridgeness * * * * * U 2 (x,y)= |U 2 |  Erik Franken, 2006

ter Haar Romeny, TU/e Repeated tensor voting Tensor voting  thinning  tensor voting Result after first stepResult after second step  Erik Franken, 2006

ter Haar Romeny, TU/e Fluoroscopy at 1/50 of the regular dose

ter Haar Romeny, TU/e

Extracted most salient paths Extraction of paths Extracted catheter tips Erik Franken, 2006

ter Haar Romeny, TU/e Extension of catheter tips Selection of the best extension candidate for each tip. Result: Erik Franken, 2006

ter Haar Romeny, TU/e Evaluation of extraction results Erik Franken, 2006

ter Haar Romeny, TU/e Sarcomers – bands of overlapping actine – myosine molecules in muscle fibres Orientation score - nonlinar diffusion