1. 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco Duits, Markus van Almsick Eindhoven University of Technology Department of Biomedical Engineering EURANDOM workshop “Image Analysis and Inverse Problems” December 13th 2006, Eindhoven, NL

2. 2 2 Outline Introduction to Orientation Scores Invertible Orientation Scores Operations in Orientation Scores The Direction Process for Contour completion Nonlinear diffusion for Contour enhancement Conclusions

3. 3 3 1.The retina contains receptive fields of varying sizes  multi-scale sampling device 2.Primary visual cortex is multi-orientation Biological Inspiration Cells in the primary visual cortex are orientation-specific Strong connectivity between cells that respond to (nearly) the same orientation Measurement in Primary Visual Cortex Bosking et al., J. Neuroscience 17:2112-2127, 1997

4. 4 4 image Orientation Scores From 2D image f(x,y) to orientation score U f (x,y,θ) with position (x,y) and orientation θ x y x orientation score y

5. 5 5 Orientation Score is a Function on SE(2) Properties of SE(2) Group element translation rotation Group product Group inverse

6. 6 6 Approach: Image Processing / analysis via Orientation Scores “Enhancement” operation Initial image “Enhanced” image Orientation score transformation Inverse orientation score transformation Segmented structures Segment structures of interest

7. 7 7 Outline Introduction to Orientation Scores Invertible Orientation Scores Operations in Orientation Scores The Direction Process for Contour completion Nonlinear diffusion for Contour enhancement Conclusions

8. 8 8 Invertible Orientation Score Transformation i.e. “fill up the entire Fourier spectrum”. Image to orientation score Orientation score to image: Stable reconstruction requires Oriented wavelet

9. 9 9 Invertible Orientation Score Transformation Design considerations: reconstruction, directional, spatial localization, quadrature, discrete number of orientations

10. 10 Outline Introduction to Orientation Scores Invertible Orientation Scores Operations in Orientation Scores The Direction Process for Contour completion Nonlinear diffusion for Contour enhancement Conclusions

11. 11 Represents the “net” operator. It is Euclidean- invariant iff is left-invariant, i.e. Left Invariant Operators Where is the left-regular representation

12. 12 Left-invariant Derivative Operators are left-invariant derivatives on Euclidean motion group, i.e. Not all left-invariant derivatives on SE(2) do commute! =Tangent to line structures = orthogonal to line structures

13. 13 Convection-diffusion PDEs on SE(2) convection diffusion Time process  Resolvent process

14. 14 Linear & Left-invariant Operators are G-convolutions Normal 3D convolution – versus G-convolution on SE(2) “G-Kernel”

15. 15 Outline Introduction to Orientation Scores Invertible Orientation Scores Operations in Orientation Scores The Direction Process for Contour completion Nonlinear diffusion for Contour enhancement Conclusions

16. 16 Direction Process on SE(2) Resolvent of linear PDE Random walker interpretation

17. 17 Stochastic Completion Fields Collision probability of 2 random walkers on SE(2): Forward Backward The mode line (in red) is the most likely connection curve between the two points

18. 18 An example Noisy input image Greens functions: “Simple” enhancement via Orientation score Stochastic completion field

19. 19 Exact Solution by Duits and Van Almsick Explicit PDE problem, case (Mumford) : Analytic Solution of Greens function?

20. 20 Practical approximations

21. 21 Exact Green’s Function versus Approximation The smaller the better approximation

22. 22 How? G-convolution with exact/approximate Green’s function Finite element implementation in Fourier domain (August / Duits) Explicit numerical schemes (Zweck and Williams) Application of the Direction process Non-linear enhancement step Initial image Image with completed contours Orientation score transformation Inverse orientation score transformation Compute Stochastic Completion Field What? Orientation-score gray-scale transformation (i.e. taking a power) Angular/spatial thinning

23. 23 Automatic Contour completion by SE(2)-convolution

24. 24 Outline Introduction to Orientation Scores Invertible Orientation Scores Operations in Orientation Scores The Direction Process for Contour completion Nonlinear diffusion for Contour enhancement Conclusions

25. 25 The Diffusion Equation on Images f = image u = scale space of image D = diffusion tensor Linear diffusion Perona&Malik Coherence-enhancing diff.

26. 26 Diffusion equation in orientation scores curvature Diffusion orthogonal to oriented structures Diffusion tangent to oriented structures Diffusion in orientation Evolving orientation score Rotating tangent space coordinate basis Left-invariant derivatives are left-invariant derivatives on Euclidean motion group, i.e.

27. 27 Example diffusion kernels

28. 28 Oriented regions: D’ 11 and D 33 small, D 22 large and κ according to estimate Non-oriented regions: D’ 11 large, D 22 =D 33 large, κ = 0 How to Choose Conductivity Coefficients

29. 29 Measure for Orientation Strength Hessian in Orientation Score Note: non-symmetric due to non-commuting operators! Gaussian Derivatives can be used, if one ensures to first take orientational derivatives and then spatial derivatives. Measure for orientation strength:

30. 30 Curvature estimation If a vector points tangent to a structure in the orientation score, the curvature in that point is: Ideally zero Estimation of v: 1.Determine eigenvectors and eigenvalues of 2.Select the 2 eigenvectors closest to the ξ,θ-plane 3.Take eigenvector corresponding to the largest eigenvalue

31. 31 Chosen Conductivity Coefficients

32. 32 Numerical scheme: explicit, left-invariant finite differences. Using B-spline interpolation cf. Unser et al. Implementation

33. 33 Diffusion in orientation scoreCoherence enhancing diffusion Results Size: 128 x 128 x 64

34. 34 Collagen image Diffusion in orientation score Coherence enhancing diffusion Size: 200 x 200 x 64

35. 35 Results – with/without curvature estimation

36. 36 Outline Introduction to Orientation Scores Invertible Orientation Scores Operations in Orientation Scores The Direction Process for Contour completion Nonlinear diffusion for Contour enhancement Conclusions

37. 37 Conclusions We developed a framework for image processing via Orientation scores. Important notion: An orientation score is a function on SE(2)  use group theory. Useful for noisy medical images with (crossing) elongated structures Found Analytic Solution of Greens functions Stochastic Completion Fields of images using G-convolutions Non-linear diffusion on orientation scores to enhance crossing line structures

38. 38 Current/Future work Improving adaptive/nonlinear evolutions on SE(2) –Numerical methods –Nonlinearities Applying in medical applications –2-photon microscopy images of Collagen fibers –High Angular Resolution Diffusion Imaging Apply same mathematics in other groups, e.g. SE(3) and similitude groups.

39. 39

40. 40 Line enhancement in 3D via invertible orientation scores Application: Enhancement Adam-Kiewitz vessel

41. 41 Enhancement Kidney-Boundaries in Ultra-sound images Via Orientation Scores:

42. 42 Medical Application: Cardiac Arrhythmias H eart rythm disorder by extra conductive path/spot Catheters in hart provide intracardiogram and can burn focal spots/lines Navigation by X-ray Detection cathethers in X-ray navigation automatic 3D-cardiac mapping from bi-plane.

43. 43 Efficient calculation of G-convolutions Using steerable filters in the orientation score + inspired by Fourier transform on SE(2) Algorithmic complexity can be reduced from to