1. 1 Lecture #5 Variational Approaches and Image Segmentation Lecture #5 Hossam Abdelmunim 1 & Aly A. Farag 2 1 Computer & Systems Engineering Department, Ain Shams University, Cairo, Egypt 2 Electerical and Computer Engineering Department, University of Louisville, Louisville, KY, USA ECE 643 – Fall 2010

2. Variational Formulation in Computer Vision and Image Analysis Image/Curve/Surface is required to evolve to minimize a certain energy function. Image/Curve/Surface is required to evolve to minimize a certain energy function. The change of the Image/Curve/Surfaceis described by a partial differential equation (PDE). The change of the Image/Curve/Surface is described by a partial differential equation (PDE). In image segmentation, the evolution continuous until the curve/surface hits the object (region) boundaries. In image segmentation, the evolution continuous until the curve/surface hits the object (region) boundaries.

3. Advantages of Introducing the Variational Formulation to the Computer Vision Field Modeling images in continuous time domain which simplifies the formalism and makes it grid independent. Research on numerical analysis makes it possible to achieve high speed, accurate, and stable solutions of the derived PDE’s. Viscosity solutions provide a framework for dealing with non- smooth signals used with these approaches. Formal analysis in this area give the possibility of providing successful algorithms and useful theoretical results like existence and uniqueness of the solution.

4. Euler Lagrange & Gradient Descent (Recall) Given a real function: Define the following energy functional: The Euler Lagrange equation will be: The Gradient Descent will be:

5. Classical Deformable Models

6. Parametric Planar Curve Representation C(0) C(0.5) C(1) Parametric Form: Arc Length is:

7. Snake/Contour Evolution The contour is required to evolve minimizing the following energy function: Contour characteristicsImage data (force)

8. Internal Energy w 1 and w 2 are non negative. The term weighted by w 1 reduces curve length and removes ripples. The second term controls the bending and smoothness of the contour.

9. External Energy G is a Gaussian smoothing filter with width. λ is the magnitude of the external force. This term derives the contour towards the object boundaries.

10. Vector Valued PDE Using calculus of variation, the contour evolves according to the following equation:

11. u v Parametric Form: Energy: Parametric Surfaces Representation and Evolution Complete the derivation.

12. Example Aorta segmentation using active contours

13. Problems of Classical Deformable Models Dependency on the parameterization. Troubles with topology changes (merging/splitting). Initialization needs to be near the final solution (a big problem in 3D). Tuning of the weighting coefficients in the PDE (w’s, λ).

14. Level Sets

15. Curve Representation using Level Sets Zero Level Set or Z=0 plane Resulting Curve (Evolving Front) Invented by Osher and Sethian 1985, to overcome disadvantages of classical deformable models. Add extra dimension to the problem

16. Topology Change Topological changes in the evolving front are handled naturally. No parameterization is needed. This representation allows merging and breaking of the front as t advances

17. Level Sets and The Signed Distance Function 17 D is the min Euclidean distance between X and the contour. Ф is continuous and differentiable around the zero level.

18. Example of Signed Distance Functions wNwN

19. Example of Image Representation Image Representation

20. Applications of the Distance Transform (DT) The DT is a fundamental operator in shape analysis, having numerous applications, some of which are briefly listed bellow. Separation of overlapping objects through watershed segmentation. Computation of morphological operators. Computation of geometrical representations and measures, such as skeletonization. Shape matching. Shape measures related to distance. Other interesting areas in which the DT has been applied are: image registration subpixel contour tracing multidimensional data analysis (classification, clustering) image enhancement ray-tracing optimization embedding surface for level set-based methods efficient belief-propagation multiphase flow simulations medical image analysis analysis of interaction between biological structures

21. Signed Distance Generation Complexity So, to compute the distance function you need to get the closest point on the boundary/curve for each position in the domain. This process is very time-consuming especially for volumes in 3D and higher. Some approximation methods are used to expedite this process. See the following slides.

22. Distance Transform Generation Faster Algorithm ∞∞∞∞∞∞∞∞∞∞∞∞ ∞∞∞∞∞∞∞∞∞∞∞∞ ∞∞∞∞∞∞∞∞∞∞∞∞ ∞∞∞00000∞∞∞∞ ∞∞∞0∞∞∞0∞∞∞∞ ∞∞∞0∞∞∞0∞∞∞∞ ∞∞∞0∞∞∞0∞∞∞∞ ∞∞∞0∞∞∞0∞∞∞∞ ∞∞∞00000∞∞∞∞ ∞∞∞∞∞∞∞∞∞∞∞∞ ∞∞∞∞∞∞∞∞∞∞∞∞ 1.Set boundary points to zeroes while others to ∞. 2.Initialize an auxiliary set of pixels (usually a queue), the so-called contour set, with the boundary pixels and their zero values. 3.While the contour set is not empty do the following:- a)Remove a pixel from the contour set. Let us call it the central pixel (x p,y p ) now. b)For every pixel (x q,y q ) of the neighbors of the central pixel, do the following:- I.Compute the distance to the central point as: dd=|x p -x q |+|y p -y q |; II.Compute a temporary distance as : dtq=min(dist(p)+dd,dist(q)); III.If dtq < dist(q), add the pixel q to the contour set; IV.Dist(q)=dtq;

23. Distance Transform Generation Faster Algorithm (Cont…) Note that computing the distance as the difference is faster than the square root function. Also, this methodology updates the distance values around the boundary points/pixels until there are no points in the queue. Try this method with the dynamic implementation using linked lists. It will be much faster than the ordinary methods…. Assignment: Implement using C++. Apply it for 2D as well as 3D shapes.