Snakes, Shapes, and Gradient Vector Flow Ayesha Mehboob Memoona Iftikhar

Content What is Snake/Active Contour Model Applications Working of traditional Snake Model Improvements to Traditional Snake GVF Snake Model Examples of GVF

Given: initial contour (model) near desirable object “Snakes” Snakes, active contours [Kass, Witkin, Terzopoulos 1987] In general, deformable models are widely used Given: initial contour (model) near desirable object

“Snakes” Snakes, active contours [Kass, Witkin, Terzopoulos 1987] Given: initial contour (model) near desirable object Goal: evolve the contour to fit exact object boundary

gradient descent w.r.t. some function describing snake’s quality A smooth 2D curve which matches to image data Initialized near target, iteratively refined Can restore missing data intermediate final initial Q: How does that work? ….

Applications

Applications

Working of Traditional Snake

Preview To find Gradient descent of 𝐹( 𝑥 𝑛 ) We have Preview To find Gradient descent of 𝐹( 𝑥 𝑛 ) 𝑥 𝑛+1 = 𝑥 𝑛 -𝛻𝐹( 𝑥 𝑛 ), 𝑛≥0 We have, 𝐹( 𝑥 0 ) ≥𝐹 𝑥 1 ≥𝐹( 𝑥 2 )≥…

Preview f(x) Q: Is snake (contour) a point in some space? ... Yes for simplicity, assume that "snake” is a vector (or point) in R1 f(x) assume some energy function f(x) describing snake’s “quality” local minima for f(x) gradient descent for 1D functions Q: Is snake (contour) a point in some space? ... Yes

Parametric Curve Representation (continuous case) A curve can be represented by 2 functions parameter closed curve open curve Note: in computer vision and medical imaging the term “snake” is commonly associated with such parametric representation of contours. (Other representations will be discussed later!)

Parametric Curve Representation (discrete case) A curve can be represented by a set of 2D points parameter

Measuring snake’s quality: Energy function We can define some energy function E(C) that assigns some number (quality measure) to all possible snakes WHY?: Somewhat philosophical question, but specifying a quality function E(C) is an objective way to define what “good” means for contours C. Moreover, one can find “the best” contour (segmentation) by optimizing energy E(C). E(C) (contours C) (scalars)

Internal energy encourages smoothness or any particular shape Energy function Usually, the total energy of snake is a combination of internal and external energies Internal energy encourages smoothness or any particular shape Internal energy incorporates prior knowledge about object boundary allowing to extract boundary even if some image data is missing External energy encourages curve onto image structures (e.g. image edges)

Internal Energy (continuous case) The smoothness energy at contour point v(s) could be evaluated as Elasticity/stretching Stiffness/bending Then, the interior energy (smoothness) of the whole snake is

Internal Energy (discrete case) elastic energy (elasticity) bending energy (stiffness)

Internal Energy (discrete case) Elasticity Stiffness

External energy The external energy describes how well the curve matches the image data locally Numerous forms can be used, attracting the curve toward different image features

External energy Suppose we have an image I(x,y) Can compute image gradient at any point Edge strength at pixel (x,y) is External energy of a contour point v=(x,y) could be continuous case External energy term for the whole snake is discrete case

elastic smoothness term Basic Elastic Snake The total energy of a basic elastic snake is continuous case discrete case elastic smoothness term (interior energy) image data term (exterior energy)

Basic Elastic Snake (discrete case) Li-1 Li Li+1 This can make a curve shrink (to a point)

Basic Elastic Snake (discrete case) The problem is to find contour that minimizes Optimization problem for function can compute local minima via gradient descent

Basic Elastic Snake Synthetic example (1) (2) (3) (4)

Basic Elastic Snake Dealing with missing data The smoothness constraint can deal with missing data:

Basic Elastic Snake Relative weighting Notice that the strength of the internal elastic component can be controlled by a parameter, Increasing this increases stiffness of curve large medium small

Discrete Snakes Optimization At each iteration we compute a new snake position within proximity to the previous snake New snake energy should be smaller than the previous one Stop when the energy can not be decreased within local neighborhood of the snake (local energy minima) Optimization Methods Gradient Descent

Gradient Descent Example: minimization of functions of 2 variables negative gradient at point (x,y) gives direction of the steepest descent towards lower values of function E

Stop at a local minima where Gradient Descent Example: minimization of functions of 2 variables update equation for a point p=(x,y) Stop at a local minima where

Gradient Descent for Snakes here, energy is a function of 2n variables C simple elastic snake energy update equation for the whole snake C

Gradient Descent for Snakes here, energy is a function of 2n variables C simple elastic snake energy update equation for each node C

Problems with snakes External energy: may need to diffuse image gradients, otherwise the snake does not really “see” object boundaries in the image unless it gets very close to it. image gradients are large only directly on the boundary

Problems with snakes (a) Convergence of a snake using (b) traditional potential forces, and (c) shown close-up within the boundary concavity.

Improvements in Traditional Snake

Alternative Way to Improve External Energy Use instead of where D() is Distance Transform (for detected binary image features, e.g. edges) binary image features (edges) Distance Transform Distance Transform can be visualized as a gray-scale image Generalized Distance Transform (directly for image gradients)

Distance Transform (see p.20-21 of the text book) Image features (2D) 3 4 2 5 1 Distance Transform Distance Transform is a function that for each image pixel p assigns a non-negative number corresponding to distance from p to the nearest feature in the image I

Problem with distance (a) Convergence of a snake using (b) distance potential forces, and (c) shown close-up within the boundary concavity.

Gradient Vector Flow (GVF) (A new external force for snakes) Detects shapes with boundary concavities. Large capture range.

Diffusing Image Gradients image gradients diffused via Gradient Vector Flow (GVF) Chenyang Xu and Jerry Prince, 98 http://iacl.ece.jhu.edu/projects/gvf/

Model for GVF snake The GVF field is defined to be a vector field V(x,y) = Force equation of GVF snake V(x,y) is defined such that it minimizes the energy functional f(x,y) is the edge map of the image.

|f | is small, energy dominated by first term ( smoothing ) |f | is large, second term dominates minimal when v= f μ is tradeoff parameter, increase with noise

Diffusion Equations GVF field can be obtained by solving following Euler equations 2 Is the Laplacian operator. Reason for detecting boundary concavities. The above equations are solved iteratively using time derivative of u and v.

Diffussion Equation The steady-state solution of these linear parabolic equations is the desired solution

For convenience, we rewrite (14) as follows:

To set up the iterative solution, let the indices i, j, and n correspond to x,y , and t, respectively,

Substituting these approximations into (15) gives our iterative solution to GVF as follows:

Stability of GVF GVF is stable whenever the step-size restriction is maintained. Since normally ∆𝑥, ∆𝑦 ,𝜇 and are fixed we find that the restriction 𝑟≤1/4, must be maintained in order to guarantee convergence of GVF:

Results of GVF Field (a) Convergence of a snake using (b) GVF external forces, and (c) shown close-up within the boundary concavity.

Traditional external force field v/s GVF field Traditional force GVF force (Diagrams courtesy “Snakes, shapes, gradient vector flow”, Xu, Prince)

Difference in Results Image with initial contour Traditional snake GVF snake

Snake Initialization and Convergence (a) Initial curve and snake results from (b) a balloon with an outward pressure, (c) a distance potential force snake, and (d) a GVF snake.

Snake Initialization and Convergence (a) Initial curve and snake results from (b) a traditional snake, (c) a distance potential force snake, and (d) a GVF snake.

References M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active contour models.“, International Journal of Computer Vision. v. 1, n. 4, pp. 321-331, 1987. Laurent D.Cohen , “Note On Active Contour Models and Balloons“, CVGIP: Image Understanding, Vol53, No.2, pp211-218, Mar. 1991. C. Xu and J.L. Prince, “Gradient Vector Flow: A New External Force for Snakes”, Proc. IEEE Conf. on Comp. Vis. Patt. Recog. (CVPR), Los Alamitos: Comp. Soc. Press, pp. 66-71, June 1997.