Chapter 7 – Segmentation II 7.1. Mean Shift Segmentation 7.2. Active Contour Models – Snakes 7.3. Geometric Deformable Models – Level Sets and Geodesic Active Contours 7.4. Fuzzy Connectivity 7.5. Towards 3D Graph-Based Image Segmentation 7.6. Graph Cut-Segmentation 7.7. Optimal Single and Multiple Surface Segmentation

7.1. Mean Shift Segmentation Idea of mean shift: 1. For each data point Fix a window around the data point. Compute the mean of data within the window. 2. Shift data points to their computed means simultaneously 3. Repeat till convergence. 7-1

Window Center of mass Mean Shift vector 7-2 2

7-3 3

7-4 4

7-5 5

Data points Objective: Find clusters Find modes Density estimator Density distribution

In practice, radially symmetric kernels are used, i.e., Density kernel K(x) Uniform Epanechnikov Normal In practice, radially symmetric kernels are used, i.e., where k : profile of kernel K 7-7

Profile of kernel K(x) Uniform Epanechnikov Normal

Given n data points in d-D space , the density estimation at point x : We are interested in locating x for which The density gradient: Use profile, i.e., (See next page)

Let Then 7-10

Assume the derivative of k is let

Let g(x) be the profile of kernel G(x), i.e., The first term is proportional to a density estimator computed with kernel 7-12

The second term is the mean shift vector The successive location of the kernel G are 7-13

Example: Mode Detection Feature vector of pixel i, (L u v)- color space Color image

Example: Image Segmentation Feature vector of pixel i, where : spatial-domain part, : range-domain part e.g.,

Two steps of mean-shift image segmentation: 1) Discontinuity Preserving Filtering -- Preserves discontinuities of images 7-17

Let : pixels of the original image : pixels of the filtered image 2) Mean Shift Clustering -- Regularizes regions Given : the mode associated with known from Step 1. 7-18

where : segmentation label of pixel i : convergence points known from Step 1 BOA : Basin Of Attraction, the set of all locations that converge to the same mode

Examples: (1) Image Segmentation (2) Image Smoothing and Segmentation Original image Smoothed image Segmentation image 7-20

Gary levels of original image The path of mean shift Gary levels after smoothing Gary levels after segmentation 7-21

7.2. Active Contour Models – Snakes -- An energy-minimization approach -- (i) A snake is a deformable model whose energy depends on its shape and location in the image (ii) Local minima of the energy correspond to desired image properties. (iii) Initial snake position should be provided.

7.2.1 Traditional Snakes and Balloons The energy functional of the snake, which is to be minimized is a weighted combination of internal and external forces The internal forces emanate from the shape of the snake The external forces come from the image and constraints

The image forces may come from lines, edges, and terminations

The constraint forces come from user specified properties to be imposed on the snake, e.g., smmothness.

To minimize , let This leads to the Euler-Lagrange motion equation Substitute into the equation written this equation in an evolution form Stop when

A balloon, which can be inflated, is extended from the snake by including an additional pressure force so that it can overcome small isolated barriers 7-28

7.2.3 Gradient Vector Flow Snakes Difficulties of previous approaches: (i) initialization , (ii) concaves of boundaries GVF: An external force field points toward boundaries when in their proximity

GVF is derived from image by minimizing an energy functional g can be obtained by solving the following Euler equations, which are diffusion equations Rewrite the above equations in an evolution form

Let in Eqs. (7.15) and (7.22) forming the GVF snake equation 7-31

3D GVF:

Deformable models, Fourier deformable model, 7.2.2 Extensions Deformable models, Fourier deformable model, Finite element snakes, B-snakes, united snakes. 7-34

7.3. Geometric Deformable Models (GDM) – Level Sets and Geodesic Active Contours 2 main groups of deformable models: 1. Parametric deformable model: Borders are represented in a parametric form, e.g., snakes 2. Geometric deformable model : Borders are represented by partial differential equations, e.g., level sets, geodesic active contours Advantages of GDM: (i) Curves are evolved using only geometric computations, independent of any parameterization

(ii) Curves are represented as level sets of higher dimensional functions yielding seamless treatment of topological changes (iii) Multiple object can be detected simultaneously Let a curve be denoted by or by positional vector The associated length function: : speed function

Assume the curve moves only in a direction normal (N) to itself, i.e. Assume also the speed function is a function of curvature c, 7-37

Constant Deformation Equation -- Deformation is similar to inflation balloon force and may introduce singularities (e.g., sharp corners) 7-38

Curvature Deformation Equation 7-39

Combined constant and curvature deformation: Let Then, k -> 0 when the curve approaches edges. Strong edges will stop the curve propagation. The above geometric deformable model starts with an initial curve and evolves its shape using a speed function. Such an evolution can be implemented using level sets technique. 7-40

Level set function is a higher-dimensional function, which represents a boundary as a level set. The curve at time t is the set of image points, for which the value of the level set function at time t is equal to zero

Using the level set representation of a curve allows its evolution by updating the level set function. 7-42

A level set function with contour as its zero-level set Differentiate w.r.t. t

Experiment: 7-44 44

: the initial contour Level set function: : the edge magnitude 7-45 45

7.4. Fuzzy Connectivity Uncertainties for image segmentation: Noise, uneven illumination, limited spatial resolution, partial occlusion, etc. Fuzzy connectivity segmentation keeps considering the likelihood (i.e., hanging togetherness) of whether nearby pixels belong to the same object Fuzzy affinity : a strength of hanging togetherness of nearby pixels, which is a function of distance between two adjacent pixels. : fuzzy adjacency e.g., f : image properties

Fuzzy adjacency : Fuzzy connectedness : 7-47

Connectedness map : c : seed; : any pixel 7-48

7-49

7-50

Thresholding yields the segmentation result. 7-51

Relative fuzzy connectivity, Extensions: Relative fuzzy connectivity, 7-52

Multi-seeded fuzzy connectivity Scale-based fuzzy connectivity 7-53

7.6. Graph Cut-Segmentation Idea: Seeds may be interactively or automatically identified.

Steps: Given an image of size n (1) Construct a graph with n+2 nodes     (2) Each node has 4 n-links, connected to neighbors. Each link is assigned a cost derived from smoothness term , where f: image value (3) Each node has 2 t-links, connected to source S and sink T. Each link is assigned a cost derived from data term 7-55

(4) Identify seed pixels and define “hard constraint” for their links (5) Find the minimum cut that minimizes (6) Obtain the segmentation result. 7-56

7-57

Example: Given an image Let I : the set of pixels N: a set of neighborhood pixel pairs : the resulting labeling vector L is obtained by finding the cut that minimizes the cost where : regional property term : boundary property term 7-58

The costs of arcs are defined in the following table e.g., 7-59

7-60

Example: 7-61 61