Segmentation and Perceptual Grouping The problem Gestalt Edge extraction: grouping and completion Image segmentation

Camouflage

Kanizsa Triangle

The image of this cube contradicts the optical image

Perceptual Organization Atomism, reductionism:  Perception is a process of decomposing an image into its parts.  The whole is equal to the sum of its parts. Gestalt (Wertheimer, Köhler, Koffka 1912)  The whole is larger than the sum of its parts.

Mona Lisa

Gestalt Principles Proximity

Gestalt Principles Proximity Similarity

Gestalt Principles Proximity Similarity Continuity

Gestalt Principles Closure Proximity Similarity Continuity

Gestalt Principles Proximity Similarity Continuity Closure Common Fate

Gestalt Principles Proximity Similarity Continuity Closure Common Fate Simplicity

Smooth Completion Isotropic Smoothness Minimal curvature Extensibility

Elastica Elastica is not scale invariant

Elastica Scale invariant measure Approximation

Finding lines from points

Parametric methods: RANSAC

RANSAC RANdom SAmple Concensus Complexity:  Need to go over all pairs: O(n 2 )  For each pair check how many more points are consistent: O(n)  Total complexity: O(n 3 )

RANSAC Another application of RANSAC: Find transformation between images Example: compute homography  Compute homography for every 4 pairs of corresponding points  Choose the homography that best explains the image  m 4 n 4 sets should be tested Another example: compute epipolar lines  How many correspondences are needed?

Hough Transform

Linear in the number of points Describe lines as Or better Prepare a 2D table θ c

Hough Transform θ c +1

Hough Transform θ c What if we want to find circles?

Curve Salience

Saliency Network Encourage Length Low curvature Closure

Saliency Network

Tensor Voting Every edge element votes to all its circular edge completions Vote attenuates with distance: e -αd Vote attenuates with curvature: e -βk Determine salience at every point using principal moments

Tensor Voting

Stochastic Completion Field Random walk: In addition, a particle may die with probability:

Stochastic Completion Fields

Most probable path: with Can be implemented as a convolution

Stochastic Completion Fields

Snakes Given a curve Г(s)=(x(s),y(s)), define: with

Extremum: Calculus of Variation Given a functional A condition for a local extrimum is obtained using the Euler-Lagrange equation Curve evolution is defined Solution obtained when

Curve evolution

Level Set Methods Curve defined implicitly by

Curve Evolution

Shortest Path

Image Segmentation: Thresholding

Histogram

Thresholding

S-T Min-Cut/Max Flow

S t

Normalized Cuts Given a graph G=(V,E), define  W = {w ij } weights  D = diag{d i },  L = D - W Laplacian Let, we seek to solve

Normalized Cuts This can be show to be equivalent to with With these constrains the problem is NP-hard. Without the constraint the solution is obtained through the generalized eigenvalue problem

Normalized Cuts Dividing into two segments:  Partition determined by the eigenvector with the second smallest eigenvalue  We need to pick a threshold Dividing into more than two segments:  Pick several thresholds.  Divide each segment recursively.  Pick the best few eigenvectors and then perform k-means.

Texture Examples

Filter Bank

Textons imagetextons texton assignment

Normalized Cuts

Mean Shift Segmentation

Given an image, convert it to a function that is inversely related to edgeness Perform mean shift from every pixel Cluster pixels that lead to the same peak

Mean Shift Segmentation

Summary Local processing is often insufficient to separate objects We reviewed several approaches for  curve extraction, completion  region segmentation

Preattentive: Parallel

Attention: Serial