CSE (c) S. Tanimoto, 2008 Image Understanding II 1 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

CSE (c) S. Tanimoto, 2008 Image Understanding II 2 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

CSE (c) S. Tanimoto, 2008 Image Understanding II 3 Blocks-World Scene Analysis (Adolfo Guzman, MIT AI Lab, 1967) Input: a line drawing representing a 3D scene consisting of polyhedra Output: lists of faces grouped into objects Method: (a) classify vertices, (b) link faces according to rules, (c) extract connected components of doubly linked regions.

CSE (c) S. Tanimoto, 2008 Image Understanding II 4

5 Guzman Vertex Categories ell, arrow, fork, tee, kay

CSE (c) S. Tanimoto, 2008 Image Understanding II 6 Guzman Linking Rules ell, arrow, fork, tee, kay

CSE (c) S. Tanimoto, 2008 Image Understanding II 7 Guzman Face Grouping Criterion: Two adjacent faces must be doubly linked to be considered part of the same object. E F D G C B A Obj1: (A B C) Obj2: (D E F G)

CSE (c) S. Tanimoto, 2008 Image Understanding II 8 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

CSE (c) S. Tanimoto, 2008 Image Understanding II 9 Impossible Figures Locally consistent, globally inconsistent

CSE (c) S. Tanimoto, 2008 Image Understanding II 10 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

CSE (c) S. Tanimoto, 2008 Image Understanding II 11 Edge Detection Why? Find boundaries of objects in order to compute their features: shape, area, moments, corner locations, etc. Methods: Compute measures of change in the neighborhood of each pixel. Trace boundaries Transform each neighborhood into a vector space whose basis includes "edge" vectors.

CSE (c) S. Tanimoto, 2008 Image Understanding II 12 Horizontal differences: a – b Both horiz. & vertical diffs. Roberts' cross operator (a-d) 2 +(b-c) 2 Local Edge Detectors ab ab cd

CSE (c) S. Tanimoto, 2008 Image Understanding II 13 Express each 3x3 neighborhood as a vector. N = Frei-Chen Edge Detection e h cd fg aab

CSE (c) S. Tanimoto, 2008 Image Understanding II 14 Frei-Chen Edge Det. (cont) Transform it linearly into the 9D vector space spanned by the following basis vectors (each shown in 3x3 format)

CSE (c) S. Tanimoto, 2008 Image Understanding II 15 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

CSE (c) S. Tanimoto, 2008 Image Understanding II 16 Instead of using 1 st derivatives, let's consider 2 nd derivatives. The Laplacian effectively finds subtle changes in intensity.  2 f(x,y) =  2 f(x,y)/  x 2 +  2 f(x,y)/  y 2 discrete approximation: Edge Finding with the Laplacian

CSE (c) S. Tanimoto, 2008 Image Understanding II 17 Remedy: First, smooth the signal with a Gaussian filter. g(x,y) = 1/(2  2 ) exp(-(x 2 +y 2 )/2  2 ) Then take the Laplacian. Or, directly convolve directly with a Laplacian of Gaussian (LOG). Laplacian: Very Sensitive to Noise

CSE (c) S. Tanimoto, 2008 Image Understanding II 18 The edges tend to be where the second derivative crosses 0. Zero Crossings

CSE (c) S. Tanimoto, 2008 Image Understanding II 19 Zero Crossings: Example (a) orig. (b) LoG with  = 1, (c) zero crossings. R Fischer et al.

CSE (c) S. Tanimoto, 2008 Image Understanding II 20 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

CSE (c) S. Tanimoto, 2008 Image Understanding II 21 If two neighboring pixels share a side, they are 4-adjacent. d's 4-adjacent neighbors are: b, c, e, g If two neighboring pixels share a corner, they are 8-adjacent d's 8-adjacent neighbors are: a,b,c,e,f,g,h 4- and 8-adjacency e h cd fg aab

CSE (c) S. Tanimoto, 2008 Image Understanding II 22 A set of pixels R is said to be 4-connected, provided for any p i, p j in R, there is a chain [(p i, p i' ), (p i', p i'' ),..., (p k, p j )] involving only pixels in R, and each pair in the chain is 4- adjacent. The chain may be of length 0. R is 8-connected, iff there exists such a chain where each pair is 8-adjacent. 4- and 8-connectedness

CSE (c) S. Tanimoto, 2008 Image Understanding II and 8-connected components 4-connected components of black: 4 8-connected components of black: 2 4-connected components of white: 2 8-connected components of white: 2

CSE (c) S. Tanimoto, 2008 Image Understanding II 24 "Dual approach" to edge detection. Let P be a digital image: P = {p 0, p 1,..., p n-1 } A segmentation S = {R 0, R 1,..., R m-1 } is a partition of P, such that For each i, R i is 4-connected, and R i is "uniform", and for all i,j if R i is adjacent to R j, then (R i U R j ) is not uniform. Segmentation into Regions

CSE (c) S. Tanimoto, 2008 Image Understanding II 25 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

CSE (c) S. Tanimoto, 2008 Image Understanding II 26 Can compute a segmentation if U(R) means all pixels in R have the same pixel value (e.g., intensity, color, etc.) Algorithm: Initialize a UNION-FIND ADT instance, with each pixel in its own subset. Scan the image. At each pixel p, check its neighbor r to the right. p and r have the same pixel value, determine whether FIND(p) = FIND(r), and if not, perform UNION(FIND(p), FIND(r)). Then check the pixel q below doing the same tests and possible UNION. When the scan is complete, each subset represents one 4- connected component of the image. Image Connected Components

CSE (c) S. Tanimoto, 2008 Image Understanding II 27 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

CSE (c) S. Tanimoto, 2008 Image Understanding II 28 Gestalt Grouping Identification of regions using a uniformity predicate based on wider-neighborhood features (e.g., texture).

CSE (c) S. Tanimoto, 2008 Image Understanding II 29 Gestalt Grouping

CSE (c) S. Tanimoto, 2008 Image Understanding II 30 Gestalt Grouping Texture element = “texel” Texel directionality Texel granularity Alignments of endpoints Spacing of texels Groups cue for surfaces, objects.

CSE (c) S. Tanimoto, 2008 Image Understanding II 31 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

CSE (c) S. Tanimoto, 2008 Image Understanding II 32 "Mathematical Morphology" Developed at the Ecoles des Mines, France by J. Serra, from mathematical ideas of Minkowski. Idea is to take two sets of points in space and obtain a new set. Set A: the main set. Set B: the "structuring element" Set C: the result.

CSE (c) S. Tanimoto, 2008 Image Understanding II 33 Math. Morph. Operation 1: "Erosion" Consider each point b of B to be a vector and use it to translate a copy of A getting A b Take the intersection of all the A b The result is called the erosion of A by B.

CSE (c) S. Tanimoto, 2008 Image Understanding II 34 Erosion: example A: B: 1 1 C:

CSE (c) S. Tanimoto, 2008 Image Understanding II 35 Dilation: Take Union instead of Intersection. A: B: 1 1 C:

CSE (c) S. Tanimoto, 2008 Image Understanding II 36 Opening: Erode by B, then Dilate by –B (B rotated by  ) A: B: 1 1 C:

CSE (c) S. Tanimoto, 2008 Image Understanding II 37 Closing: Dilate by B, then Erode by –B (B rotated by  ) A: B: 1 1 C: (In this example, erosion undoes the dilation.)

CSE (c) S. Tanimoto, 2008 Image Understanding II 38 Closing – another example A: B: 1 1 C: (In this example, closing fills in holes.)

CSE (c) S. Tanimoto, 2008 Image Understanding II 39 Math. Morphol. Uses Repair regions corrupted by noise; Eliminate small regions Identify holes, bays, protrusions, isthmi, peninsulas, and specific shapes such as crosses. Printed circuit board inspection (find cracks, spurious wires.) Popular primitives for writing industrial machine vision apps.