Digital Image Processing, 3rd ed.

Digital Image Processing, 3rd ed.
Chapter 11 Representation and Description Dept. Computer Science & Engineering Hongtao Lu 5/5/2019

Outline Representation Boundary Descriptors Regional Descriptors
Principal Components for Description Relational Descriptors

Preview Basically, representing a region involves two choices:
External characteristics (e.g., boundary) Internal characteristics (e.g., pixels) The next task is to describe the region based on the chosen representation. The features selected as descriptors should be as insensitive as possible to variations in size, translation, and rotation.

11.1 Representation Boundary (Border) Following
The points in the boundary of a region be ordered in a clockwise (or counterclockwise) direction. The output is an ordered sequence of points. Two assumptions: binary images: object: 1, background: 0; images are padded with a border of 0s to eliminate the possibility of an object merging with the image border.

Figure 11.1 Illustration of the first few steps in the boundary-following algorithm.The point to be processed next is labeled in black, the points yet to be processed are gray, and the points found by the algorithm are labeled as gray squares.

Starting point Figure 11.1 Illustration of the first few steps in the boundary-following algorithm.The point to be processed next is labeled in black, the points yet to be processed are gray, and the points found by the algorithm are labeled as gray squares.

Starting point background point Figure 11.1 Illustration of the first few steps in the boundary-following algorithm.The point to be processed next is labeled in black, the points yet to be processed are gray, and the points found by the algorithm are labeled as gray squares.

c1 b1 Starting point background point Figure 11.1 Illustration of the first few steps in the boundary-following algorithm.The point to be processed next is labeled in black, the points yet to be processed are gray, and the points found by the algorithm are labeled as gray squares.

c1 b1 b = b1, c = c1 Starting point background point Figure 11.1 Illustration of the first few steps in the boundary-following algorithm.The point to be processed next is labeled in black, the points yet to be processed are gray, and the points found by the algorithm are labeled as gray squares.

Figure 11.2 Illustration of an erroneous result when the stopping rule is such that boundary-following stops when the starting point, b0, is encountered again.

Sometimes is referred to as the Moore boundary tracking algorithm after Moore [1968]. Works equally well if a region, rather than its boundary is given. That is, the procedure extracts the outer boundary of a binary region. Also can be applied to find the boundaries of holes in a region (i.e., the inner boundaries of the original region).

11.1 Representation Chain Codes
Chain codes are used to represent a boundary by a connected sequence of straight-line segments of specified length and direction. Typically, this representation is based on 4- or 8-connectivity of the segments. The direction of each segment is coded by using a numbering scheme, as in Fig

11.1 Representation Chain Codes Figure 11.3 Direction numbers for
(a) 4-directional chain code, and (b) 8-directional chain code.

This method generally is unacceptable for two principal reasons: The resulting chain tends to be quite long any small disturbances along the boundary due to noise or imperfect segmentation cause changes in the code that may not be related to the principal shape features of the boundary. An approach frequently used to circumvent these problems is to resample the boundary by selecting a larger grid spacing, as Fig. 11.4(a) shows.

Figure 11.4 (a) Digital boundary with resampling grid superimposed. (b) Result of resampling. (c) 8-directional chain-coded boundary.

The chain code of a boundary depends on the starting point. We can normalize also for rotation by using the first difference of the chain code instead of the code itself. The first difference of the 4-direction chain code is If we treat the code as a circular sequence. Here, the result is

Figure 11.5 (a) Noisy image. (b) Image smoothed with a 9×9 averaging mask. (c) Smoothed image, thresholded using Otsu’s method. (d) Longest outer boundary of (c). (e) Subsampled boundary (the points are shown enlarged for clarity). (f) Connected points from (e).

11.1 Representation Polygonal Approximations
A digital boundary can be approximated with arbitrary accuracy by a polygon. In practice, the goal of a polygonal approximation is to capture the “essence” of the boundary shape with the fewest possible polygonal segments. one of the most powerful approximation techniques is representing a boundary by a minimum-perimeter polygon (MPP).

11.1.3 Polygonal Approximations
Foundation enclose a boundary [Fig. 11.6(a)] by a set of concatenated cells, as in Fig. 11.6(b). Think of the boundary as a rubber band. It will be constrained by the inner and outer walls of the bounding region defined by the cells. Ultimately, this shrinking produces the shape of a polygon of minimum perimeter that circumscribes the region enclosed by the cell strip, as Fig. 11.6(c) shows.

Figure 11.6 (a) An object boundary (black curve). (b) Boundary enclosed by cells (in gray). (c) Minimum-perimeter polygon obtained by allowing the boundary to shrink. The vertices of the polygon are created by the corners of the inner and outer walls of the gray region.

Figure 11.7 (a) Region (dark gray) resulting from enclosing the original boundary by cells (see Fig. 11.6). (b) Convex (white dots) and concave (black dots) vertices obtained by following the boundary of the dark gray region in the counterclockwise direction. (c) Concave vertices (black dots) displaced to their diagonal mirror locations in the outer wall of the bounding region; the convex vertices are not changed.

MPP Algorithm The MPP bounded by a simply connected cellular complex is not self-intersecting. Every convex vertex of the MPP is a W vertex, but not every W vertex of a boundary is a vertex of the MPP. Every mirrored concave vertex of the MPP is a B vertex, but not every B vertex of a boundary is a vertex of the MPP. All B vertices are on or outside the MPP, and all W vertices are on or inside the MPP. The uppermost, leftmost vertex in a sequence of vertices contained in a cellular complex is always a W vertex of the MPP.

Calculate the orientation of triplets of points a = (x1, y1), b = (x2, y2), c = (x3, y3) arrange these points as the rows of the matrix: if (a, b, c) is a counterclockwise sequence if the points are collinear if (a, b, c) is a clockwise sequence

Applying the MPP algorithm
Figure 11.8 (a) 566×566 binary image. (b) 8-connected boundary. (c)-(i), MMPs obtained using square cells of sizes 2, 3, 4, 6, 8, 16, and 32, respectively (the vertices were joined by straight lines for display). The number of boundary points in (b) is 1900.The numbers of vertices in (c) through (i) are 206, 160, 127, 92, 66, 32, and 13, respectively.

11.1.4 Other Polygonal Approximations Approaches
Merging techniques One approach is to merge points along a boundary until the least square error line fit of the points merged so far exceeds a preset threshold. Store parameters, set error to 0, and merging new points along the boundary until the error again exceeds the threshold. At the end, the intersections of adjacent line segments form the vertices of the polygon.

11.1.4 Other Polygonal Approximations Approaches
Splitting techniques One approach to boundary segment splitting is to subdivide a segment successively into two parts until a specified criterion is satisfied. For a closed boundary, the best starting points usually are the two farthest points in the boundary. Figure 11.9(c) shows the result of using the splitting procedure with a threshold equal to times the length of line ab.

Splitting techniques Figure 11.9 (a) Original boundary. (b) Boundary divided into segments based on extreme points. (c) Joining of vertices. (d) Resulting polygon.

11.1 Representation Signatures
A signature is a 1-D functional representation of a boundary and may be generated in various ways. One of the simplest is to plot the distance from the centroid to the boundary as a function of angle, as illustrated in Fig

Signatures Figure Distance-versus-angle signatures. In (a) r(θ) is constant. In (b), the signature consists of repetitions of the pattern r(θ) = A sec θ for 0 ≤ θ ≤ π/4 and r(θ) = A csc θ for π/4 ≤ θ ≤ π/2.

Signatures generated by the approach just described are invariant to translation, but they do depend on rotation and scaling. Normalization with respect to rotation can be achieved by finding a way to select the same starting point. One way is to select the starting point as the point farthest from the centroid. Another way is to select the point on the eigen axis.

One way to normalize for this is to scale all functions so that they always span the same range of values, e.g., [0, 1]. Whatever the method used, keep in mind that the basic idea is to remove dependency on size while preserving the fundamental shape of the waveforms.

Signatures of two simple objects
Figure 11.11 Two binary regions, their external boundaries, and their corresponding r(θ) signatures. The horizontal axes in (e) and (f) correspond to angles from 0° to 360°, in increments of 1°.

11.1 Representation Boundary Segments
Decomposing a boundary into segments is often useful. Decomposition reduces the boundary’s complexity and thus simplifies the description process. In this case, use of the convex hull of the region enclosed by the boundary is a powerful tool for robust decomposition of the boundary.

11.1 Representation Boundary Segments
The convex hull H of an arbitrary set S is the smallest convex set containing S. The set difference H – S is called the convex deficiency D of the set S. Figure 11.12(b) shows the result in this case. Note that, in principle, this scheme is independent of region size and orientation.

11.1.6 Boundary Segments Figure 11.12
(a) A region, and its convex deficiency (shaded). (b) Partitioned boundary.

11.1 Representation Skeletons
An important approach to representing the structural shape of a plane region is to reduce it to a graph. This reduction may be accomplished by obtaining the skeleton of the region via a thinning (also called skeletonizing) algorithm.

Skeletons The skeleton of a region may be defined via the medial axis transformation (MAT) proposed by Blum [1967] The MAT of a region has an intuitive definition based on the so-called “prairie fire concept.” Consider an image region as a prairie of uniform, dry grass, and suppose that a fire is lit along its border. All fire fronts will advance into the region at the same speed. The MAT of the region is the set of points reached by more than one fire front at the same time.

Skeletons Figure Medial axes (dashed) of three simple regions.

Skeletons Numerous algorithms have been proposed for improving computational efficiency. Typically, these are thinning algorithms that iteratively delete boundary points of a region subject to the constraints that deletion of these points: does not remove end points does not break connectivity does not cause excessive erosion of the region.

Skeletons With reference to the 8-neighborhood notation in Fig , Step 1 flags a contour point p1 for deletion if the following conditions are satisfied: (a) 2 ≤ N(p1) ≤ 6 (b) T(p1) = 1 (c) p2 · p4 · p6 = 0 (d) p4 · p6 · p8 = 0 where N(p1) is the number of nonzero neighbors of p1; that is, N(p1) = p2 + p3 + … + p8 + p9 (11.1-4)

Skeletons Figure Neighborhood arrangement used by the thinning algorithm.

Skeletons T(p1) is the number of 0–1 transitions in the ordered sequence p2, p3, …, p8, p9, p2. For example, N(p1) = 4 and T(p1) = 3 in Fig In Step 2, conditions (a) and (b) remain the same, but conditions (c) and (d) are changed to (c’) p2 · p4 · p8 = 0 (d’) p2 · p6 · p8 = 0 Figure Illustration of conditions (a) and (b) in Eq. (11.1-4). In this case N(p1) = 4 and T(p1) = 3.

Skeletons Step 1 is applied to every border pixel in the binary region under consideration. If all conditions are satisfied, the point is flagged for deletion. However, the point is not deleted until all border points have been processed. After Step 1 has been applied to all border points, those that were flagged are deleted (changed to 0). Then Step 2 is applied to the resulting data in exactly the same manner as Step 1. The basic procedure is applied iteratively until no further points are deleted.

Skeletons Figure 11.16 Human leg bone and skeleton of the region shown superimposed.

11.2 Boundary Descriptors Some Simple Descriptors
The length of a boundary is one of its simplest descriptors. The diameter of a boundary B is defined as: this line is called the major axis of the boundary. The minor axis of a boundary is defined as the line perpendicular to the major axis.

11.2 Boundary Descriptors Shape Numbers
As explained in Section , the first difference of a chain-coded boundary depends on the starting point. The shape number of such a boundary, based on the 4-directional code of Fig. 11.3(a), is defined as the first difference of smallest magnitude. The order n of a shape number is defined as the number of digits in its representation. Moreover, n is even for a closed boundary.

Shape Numbers Figure 11.17 All shapes of order 4, 6, and 8. The directions are from Fig. 11.3(a), and the dot indicates the starting point.

Figure 11.18 Steps in the generation of a shape number.

11.2.3 Fourier Descriptors Figure 11.19
A digital boundary and its representation as a complex sequence. The points (x0, y0) and (x1, y1) shown are (arbitrarily) the first two points in the sequence.

Figure 11.20 (a) Boundary of human chromosome (2868 points).
(b)–(h) Boundaries reconstructed using 1434, 286, 144, 72, 36, 18, and 8 Fourier descriptors, respectively. These numbers are approximately 50%, 10%, 5%, 2.5%, 1.25%, 0.63%, and 0.28% of 2868, respectively.

Fourier Descriptors Some basic properties of Fourier descriptors

11.2.4 Statistical Moments Figure 11.21 (a) Boundary segment.
The shape of boundary segments (and of signature waveforms) can be described quantitatively by using statistical moments, such as the mean, variance, and higher-order moments. Figure (a) Boundary segment. (b) Representation as a 1-D function.

11.3 Regional Descriptors Some Simple Descriptors
The area of a region is defined as the number of pixels in the region. The perimeter of a region is the length of its boundary. Compactness of a region, defined as (perimeter)2/area. A slightly different descriptor of compactness is the circularity ratio, defined as the ratio of the area of a region to the area of a circle having the same perimeter.

11.3 Regional Descriptors Some Simple Descriptors
Other simple measures used as region descriptors include the mean and median of the intensity levels, the minimum and maximum intensity values, and the number of pixels with values above and below the mean. Even a simple region descriptor such as normalized area can be quite useful in extracting information from images.

Figure 11.22 Infrared images of the Americas at night.

11.3 Regional Descriptors Topological Descriptors
If a topological descriptor is defined by the number of holes in the region, this property obviously will not be affected by a stretching or rotation transformation. Another topological property useful for region description is the number of connected components. The number of holes H and connected components C in a figure can be used to define the Euler number E: E = C – H

11.3 Regional Descriptors Topological Descriptors Figure 11.23
A region with two holes. Figure 11.24 A region with three connected components.

11.3 Regional Descriptors Topological Descriptors
The Euler number is also a topological property. The regions shown in Fig , for example, have Euler numbers equal to 0 and -1, respectively. Denoting the number of vertices by V, the number of edges by Q, and the number of faces by F gives the following relationship, called the Euler formula: V – Q + F = C – H = E

11.3 Regional Descriptors Topological Descriptors Figure 11.25
Regions with Euler numbers equal to 0 and -1, respectively. Figure 11.26 Region containing a polygonal network.

Figure 11.27 (a) Infrared image of the Washington, D.C. area. (b) Thresholded image. (c) The largest connected component of (b). (d) Skeleton of (c).

11.3 Regional Descriptors Texture
An important approach to region description is to quantify its texture content. This descriptor provides measures of properties such as smoothness, coarseness, and regularity. The three principal approaches used in image processing to describe the texture of a region are statistical, structural, and spectral.

Figure 11.28 The white squares mark, from left to right, smooth, coarse, and regular textures. These are optical microscope images of a superconductor, human cholesterol, and a microprocessor.

11.3.3 Texture Statistical approaches Intensity: random variable z
Histogram: p(zi), i = 0, 1, 2, …, L – 1 The n-th moment of z about the mean is: where m is the mean value of z (the average intensity): Note that μ0 = 1 and μ1 = 0.

11.3.3 Texture Statistical approaches
The second moment [the variance σ2(z) = μ2(z)] is of particular importance in texture description. It is a measure of intensity contrast that can be used to establish descriptors of relative smoothness. For example, the measure is 0 for areas of constant intensity (the variance is zero there) and approaches 1 for large values of σ2(z).

The third moment is a measure of the skewness of the histogram while the fourth moment is a measure of its relative flatness. Some useful additional texture measures based on histograms include a measure of “uniformity,” given by Average entropy measure

11.3.3 Texture Texture measures based on histograms Table 11.2
Texture measures for the subimages shown in Fig

G: gray-level (or intensity) co-occurrence matrix. Figure 11.29 How to generate a co-occurrence matrix.

Texture Statistical approaches

Using descriptors to characterize co-occurrence matrices.
Figure 11.30 Images whose pixels have (a) random, (b) periodic, and (c) mixed texture patterns. Each image is of size 263 × 800 pixels.

11.3.3 Texture Statistical approaches Figure 11.31
256 × 256 co-occurrence matrices and corresponding from left to right to the images in Fig

11.3.3 Texture Statistical approaches Table 11.4
Descriptors evaluated using the co-occurrence matrices displayed in Fig

Figure Values of the correlation descriptor as a function of offset (distance between “adjacent” pixels) corresponding to the (a) noisy, (b) sinusoidal, and (c) circuit board images in Fig

11.3.3 Texture Statistical approaches Figure 11.33
A zoomed section of the circuit board image showing periodicity of components.

11.3.3 Texture Structural approaches Figure 11.34
(a) Texture primitive. (b) Pattern generated by the rule S → aS. (c) 2-D texture pattern generated by this and other rules.

11.3.3 Texture Spectral approaches Figure 11.35
(a) and (b) Images of random and ordered objects. (c) and (d) Corres-ponding Fourier spectra. All images are of size 600 × 600 pixels.

Texture Figure 11.36 Plots of (a) S(r) and (b) S(θ) for Fig (a). (c) and (d) are plots of S(r) and S(θ) for Fig (b). All vertical axes are ×105.

Moment Invariants The 2-D moment of order (p + q) of a digital image f(x, y) of size M × N is defined as: The central moment of order (p + q) is defined as: where p, q = 0, 1, 2, …, and

Moment Invariants The normalized central moments, denoted ηpq, are defined as: where , for p + q = 2, 3, … A set of seven invariant moments can be derived from the second and third moments. This set of moments is invariant to translation, scale change, mirroring (within a minus sign) and rotation.

seven invariant moments

Figure 11. 37 (a) Original image
Figure (a) Original image. (b)-(f) Images translated, scaled by one-half, mirrored, rotated by 45° and rotated by 90°, respectively.

11.3.4 Moment Invariants Table 11.5
Moment invariants for the images in Fig

11.4 Use of Principal Components for Description
Let the x1, x2, and x3 be the values of a pixel in each of the three RGB component images. These three elements can be expressed in the form of a 3-D column vector, x, If we have registered images, the vectors will be n-dimensional: x = (x1, x2, …, xn)T.

The mean vector of the population is defined as mx = E{x} The covariance matrix of the vector population is defined as Cx = E{(x – mx)(x – mx)T} For K vector samples from a random population:

Hotelling transform y = A(x – mx) Some properties: my = E{y} = 0 Cy = ACxAT x = A-1y + mx = ATy + mx Ak: a k × n matrix formed from the k eigenvectors corresponding to the k largest eigenvalues.

Figure Multispectral images in the (a) visible blue, (b) visible green, (c) visible red, (d) near infrared, (e) middle infrared, and (f) thermal infrared bands.

Figure Formation of a vector from corresponding pixels in six images.

Table 11.6 Eigenvalues of the covariance matrices obtained from the images in Fig

Figure The six principal component images obtained from vectors computed using Eq. (11.4-6). Vectors are converted to images by applying Fig in reverse.

Figure Multispectral images reconstructed using only the two principal component images corresponding to the two principal component images with the largest eigenvalues (variance).

Figure Differences between the original and reconstructed images. All difference images were enhanced by scaling them to the full [0, 255] range to facilitate visual analysis.

Figure 11.43 (a) An object. (b) Object showing eigenvectors of its covariance matrix. (c) Transformed object, obtained using Eq. (11.4-6). (d) Object translated so that all its coordinate values are greater than 0.

Figure 11.44 A manual example. (a) Original points. (b) Eigenvectors of the covariance matrix of the points in (a). (c) Transformed points obtained using Eq. (11.4-6). (d) Points from (c), rounded and translated so that all coordinate values are integers greater than 0.

11.5 Relational Descriptors
To capture in the form of rewriting rules basic repetitive patterns in a boundary or region. Figure 11.45 (a) A simple staircase structure. (b) Coded structure.

Rewriting rules: (1) S → aA, (2) A → bS, (3) A → b where S and A are variables and the elements and are constants corresponding to the primitives just defined.

Rewriting rules: Figure 11.46 Sample derivations for the rules S → aA, A → bS, and A → b.

Figure 11.47 Coding a region boundary with directed line segments.

Figure 11.48 (a) Abstracted primitives. (b) Operations among primitives. (c) A set of specific primitives. (d) Steps in building a structure.

Figure 11.49 A simple tree with root $ and frontier xy.

Figure 11.50 (a) A simple composite region. (b) Tree representation obtained by using the relationship “inside of.”

Thank you!

Digital Image Processing, 3rd ed.

Similar presentations

Presentation on theme: "Digital Image Processing, 3rd ed."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Digital Image Processing, 3rd ed.

Similar presentations

Presentation on theme: "Digital Image Processing, 3rd ed."— Presentation transcript:

Similar presentations

About project

Feedback