Chapter 6 Neighborhood Operators Computer Vision Chapter 6 Neighborhood Operators Presented by: 黃士展 R02922090@ntu.edu.tw 傅楸善 教授 Digital Camera and Computer Vision Laboratory Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R.O.C.

6.1 Introduction Visit each point p in the image data and do { N = a neighborhood or region of the image data around the point p result(p) = f(N) } After applying Neighborhood Operation Same dimension Might be interpreted as image Element type based on the functions RAW image DC & CV Lab. CSIE NTU

6.1 Introduction Domain type: Numeric, Symbolic. Neighbor type: 4-neighbor, 8-neighbor, etc. Recursive type: Rec, Non-Rec. DC & CV Lab. CSIE NTU

6.1 Introduction: Domain Type Numeric domain: arithmetic operations, +, -, min, max Symbolic domain: Boolean operations, AND, OR, NOT, table-look-up DC & CV Lab. CSIE NTU

6.1 Introduction: Neighbor Type Neighborhood might be asymmetric DC & CV Lab. CSIE NTU

6.1 Introduction: Neighbor Type (X ± 1, Y) and (X, Y ± 1) (X ± 1 , Y ± 1) (X or X ± 1, Y or Y ± 1) – (X, Y) DC & CV Lab. CSIE NTU

6.1 Introduction: Recursive Type Non-recursive neighborhood operators: output is function of input (Parallel) Recursive neighborhood operators: output depends partly on previous output (Sequential) DC & CV Lab. CSIE NTU

6.1 Introduction common masks for noise cleaning  Box filter = unweighted filter DC & CV Lab. CSIE NTU

6.1 Introduction common masked for noise cleaning DC & CV Lab. CSIE NTU

6.1 Introduction Different N x N box filter effects comparison Original Image 3 x 3 5 x 5 9 x 9 15 x 15 35 x 35 DC & CV Lab. CSIE NTU

6.1 Introduction 7.69 DC & CV Lab. CSIE NTU

6.2 Symbolic Neighborhood Operators indexing of neighborhoods DC & CV Lab. CSIE NTU

6.2.0 Dilation and Erosion (Revisit) DC & CV Lab. CSIE NTU

6.2.0 Dilation DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU

6.2.0 Erosion DC & CV Lab. CSIE NTU

6.2.0 Erosion DC & CV Lab. CSIE NTU

6.2.0 Erosion DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU

6.2.0 Opening & Closing DC & CV Lab. CSIE NTU

6.2.0 Opening DC & CV Lab. CSIE NTU

6.2.0 Opening DC & CV Lab. CSIE NTU

6.2.0 Closing DC & CV Lab. CSIE NTU

6.2.0 Closing DC & CV Lab. CSIE NTU

6.2.0 Application DC & CV Lab. CSIE NTU

Opening Closing

6.2.1 Region Segmentation Region Segmentation: Image into many regions Connected, Similarity Grayscale, Color, Texture, etc. Spatial Continuity DC & CV Lab. CSIE NTU

6.2.1 Region Segmentation 整張影像的每一像素都必須被分區 每一區塊必須是一個完整的連結區塊 沒有任一像素可以被分類到兩不同區塊 DC & CV Lab. CSIE NTU

6.2.1 Region Segmentation 同一區塊內的像素都具備相似的特性 不同特性的像素一定分到不同的區塊 DC & CV Lab. CSIE NTU

6.2.1 Region-Growing Operator : projection, outputs first or second argument : background DC & CV Lab. CSIE NTU

6.2.1 Region-Growing Operator 4-connected: output: DC & CV Lab. CSIE NTU

6.2.1 Region-Growing Operator 8-connected: output: DC & CV Lab. CSIE NTU

6.2.1 Region-Growing Operator x7 x2 X6 X3 x0 X1 X8 X4 x5 DC & CV Lab. CSIE NTU

6.2.1 Region-Growing Operator Step1. Initial Seed Sets -> New Region Step2. Neighboring pixels similarity. Step3. Pixels remaining -> Step1, or stop DC & CV Lab. CSIE NTU

6.2.1 Region-Growing Operator Seed sets choice is based on user criterion, thus image info is important Grayscale -> Histogram Texture -> Orientation And so on Keep examining another pixels with seed points, classify them. DC & CV Lab. CSIE NTU

6.2.1 Region-Growing Operator Some important issues : The suitable selection of seed points is important More information of the image is better The value, “minimum area threshold” The value, “similarity threshold value“ DC & CV Lab. CSIE NTU

6.2.1 Region-Growing Operator DC & CV Lab. CSIE NTU

6.2.1 Region-Growing Operator Grayscale lightning image. The grayscale value of this image is from 0 to 255 DC & CV Lab. CSIE NTU

6.2.1 Region-Growing Operator All the points with grayscale 255 are considered as seed points DC & CV Lab. CSIE NTU

Threshold: (a) >225 (b) >190 (c) >155 DC & CV Lab. CSIE NTU

6.2.1 Region-Growing Operator Advantages： Can provide the original images which have clear edges with good segmentation results. Only need a small number of seed points to grow the region we want. We can determine the seed points and the criteria we want to make. We can choose the multiple criteria at the same time. It performs well with respect to noise. DC & CV Lab. CSIE NTU

6.2.1 Region-Growing Operator The differences between Dilation and Region-Growing are ? Dilation: Initial Area, Structure elements ,Binary Image Region-Growing: Seeds, criterion ,Labeled Regions DC & CV Lab. CSIE NTU

Take a Break DC & CV Lab. CSIE NTU

6.2.2 Nearest Neighbor Sets and Influence Zones DC & CV Lab. CSIE NTU

6.2.3 Region-Shrinking Operator Can change all border pixels to background Can change connectivity Can entirely delete region if repeatedly applied DC & CV Lab. CSIE NTU

6.2.3 Region-Shrinking Operator : whether or not arguments identical : background border: has different neighbor and becomes background DC & CV Lab. CSIE NTU

6.2.3 Region-Shrinking Operator 4-connected: output: DC & CV Lab. CSIE NTU

6.2.3 Region-Shrinking Operator 8-connected: output: DC & CV Lab. CSIE NTU

6.2.3 Region-Shrinking Operator region shrinking: related to binary erosion except on labeled region DC & CV Lab. CSIE NTU

6.2.4 Mark-Interior/Border-Pixel Operator mark-interior/border-pixel operator marks all interior pixels with the label and all border pixels with the label DC & CV Lab. CSIE NTU

6.2.4 Mark-Interior/Border-Pixel Operator : whether or not arguments identical : recognizes whether or not its argument is symbol DC & CV Lab. CSIE NTU

6.2.4 Mark-Interior/Border-Pixel Operator 4-connected: output: DC & CV Lab. CSIE NTU

6.2.4 Mark-Interior/Border-Pixel Operator 8-connected: output: DC & CV Lab. CSIE NTU

6.2.5 Connectivity Number Operator connectivity number: nonrecursive and symbolic data domain connectivity number: classify the way pixel connected to neighbors six values of connectivity: five for border, one for interior border: isolated, edge, connected, branching, crossing DC & CV Lab. CSIE NTU

6.2.5 Connectivity Number Operator DC & CV Lab. CSIE NTU

6.2.5 Connectivity Number Operator DC & CV Lab. CSIE NTU

6.2.5 Connectivity Number Operator DC & CV Lab. CSIE NTU

Yokoi Connectivity Number 4- Connectivity number (= y) DC & CV Lab. CSIE NTU

Yokoi Connectivity Number   DC & CV Lab. CSIE NTU

Yokoi Connectivity Number 5: all corners are consistent, interior : same as neighbor, but corner DC & CV Lab. CSIE NTU

Yokoi Connectivity Number DC & CV Lab. CSIE NTU

Yokoi Connectivity Number DC & CV Lab. CSIE NTU

Yokoi Connectivity Number lena.64*64 DC & CV Lab. CSIE NTU

Yokoi Connectivity Number DC & CV Lab. CSIE NTU

Yokoi Connectivity Number 8-connectivity, only slightly different DC & CV Lab. CSIE NTU

Yokoi Connectivity Number DC & CV Lab. CSIE NTU

Rutovitz Connectivity Number Rutovitz connectivity number: sometimes called crossing number h(a,b,c) = { 1, if (a=b and a != c) or (a != b and a =c) 0, otherwise } a1 = h( X0, X1, X6) a2 = h( X0, X6, X2) a3 = h( X0, X2, X7) a4 = h( X0, X7, X3) a5 = h( X0, X3, X8) a6 = h( X0, X8, X4) a7 = h( X0, X4, X5) a8 = h( X0, X5, X1) DC & CV Lab. CSIE NTU

Take a Break DC & CV Lab. CSIE NTU

6.2.6 Connected Shrink Operator connected shrink: recursive operator, symbolic data domain connected shrink: deletes border pixels without disconnecting regions DC & CV Lab. CSIE NTU

6.2.6 Connected Shrink Operator Top-down, left-right scan: deletes edge pixels not right-boundary DC & CV Lab. CSIE NTU

DC & CV Lab. CSIE NTU

6.2.6 Connected Shrink Operator : determines whether corner connected DC & CV Lab. CSIE NTU

6.2.6 Connected Shrink Operator : background output symbol DC & CV Lab. CSIE NTU

6.2.6 Connected Shrink Operator DC & CV Lab. CSIE NTU

6.2.6 Connected Shrink Operator h (b, c, d, e) = { 1, if b = c and (d != b or e != b) 0, otherwise } Output symbol y = f (1 , 0 , 0 , 0 , 1) = g DC & CV Lab. CSIE NTU

6.2.6 Connected Shrink Operator g 1 DC & CV Lab. CSIE NTU

6.2.7 Skeleton / Medial Axis (Revisit) DC & CV Lab. CSIE NTU

6.2.7 Skeleton / Medial Axis Foreground regions are made of some uniform slow-burning material Light fires simultaneously at all points along the boundary of this region Watch the fire move into the interior Fire meets -> extinguished The so called `quench line‘ which is the skeleton arises.  DC & CV Lab. CSIE NTU

6.2.7 Skeleton / Medial Axis DC & CV Lab. CSIE NTU

6.2.7 Skeleton / Medial Axis   DC & CV Lab. CSIE NTU

6.2.7 Skeleton / Medial Axis   DC & CV Lab. CSIE NTU

6.2.7 Skeleton / Medial Axis Take this image for example DC & CV Lab. CSIE NTU

6.2.8 Thinning Operator DC & CV Lab. CSIE NTU

6.2.8 Thinning Operator Goal : Thin the regions in binary image into a one-pixel-width line figure. Thin != Skeleton DC & CV Lab. CSIE NTU

6.2.8 Thinning Operator The 4 constraints on thinning operator: The subtle edge-end must be reserved Connected part couldn’t be broken Connected areas must lead to continuous line Thinned graph must be similar with the medial axis DC & CV Lab. CSIE NTU

6.2.8 Thinning Operator Algorithm: It can be separated into 2 parts A and B. Also, we use 8-neighbor here. DC & CV Lab. CSIE NTU

6.2.8 Thinning Operator   DC & CV Lab. CSIE NTU

6.2.8 Thinning Operator Goal of part A: Eliminate the up-left corner, right, bottom pixels. DC & CV Lab. CSIE NTU

Fig. After applying algo part A to original input image 6.2.8 Thinning Operator Fig. After applying algo part A to original input image DC & CV Lab. CSIE NTU

6.2.8 Thinning Operator   DC & CV Lab. CSIE NTU

6.2.8 Thinning Operator Goal of part B: Eliminate the bottom-right corner, left, top pixels. DC & CV Lab. CSIE NTU

Fig. After applying algo part B to previously output image 6.2.8 Thinning Operator Fig. After applying algo part B to previously output image DC & CV Lab. CSIE NTU

Fig. Apply thinning algorithm once again 6.2.8 Thinning Operator Fig. Apply thinning algorithm once again DC & CV Lab. CSIE NTU

6.2.8 Thinning Operator During the thinning algo, some rules should be satisfied: Connected regions are thinned to 8-connected paths. Thinning results be as close to skeleton as possible. Extra spurs (short branches) caused by thinning must be minimized. DC & CV Lab. CSIE NTU

6.2.8 Thinning Operator DC & CV Lab. CSIE NTU

6.2.8 Thinning Operator DC & CV Lab. CSIE NTU

Take a Break DC & CV Lab. CSIE NTU

6.2.9 Distance Transformation Operator distance transformation: produces distance to closest border pixel DC & CV Lab. CSIE NTU

6.2.9 Distance Transformation Operator nonrecursive, iterative, start with : background : border, because distance to border is : interior pixels DC & CV Lab. CSIE NTU

6.2.9 Distance Transformation Operator iteration: label with all with neighbor DC & CV Lab. CSIE NTU

6.2.9 Distance Transformation Operator equivalent algorithm: nonrecursive iterative : if no neighbor labeled, still interior, leave it alone : self interior but neighbor labeled : already labeled, won’t change value since later route farther 4-connected: output DC & CV Lab. CSIE NTU

6.2.9 Distance Transformation Operator distance transformation: produces distance to closest background recursive, two-pass DC & CV Lab. CSIE NTU

6.2.9 Distance Transformation Operator first pass: left-right, top-bottom : if background still background, since min: min of upper and left neighbors and add 4-connected output DC & CV Lab. CSIE NTU

6.2.9 Distance Transformation Operator second pass: right-left, bottom-up 4-connected output DC & CV Lab. CSIE NTU

6.2.9 Distance Transformation Operator DC & CV Lab. CSIE NTU

6.2.9 Distance Transformation Operator after pass 1 DC & CV Lab. CSIE NTU

6.2.9 Distance Transformation Operator pass 1 DC & CV Lab. CSIE NTU

Take a Break DC & CV Lab. CSIE NTU

6.2.10 Radius of Fusion DC & CV Lab. CSIE NTU

6.2.10 Radius of Fusion DC & CV Lab. CSIE NTU

6.2.11 Number of Shortest Paths number of shortest paths for each 0-pixel: to binary-1 pixel set given binary image : can be 4-neighborhood or 8-neighborhood : nonzero, stays unchanged since shortest paths counted : if zero then sum of neighboring shortest paths DC & CV Lab. CSIE NTU

6.2.11 Number of Shortest Paths DC & CV Lab. CSIE NTU

6.2.11 Number of Shortest Paths DC & CV Lab. CSIE NTU

6.2.11 Number of Shortest Paths DC & CV Lab. CSIE NTU

Take a Break DC & CV Lab. CSIE NTU

6.3 Extremum-Related Neighborhood Operators DC & CV Lab. CSIE NTU

6.3.1 Non-Minima-Maxima Operator non-minima-maxima: nonrecursive operator, symbolic output DC & CV Lab. CSIE NTU

6.3.1 Non-Minima-Maxima Operator a pixel can be neighborhood maximum not relative maximum DC & CV Lab. CSIE NTU

6.3.1 Non-Minima-Maxima Operator DC & CV Lab. CSIE NTU

6.3.1 Non-Minima-Maxima Operator 4-connected: output pixel DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator relative extrema operators relative maximum and minimum operators relative extrema recursive operator, numeric data domain input not changed；output modified top-down, left-right scan then bottom-up, right-left scan until no change DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator output value: of highest extrema reachable by monotonic path relative extrema pixels: output same as input pixels DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator pixel designations for the normal and reverse scans DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator two primitive functions and : use new maximum value when ascending : keep original maximum when descending DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator 4-connected, output : top-down , left-right : bottom-up , right-left DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator a0 = l0 a1 = h (x0, x1, a0, l1) a2 = h (x0, x2, a1, l2) 3 3,3,3,3 ) x2 , l2 x1 , l1 a0 l0, x0 x2 , l2 x1 , l1 a0 l0, x0 DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator a0 = l0 a1 = h (x0, x1, a0, l1) a2 = h (x0, x2, a1, l2) 5 5, 5, 5, 5) x2 , l2 x1 , l1 a0 l0, x0 x2 , l2 x1 , l1 a0 l0, x0 DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator 4 DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator a0 = l0 a1 = h (x0, x1, a0, l1) a2 = h (x0, x2, a1, l2) 6 x2 1 9 1 9 x1 l1 a0 l0 x0 l2 x0 2 x2 x1 l2 a0 l0 DC & CV Lab. CSIE NTU

6.3.2 Relative Extrema Operator DC & CV Lab. CSIE NTU

6.3.3 Reachability Operator successively propagating labels that can reach by monotonic paths descending reachability operator employs DC & CV Lab. CSIE NTU

6.3.3 Reachability Operator g: pixels that are not relative extrema labeled background a: unchanged when neighbor is g or the same with neighbor b: become first propagated label if originally g c: common region more than one extremum can reach it by monotonic path c: already labeled and neighbor has different label, then two extrema reach a: propagate from extremum DC & CV Lab. CSIE NTU

6.3.3 Reachability Operator 4-connected, output a0 = l0 a1 = h (a0 , l1 , x0 , x1) a2 = h (a1 , l2 , x0 , x2) DC & CV Lab. CSIE NTU

6.3.3 Reachability Operator DC & CV Lab. CSIE NTU

6.3.3 Reachability Operator DC & CV Lab. CSIE NTU

6.3.3 Reachability Operator DC & CV Lab. CSIE NTU

Take a Break DC & CV Lab. CSIE NTU

6.4 Linear Shift-Invariant Neighborhood Operators convolution: commutative, associative, distributor over sums, homogeneous DC & CV Lab. CSIE NTU

6.4.1 Convolution and Correlation convolution of an image f with kernel w DC & CV Lab. CSIE NTU

6.4.1 Convolution and Correlation DC & CV Lab. CSIE NTU

6.4.1 Convolution and Correlation DC & CV Lab. CSIE NTU

6.4.1 Convolution and Correlation DC & CV Lab. CSIE NTU

6.4.1 Convolution and Correlation DC & CV Lab. CSIE NTU

6.4.1 Convolution and Correlation DC & CV Lab. CSIE NTU

6.4.1 Convolution and Correlation DC & CV Lab. CSIE NTU

6.4.1 Convolution and Correlation DC & CV Lab. CSIE NTU

6.4.1 Convolution and Correlation DC & CV Lab. CSIE NTU

6.4.2 Separability straightforward computation: multiplications, additions decomposition: DC & CV Lab. CSIE NTU

6.4.2 Separability DC & CV Lab. CSIE NTU

Project due Nov. 6 Write a program to generate Yokoi connectivity number You can down sample lena.bmp from 512*512 to 64*64 first. Sample pixels positions at (r, c) = (0,0), (0,8) ……, so everyone will get the same answer . DC & CV Lab. CSIE NTU

Project due Nov. 20 Write a program to generate thinned image. You can down sample lena.bmp from 512*512 to 64*64 first. Sample pixels positions at (r, c) = (0,0), (0,8) ……, so everyone will get the same answer . DC & CV Lab. CSIE NTU

Midterm Nov. 13 extent: whatever covered up to Nov. 6 (inclusive) DC & CV Lab. CSIE NTU