1. A Generic Approach to Detect Edges, Corners and Junctions Simultaneously Jiqiang Song The Chinese University of Hong Kong

2. Related work Low-level image features 1-D feature: edge 2-D features: corner, junction Edge detection: surface model Gradient-based: Sobel Derivative-based: Canny, Zero-crossing, LoG Surface fitting Corner & junction detection Edge-based Template-matching

3. Related work (cont’d) Two new models based on the statistic color distribution in a circular neighborhood: SUSAN detector – "USAN" principle. Handle edge, corner, and junction. Cannot handle textures. Compass operator – color distributions of two parts. Handle edge, corner, and "Y"-junction in different passes. Using a group of colors to handle textures. Corner detection is very time-consuming.

4. Problems Existing methods have at least one of the following four disadvantages: Requiring different passes or methods to detect edges, corners and junctions; Unable to handle textured regions; Unable to detect the boundary direction of corners and junctions; Holding an assumption on the junction shape.

5. Our solutions Our objectives: detect edges, corners, and junctions simultaneously; handle textures; detect free-shaped corners and junctions as well as their boundary directions. We emphasize: the circular neighborhood to ensure the isotropy; the statistic color distribution; the spatial color distribution.

6. Generic neighborhood model NB(P) – the circular neighborhood of any pixel P, NB(P) = { Sector i ; i=1..n} The sectors satisfy the following two conditions: The global optimization conditions: Each sector is of the most homogeneous color distribution inside; Every two adjacent sectors are of the most significant difference in color distribution.

7. Pixel type classification After getting the best-divided sectors, the type of P can be classified according to the number and relationship of the sectors. Sector numberRelationshipPixel typeExample n = 1AnyPlainFig. 2-a n = 2 |  1 -  2 |   EdgeFig. 2-b |  1 -  2 | >  CornerFig. 2-c n > 2AnyJunctionFig. 2-d (a) Plain (b) Edge (c) Corner (d) Junction Figure 2. Classification of the pixel type Table 1. Classification criteria of the pixel type based on the best-divided sectors

8. Our detection approach A straightforward way – design a proper function to represent the global optimization condition and then maximize/minimize it to get the best solution. Unknown number of variables Time-consuming Since this problem is similar to segmenting the neighborhood into several regions, the splitting- and-merging scheme is applied in our algorithm.

9. Step 1: Splitting NB(P) is split equally into n slices. n is even and is constrained by two conditions: area condition and granularity condition. After n is determined, NB(P) is split as follows: NB(P) = {Slice i ; i=1..n | (i-1)    i < i , where  i is the central angle subtended by Slice i }

10. Step 2: Color distribution feature CDF i The color distribution feature of Slice i. Chosen according to the application demand. Dist(CDF i, CDF j ) to calculate the distance between two CDFs, which returns a real number between [0,1].

11. Step 3: Integrated slice distance Adjacent Slice Distance (ASD) ASD i = Dist( CDF i, CDFNext(i) ) Global Slice Distance (GSD) GSD i = Dist( CDF i, CDFmin_inx ) Relative GSD (RGSD) Integrated Slice Distance (ISD) ISD i = MAX(ASD i, RGSD i ) Figure 4. Slice distance analyses (a) ASD profile (b) GSD/RGSD profiles (c) ISD profile 0.0 1.0 0.8 0.6 0.4 0.2 ASD 12 1 2 3 4 5 6 7 8 9 10 11 12 Slices TdTd 0.0 1.0 0.8 0.6 0.4 0.2 ISD 12 1 2 3 4 5 6 7 8 9 10 11 12 Slices TdTd 0.0 1.0 0.8 0.6 0.4 0.2 GSD / RGSD 12 1 2 3 4 5 6 7 8 9 10 11 12 Slices TdTd GSD RGSD cur_slice = min_inx; base_dist = GSD min_inx ; do { next_dist = GSD Next(cur_slice) ; if (ASD cur_slice < T d ) { relative_dist = |next_dist - base_dist|; if (relative_dist > T d ) { RGSD cur_slice = relative_dist; base_dist = next_dist; }else{ RGSD cur_slice = 0; } }else{ RGSD cur_slice = 0; base_dist = next_dist; } cur_slice = Next(cur_slice); } while (cur_slice  min_inx); GSD  RGSD 1 2 43 11 10 127 8 9 5 6 Figure 3. A split neighborhood

12. Step 4: Boundary detection If a point in the ISD profile  T d, it is called a peak. A peak  a boundary slice. The conditions for one boundary slice  one peak: Each region covers at least two slices, where "cover" means the pixels of this region occupies more than 90% area of a slice. The real boundary of two regions is near the border of the boundary slice.

13. Step 4 (cont’d) Combinations of "Yes" or "No" responses to the two conditions indicate four typical cases. Real Sector Strength (RSS) – discriminate between Case 2 and Case 3. RSS = Dist (2  CDF k, CDF Prev(k) +CDF Next(k) ). 1.0 0.5 0.0 ISD i-2 i-1 i i+1 i+2 Slices 1.0 0.5 0.0 ISD i-2 i-1 i i+1 i+2 Slices 1.0 0.5 0.0 ISD i-2 i-1 i i+1 i+2 Slices i+1i i-1 i+1i i-1 i+1i i-1 i+1i i-1 (a) Case 1 (b) Case 2 (c) Case 3 (d) Case 4 Figure 5. Four typical cases of adjacent sectors 1.0 0.5 0.0 ISD i-2 i-1 i i+1 i+2 Slices

14. Step 4 (cont’d) The final number of boundary slices is 0 – "Plain" type, 2 – "Edge" or "Corner" type more than 2 – "Junction" type If it is "Plain" type, all the following steps are skipped. Otherwise, the best-divided sectors are formed by merging the slices between every two adjacent boundary slices.

15. Step 5: False junction elimination Junction strength: Edge (or corner) strength: Strength = MAX{ D k (g 1, g 2 ), if MIN[ S k (g 1 ), S k (g 2 ) ] > T h }, k=1,2,..,.

16. Step 6: Boundary refinement A boundary Slice i  the real boundary is within Slice i The real boundary is between the bisectors of Slice i and Slice Next(i), i.e., x  [  1,  2 ]. DR changes monotonously when x moves from  1 to  2. (a) An instance. (b) The ISD i -to-x curve (c) The DR-to-x curve Figure 6. Boundary refinement Slice Next(i) Slice i x 1 1 2 2  1 (  1+  2)/2  2 0.0 1.0 0.5 ISD i x  1 (  1+  2)/2  2 0.0 1.0 0.5 DR x

17. Step 7: Classification With the best-divided sectors and their accurate boundaries, edges, corners and junctions are classified clearly according to the criteria listed in Table 1. With the accurate boundary information, the edge strength and the corner strength as follows:

18. Step 8: Localization Directional non-maximal suppression to localize edges and to eliminate false corners. Regional non-maximal suppression to localize both corners and junctions. Integrity verification. to match the branches of corners and junctions with edges. to eliminate false junctions. Figure 7. A false corner P P2 P2 P1P1 Sector 1 Sector 2 Figure 8: A false junction M1 M1 M2 M2 V1 V1 V2 V2 O

19. Implementation techniques Weighting the circular neighborhood. Rayleigh distribution with a central hole. CDF is represented by the Color Signature. {(x 1,v 1 ), (x 2,v 2 ),..., (x n,v n )}, where v i is a vector in a color space to which the weight x i is assigned. Dist(CDF i, CDF j ) is calculated by the Earth Mover's Distance (EMD). The saturating distance between two colors (v i and v j ) is calculated by dist ij =1 - exp{-E ij /14}. r 2 0 2 r

20. Experimental results Figure 9. Localization accuracy of edge, corner and junction with different . Color description (same for the following figures): red - edge; white - corner; yellow - junction. (a) Original image. (b) Strength:  =2. (c) Strength:  =4. (d) Strength:  =6. (e) Location:  =2. (f) Location:  =4. (g) Location:  =6. (h) Boundary:  =2. (i) Boundary:  =4. (j) Boundary:  =6. Our experiments focus on the localization accuracy, the boundary accuracy, the ability of detecting the junctions with many boundaries, the ability of handling textured regions, and the speed.

21. Experimental results (cont’d) (a) Original image (b) Strength (c) Boundaries (d) Original image (e) Strength (f) Boundaries Figure 10. Detection of 6-region junctions (  = 3). Figure 11. Edge, corner and junction detection on a textured image (a) Original image (b) Ours: all features. (c) Compass: edge. (d) Compass: corner. (e) Compass: junction. (f) SUSAN: edge. (g) SUSAN: corner. (h) Ours: final results.

22. Experimental results (cont’d) Figure 12. Results of our detector on a real image (a) Original snake image. © 2001 IEEE. (b) Our final features: edges in red, corners in white, and junctions in yellow. (  =3) (c) Boundary direction of corners and junctions. (Corners in white and junctions in yellow)

23. Experimental results (cont’d) Table 2 shows their processing time (in seconds) on a 512  512 real image with different . for Compass operator and our detector, the radius of circular neighborhood is 3 , for SUSAN detector, the radius is .

24. Conclusions Solved the problem of detecting edges, corners and junctions from color images simultaneously, while handling textured regions. A generic neighborhood model to classify edges, corners and junctions clearly by their color distributions. A low-level feature detection approach which is the first one with all the following four capabilities: detecting edges, corners and junctions in the same pass; handling both uniform-colored regions and textured regions; able to detect free-shaped corners and junctions; producing the accurate boundary direction for corners and junctions.