1. CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator

2. CSC508 Effects of Threshold Selection

3. CSC508 Effects of Threshold Selection The preceding images were edge-detected with a Sobel operator then binarized with various thresholds The overall trend is predictable but the individual results are not This threshold dependency is the curse of computer vision We need smarter algorithms

4. CSC508 Scale-Space Processing One method of dealing with the threshold dependency is through scale-space processing –An image is processed at various scales revealing different sets of features at each scale –Features common to all [or many] sets are deemed “important”

5. CSC508 Gaussian Pyramid Selection of the “appropriate” parameters (sigma and width) of a Gaussian filter is analogous to setting a binarization threshold The Gaussian Pyramid helps in such a situation

6. CSC508 Gaussian Pyramid Formation 1.Select a sigma and width 2.Filter the image 3.Subsample the image by removing every other line and sample –Resultant image is ¼ the size of the original 4.Repeat on the subsampled image

7. CSC508 Gaussian Pyramid Formation You end up with a series of images (a pyramid) each exposing different features –Coarse features at the top of the pyramid –Fine features at the bottom You can now use the coarse features to select where to concentrate your processing of the fine features

8. CSC508 Effects of Threshold Selection Another method of dealing with the threshold dependency is through more complex algorithms

9. CSC508 A Better Kernel The Sobel mask is a derivative mask (computes the weighted difference of neighborhood pixels) So is the Laplacian From calculus: the 2 nd derivative of a signal is zero when the derivative magnitude is extremal

10. CSC508 A Better Edge Detector Combining these observations we can derive the Laplacian-Gaussian filter due to Marr and Hildreth –Also known as a DoG (Difference of Gaussians) because it can be approximated by subtracting two Gaussian kernels of same width and differing sigmas

11. CSC508 Laplacian-Gaussian Filter Bypassing the math and jumping to the results of applying the rules of calculus… Similar in form to the Gaussian kernel

12. CSC508 Laplacian-Gaussian Filter In 1 dimension it looks like this…

13. Laplacian-Gaussian Filter In 2 dimensions it looks like this… CSC508

14. Laplacian-Gaussian Filter Convolution with this kernel results in edge magnitudes of both positive and negative values We’re interested in where the sign changes occur These are where the edges are (recall the calculus result)

15. Zero-Crossing Detection For each pixel, look at it’s 4 previous (in scan line order) neighbors If the sign is different in the edge detected image, there is an edge at pixel I(i, j) in the original image Why don’t we look at the other 4 neighbors? CSC508 +- --- i j -+ +++ i j

16. Width = 15, Sigma = 1.6

17. CSC508 Comparable Sobel vs. Laplacian-Gaussian

18. CSC508 Width = 15, Sigma = 2.6

19. CSC508 Comparable Sobel vs. Laplacian-Gaussian

20. CSC508 Improvement Results of the Marr-Hildreth method are better than those of the Sobel/Binarization method –Edge placement accuracy –Removal of edges due to noise But, there’s still something missing…

21. CSC508 What Else Can We Do? What are we ignoring in the following edge detection processes? 1.Gaussian filter 2.Sobel edge detection 3.Threshold based binarization of Sobel magnitude And 1.Laplacian-Gaussian filter 2.Zero crossing detection

22. CSC508 Context! We merely selected each edge based on its magnitude and zero-crossing We completely ignored its immediate surroundings Why is this bad? –Detects edges we don’t want Prone to noise Prone to clutter –Doesn’t detect edges we do want Weak magnitude edges are thresholded (thresheld?) away We need a smarter algorithm!

23. CSC508 Canny Edge Detector John Canny – MIT 1986 Developed a process (series of steps) for extracting edges from an image –Gaussian smoothing of input image –Basic derivative operator to detect edge candidates –Removal of edges that are clustered around a single location –Adaptive thresholding algorithm for final edge selection

24. CSC508 Canny Edge Detector Gaussian smooth original image –Reduce the effects of noise –In the code I posted the implementation is somewhat strange Broken up based on filter size Some odd scaling goes on –This is due to constraints of the original system on which the implementation was created

25. CSC508 Canny Edge Detector Select two thresholds –The high threshold is used to identify large magnitude (strong) edges This is applied first –The low threshold is used to identify small (weak) magnitude edges that are spatially connected to large (strong) magnitude edges –My code accepts these as percentages of the total range of edge magnitudes

26. CSC508 Canny Edge Detector Non-maximal Suppression –This process thins out clusters of edge points –Ideal step edges rarely occur in natural images

27. CSC508 Non-Maximal Suppression Life would be grand if edges looked like this (1 dimension) But, in reality they usually look like this (1 dimension) This is why our Sobel magnitude images look so awful

28. CSC508 Non-Maximal Suppression We need to employ an algorithm that eliminates all the weak edges (due to noise) and retains the strong edges (due to signal) The basic algorithm is given on page 180 of the textbook (this actually combines two steps) –The implementation in the posted code is somewhat cryptic but follows the basic philosophy which is

29. CSC508 Non-Maximal Suppression Find an intermediate or strong edge Search a neighborhood along the direction of the gradient (perpendicular to the edge) for stronger edges Eliminate those that are of lesser magnitude than the strongest found The result is the removal of [relatively] weak edges in the neighborhood of a [relatively] strong edge Notice that we have not applied any thresholds to this point in the process (other than Gaussian kernel parameters)

30. Non-Maximal Suppression Assume the “boldness” of the arrow represents the magnitude of the edge Assume the direction of the arrow represents the direction of the edge (perpendicular to the direction of the gradient) CSC508

31. Non-Maximal Suppression

32. CSC508 Contour Following We now have a set of “reasonable” candidate edges –We used some local context to eliminate some candidates (non-maximal suppression) Next we use global context to eliminate additional candidates

33. CSC508 Contour Following Find a strong edge –Apply the high threshold –This isn’t so bad as we can set it very high with the expectation that some real edges will meet this criteria

34. CSC508 Strong Edges

35. CSC508 Contour Following Compute the gradient direction of the edge –Similar to the computation done in Sobel Search for a neighboring edge in the direction of the edge (perpendicular to the gradient) that is above the weak threshold –We’re using a strong edge as an anchor point for a contour of weak and strong edges – known as Hysteresis Repeat the process until we have no more connected weak edges Find the next disconnected strong edge and repeat the entire contour following process

36. CSC508 Final Thinning An additional thinning step is performed This is similar to the non-maximal suppression step Just one last clean-up operation

37. CSC508 Final Results

38. CSC508 Canny Summary All edgesWeak edges Strong edges Contours

39. CSC508 Canny Summary Through more complex processing we arrive at a set of edges in which we have confidence –Minimal reliance on thresholds The cost is complex (slow, memory intensive) processing We must trade resources for accuracy!

40. CSC508 Things To Do Reading for Next (few) Week(s) –Chapter 9 – Texture 9.1 – 9.3 describe one approach I’ll describe a second approach –Chapter 14 – Segmentation by Clustering We’ll consider various clustering methods –Chapter 15 – Segmentation by Models We’ll analyze the Hough Transform

41. CSC508 Things To Do Additions to current programming assignment –Use your Gaussian filter code to create a Gaussian Pyramid of Lenna of 4 levels Resultant image sizes: (512x512), (256x256), (128x128), (64x64) –Use your Sobel filter code to create edge images of the Gaussian Pyramid images –Turn in your resultant image files –Comment on the results