1. Feature Detection: Corners and Lines

2. Edges vs. Corners Edges = maxima in intensity gradientEdges = maxima in intensity gradient

3. Edges vs. Corners Corners = lots of variation in direction of gradient in a small neighborhoodCorners = lots of variation in direction of gradient in a small neighborhood

4. Detecting Corners How to detect this variation?How to detect this variation? Not enough to check average andNot enough to check average and

5. Detecting Corners Claim: the following covariance matrix summarizes the statistics of the gradient Summations over local neighborhoodsClaim: the following covariance matrix summarizes the statistics of the gradient Summations over local neighborhoods

6. Detecting Corners Examine behavior of C by testing its effect in simple casesExamine behavior of C by testing its effect in simple cases Case #1: Single edge in local neighborhoodCase #1: Single edge in local neighborhood

7. Case#1: Single Edge Let ( a, b ) be gradient along edgeLet ( a, b ) be gradient along edge Compute C  ( a, b ):Compute C  ( a, b ):

8. Case #1: Single Edge However, in this simple case, the only nonzero terms are those where  f = ( a, b )However, in this simple case, the only nonzero terms are those where  f = ( a, b ) So, C  ( a, b ) is just some multiple of ( a, b )So, C  ( a, b ) is just some multiple of ( a, b )

9. Case #2: Corner Assume there is a corner, with perpendicular gradients ( a, b ) and ( c, d )Assume there is a corner, with perpendicular gradients ( a, b ) and ( c, d )

10. Case #2: Corner What is C  ( a, b )?What is C  ( a, b )? – Since ( a, b )  ( c, d ) = 0, the only nonzero terms are those where  f = ( a, b ) – So, C  ( a, b ) is again just a multiple of ( a, b ) What is C  ( c, d )?What is C  ( c, d )? – Since ( a, b )  ( c, d ) = 0, the only nonzero terms are those where  f = ( c, d ) – So, C  ( c, d ) is a multiple of ( c, d )

11. Corner Detection Matrix times vector = multiple of vectorMatrix times vector = multiple of vector Eigenvectors and eigenvalues!Eigenvectors and eigenvalues! In particular, if C has one large eigenvalue, there’s an edgeIn particular, if C has one large eigenvalue, there’s an edge If C has two large eigenvalues, have cornerIf C has two large eigenvalues, have corner Tomasi-Kanade corner detectorTomasi-Kanade corner detector

12. Corner Detection Implementation 1.Compute image gradient 2.For each m  m neighborhood, compute matrix C 3.If smaller eigenvalue 2 is smaller than threshold , record a corner 4.Nonmaximum suppression: only keep strongest corner in each m  m window

13. Corner Detection Results Checkerboard with noiseCheckerboard with noise Trucco & Verri

14. Corner Detection Results

15. Histogram of 2 (smaller eigenvalue )

16. Corner Detection Application: good features for tracking, correspondence, etc.Application: good features for tracking, correspondence, etc. – Why are corners better than edges for tracking? Other corner detectorsOther corner detectors – Look for curvature in edge detector output – Perform color segmentation on neighborhoods – Others…

17. Detecting Lines What is the difference between line detection and edge detection?What is the difference between line detection and edge detection? – Edges = local – Lines = nonlocal Line detection usually performed on the output of an edge detectorLine detection usually performed on the output of an edge detector

18. Detecting Lines Possible approaches:Possible approaches: – Brute force: enumerate all lines, check if present – Hough transform: vote for lines to which detected edges might belong – Fitting: given guess for approximate location, refine it Second method efficient for finding unknown lines, but not always accurateSecond method efficient for finding unknown lines, but not always accurate

19. Hough Transform General idea: transform from image coordinates to parameter space of featureGeneral idea: transform from image coordinates to parameter space of feature – Need parameterized model of features – For each pixel, determine all parameter values that might have given rise to that pixel; vote – At end, look for peaks in parameter space

20. Hough Transform for Lines Generic line: y = ax + bGeneric line: y = ax + b Parameters: a and bParameters: a and b

21. Hough Transform for Lines 1.Initialize table of buckets, indexed by a and b, to zero 2.For each detected edge pixel ( x, y ): a. Determine all ( a, b ) such that y = ax + b b. Increment bucket ( a, b ) 3.Buckets with many votes indicate probable lines

22. Hough Transform for Lines b a

23. b a

24. Bucket Selection How to select bucket size?How to select bucket size? – Too small: poor performance on noisy data – Too large: poor accuracy, long running times, possibility of false positives Large buckets + verification and refinementLarge buckets + verification and refinement – Problems distinguishing nearby lines Be smarter at selecting bucketsBe smarter at selecting buckets – Use gradient information to select subset of buckets – More sensitive to noise

25. Difficulties with Hough Transform for Lines Slope / intercept parameterization not idealSlope / intercept parameterization not ideal – Non-uniform sampling of directions – Can’t represent vertical lines Angle / distance parameterizationAngle / distance parameterization – Line represented as ( r,  ) where x cos  + y sin  = r r 

26. Angle / Distance Parameterization Advantage: uniform parameterization of directionsAdvantage: uniform parameterization of directions Disadvantage: space of all lines passing through a point becomes a sinusoid in ( r,  ) spaceDisadvantage: space of all lines passing through a point becomes a sinusoid in ( r,  ) space

27. Hough Transform Results Forsyth & Ponce

28. Hough Transform Results Forsyth & Ponce

29. Hough Transform What else can be detected using Hough transform?What else can be detected using Hough transform? Anything, but dimensionality is keyAnything, but dimensionality is key

30. Hough Transform for Circles Space of circles has a 3-dimensional parameter space: position (2-d) and radiusSpace of circles has a 3-dimensional parameter space: position (2-d) and radius So, each pixel gives rise to 2-d sheet of values in 3-d spaceSo, each pixel gives rise to 2-d sheet of values in 3-d space

31. Hough Transform for Circles In many cases, can simplify problem by using more informationIn many cases, can simplify problem by using more information Example: using gradient informationExample: using gradient information Still need 3-d bucket space, but each pixel only votes for 1-d subsetStill need 3-d bucket space, but each pixel only votes for 1-d subset

32. Hough Transform for Circles + +... =

33. Simplifying Hough Transforms Another trick: use prior informationAnother trick: use prior information – For example, if looking for circles of a particular size, reduce votes even further

34. Fitting Output of Hough transform often not accurate enoughOutput of Hough transform often not accurate enough Use as initial guess for fittingUse as initial guess for fitting

35. Fitting Lines Initial guess

36. Fitting Lines Least-squaresminimization

38. As before, have to be careful about parameterizationAs before, have to be careful about parameterization Simplest line fitting formulas minimize vertical (not perpendicular) point-to-line distanceSimplest line fitting formulas minimize vertical (not perpendicular) point-to-line distance Closed-form solution for point-to-line distance, not necessarily true for other curvesClosed-form solution for point-to-line distance, not necessarily true for other curves