From Last Time… Normalization If using a filter for smoothing want to preserve average brightnessIf using a filter for smoothing want to preserve average brightness Going continuous to discrete introduces approximation error and finite-window error, so need to re-normalizeGoing continuous to discrete introduces approximation error and finite-window error, so need to re-normalize

From Last Time… Normalization If finding derivative, want to preserve average slope, i.e. y=x should retain slope 1If finding derivative, want to preserve average slope, i.e. y=x should retain slope 1 Same thing in discrete caseSame thing in discrete case

From Last Time… Normalization Failure to normalize correctly:Failure to normalize correctly: – If smoothing, image will get dimmer or brighter – If finding edges, edge strength will be higher or lower

More Feature Detection: Corners and Hough Transforms

Edges vs. Corners Edges = maxima in intensity gradientEdges = maxima in intensity gradient

Edges vs. Corners Corners = lots of variation in direction of gradient in a small neighborhoodCorners = lots of variation in direction of gradient in a small neighborhood

Detecting Corners How to detect this variation?How to detect this variation? Not enough to check average andNot enough to check average and

Detecting Corners Claim: the following “covariance matrix” summarizes the statistics of the gradient The summations are over the local neighborhood, andClaim: the following “covariance matrix” summarizes the statistics of the gradient The summations are over the local neighborhood, and

Detecting Corners Examine behavior of C by testing its effect in simple casesExamine behavior of C by testing its effect in simple cases Case #1: Single edge in local neighborhoodCase #1: Single edge in local neighborhood

Case#1: Single Edge Let (a,b) be gradient along edgeLet (a,b) be gradient along edge Compute C  (a,b):Compute C  (a,b):

Case #1: Single Edge However, in this simple case, the only nonzero terms are those where  f = (a,b)However, in this simple case, the only nonzero terms are those where  f = (a,b) So, C  (a,b) is just some multiple of (a,b)So, C  (a,b) is just some multiple of (a,b)

Case #2: Corner Assume there is a corner, with perpendicular gradients (a,b) and (c,d)Assume there is a corner, with perpendicular gradients (a,b) and (c,d)

Case #2: Corner What is C  (a,b)?What is C  (a,b)? – Since (a,b)  (c,d) = 0, the only nonzero terms are those where  f = (a,b) – So, C  (a,b) is again just a multiple of (a,b) What is C  (c,d)?What is C  (c,d)? – Since (a,b)  (c,d) = 0, the only nonzero terms are those where  f = (c,d) – So, C  (c,d) is a multiple of (c,d)

Corner Detection Matrix times vector = multiple of vectorMatrix times vector = multiple of vector Eigenvectors and eigenvalues!Eigenvectors and eigenvalues! In particular, if C has one large eigenvalue, have an edgeIn particular, if C has one large eigenvalue, have an edge If C has two large eigenvalues, have a cornerIf C has two large eigenvalues, have a corner

Corner Detection Implementation 1.Compute image gradient 2.For each m  m neighborhood, compute matrix C 3.If smaller eigenvalue 2 is greater than threshold , record a corner 4.Nonmaximum suppression: only keep strongest corner in each m  m window

Corner Detection Results Checkerboard with noiseCheckerboard with noise Trucco & Verri

Corner Detection Results

Histogram of 2 (smaller eigenvalue )

Corner Detection Application: good features for tracking, correspondence, etc.Application: good features for tracking, correspondence, etc. – Why are corners better than edges for tracking? Other corner detectorsOther corner detectors – Look for curvature in edge detector output – Perform color segmentation on neighborhoods – Others…

Detecting Lines What is the difference between line detection and edge detection?What is the difference between line detection and edge detection? – Edges = local – Lines = nonlocal Line detection usually performed on the output of an edge detectorLine detection usually performed on the output of an edge detector

Detecting Lines Several different approaches:Several different approaches: – For each possible line, check whether the line is present: “brute force” – Given detected edges, record lines to which they might belong: “Hough transform + voting” – Given guess for approximate location of a line, refine that guess: “fitting” Second method efficient for finding unknown lines, but not always accurateSecond method efficient for finding unknown lines, but not always accurate

Hough Transform General idea: transform from image coordinates to parameter space of featureGeneral idea: transform from image coordinates to parameter space of feature – Need parameterized model of features – For each pixel, determine all parameter values that might have given rise to that pixel; vote – At end, look for peaks in parameter space

Hough Transform for Lines Generic line: y = ax+bGeneric line: y = ax+b Parameters: a and bParameters: a and b

Hough Transform for Lines 1.Initialize table of buckets, indexed by a and b, to zero 2.For each detected edge pixel (x,y): a. Determine all (a,b) such that y = ax+b b. Increment bucket (a,b) 3.Buckets with many votes indicate probable lines

Hough Transform for Lines a b

Difficulties with Hough Transform How to select bucket size?How to select bucket size? – Too small: poor performance on noisy data, long running times – Too large: poor accuracy, possibility of false positives Large buckets + verification and refinementLarge buckets + verification and refinement – Best of both worlds? – Problems distinguishing nearby lines

Difficulties with Hough Transform for Lines Slope / intercept parameterization not idealSlope / intercept parameterization not ideal – Non-uniform sampling of directions – Can’t represent vertical lines Angle / distance parameterizationAngle / distance parameterization – Line represented as (r,  ) where x cos  + y sin  = r r 

Angle / Distance Parameterization Advantage: uniform parameterization of directionsAdvantage: uniform parameterization of directions Disadvantage: space of all lines passing through a point becomes a sinusoid in (r,  ) spaceDisadvantage: space of all lines passing through a point becomes a sinusoid in (r,  ) space

Hough Transform Results Forsyth & Ponce

Hough Transform Results Forsyth & Ponce

Hough Transform What else can be detected using Hough transform?What else can be detected using Hough transform? Anything, but dimensionality is keyAnything, but dimensionality is key

Hough Transform for Circles Space of circles has a 3-dimensional parameter space: position (2-d) and radiusSpace of circles has a 3-dimensional parameter space: position (2-d) and radius So, each pixel gives rise to 2-d sheet of values in 3-d spaceSo, each pixel gives rise to 2-d sheet of values in 3-d space

Hough Transform for Circles In many cases, can simplify problem by using more informationIn many cases, can simplify problem by using more information Example: using gradient informationExample: using gradient information Still need 3-d bucket space, but each pixel only votes for 1-d subsetStill need 3-d bucket space, but each pixel only votes for 1-d subset

Hough Transform for Circles =

Simplifying Hough Transforms Another trick: use prior informationAnother trick: use prior information – For example, if looking for circles of a particular size, reduce votes even further

Fitting Output of Hough transform often not accurate enoughOutput of Hough transform often not accurate enough Use as initial guess for fittingUse as initial guess for fitting

Fitting Lines Initial guess

Fitting Lines Least-squaresminimization

As before, have to be careful about parameterizationAs before, have to be careful about parameterization Most popular line fitting algorithms minimize vertical (not perpendicular) point-to-line distanceMost popular line fitting algorithms minimize vertical (not perpendicular) point-to-line distance Algorithm in textbook, section 16.2Algorithm in textbook, section 16.2