06 - Boundary Models Overview Edge Tracking Active Contours Conclusion

Overview Suppose we wish to find the boundary of an object in an image One approach is to find edge segments and link them together

Overview The process of linking edge segments to obtain boundary models is deceptively difficult –How do we select edge segments for the object? –How do connect these edge segments? –What do we do with missing or conflicting data? –What domain knowledge can be used? –How much user input is necessary? –How do measure the quality of our solution?

Overview In this section we will consider the design and implementation of two techniques Edge tracking – performs a greedy search to find a path around the object boundary Active contours – solves for object boundary by minimizing an energy functional that describes constraints on the object boundary

Edge Tracking Edge tracking is a conceptually simple approach for finding object boundaries Edge tracking algorithm: –Start at a point (x,y) on the object boundary –Walk clockwise around the boundary using gradient information to choose next point –Stop when we return to the starting point

Edge Tracking User selects (x,y) starting point

Edge Tracking Gradient information used to find next (x,y) point on boundary

Edge Tracking Process repeats to get sequence of points on boundary

Edge Tracking Sometimes we have to make hard choices to pick boundary to track

Edge Tracking Process continues until we reach starting point (x,y) again

Edge Tracking Or until the edge linking reaches a dead end or the edge of the image

Edge Tracking If we are at pixel (x,y) on the boundary how do we select next point (x’,y’) on boundary? There are a number of factors to consider: –Find point with highest gradient magnitude –Find point with most similar gradient magnitude –Find point with most similar gradient direction –Find point that gives smoothest boundary

Edge Tracking To find point with highest gradient magnitude –Calculate gradient magnitude of image –Search NxN neighborhood around point (x,y) –Ignore points that are already on object boundary –Find point (x’,y’) that has the highest gradient magnitude value G Very simple to implement, but might get stuck in local gradient maxima

Edge Tracking To find point with most similar gradient magnitude to current boundary point –Search NxN neighborhood around point (x,y) –Find unused point (x’,y’) with smallest difference in gradient magnitude M

Edge Tracking To find point with most similar gradient direction to current boundary point –Search NxN neighborhood around point (x,y) –Find unused point (x’,y’) with smallest difference in gradient direction D

Edge Tracking To find point that gives smoothest boundary –Search NxN neighborhood around point (x,y) –Calculate change in direction angle relative to current point C and previous point P on boundary –Select point (x’,y’) with smallest angle A PC C45 P13590

Edge Tracking We can combine edge tracking criteria by minimizing a weighted average of scores We stop edge tracking when we are close enough to the starting point (when current point is within radius R of starting point)

Edge Tracking Edge tracking works well when the edge strengths are strong in an image and the user provides a good starting point on the edge Edge tracking works poorly if there are weak or missing edges since there are no global constraints on the shape of object boundary

Active Contours Edge tracking approach may produce very rough object boundaries –No boundary smoothness constraints Active contours solves this problem –Define object boundary as a parametric curve –Define smoothness constraints on the curve –Define data matching constraints on the curve –Deform curve to find optimal object boundary

Active Contours Example below shows active contour in action fitting itself to desired object boundary

Active Contours Interactive active contours in operation (from Snakes: Active contour models by Kass 1988)

Active Contours Interactive active contours in operation (from Snakes: Active contour models by Kass 1988)

Active Contours Represent the curve with a set of n points

Active Contours Smoothness is controlled by an internal energy constraint on the parametric curve If we minimize the elastic energy the curve will shrink like a rubber band

Active Contours The external energy constraint describes how well the curve matches the image data If we maximize the gradient magnitudes g(x,y)along the curve we will fit the strong edges that correspond to the object boundary

Active Contours We combine internal and external constraints to obtain a total energy functional to minimize Notice that we changed the sign of external constraint to make this a minimization problem

Active Contours Our next task is to solve for the parametric curve (x i, y i ) that minimizes E total Consider E internal for three consecutive points on the curve (x i-1, y i-1 ), (x i, y i ), (x i+1, y i+1 )

Active Contours If we take the derivative of E internal wrt x i and set it to zero we can solve for x i

Active Contours Similarly, we take the derivative of E internal wrt y i and set it to zero to solve for y i

Active Contours Hence we can deform the active contour to minimize the internal energy by iteratively averaging adjacent boundary coordinates (x i-1,y i-1 ) (x i,y i ) (x i+1,y i+1 )

Active Contours To minimize E total we must consider the partial derivatives of the E external term wrt x i and y i This moves the contour in the direction of the gradient of the gradient magnitude g(x,y)

Active Contours We can deform the active contour to minimize the external energy by making a step of size s in the direction of the gradient of g(x i,y i ) (x i-1,y i-1 ) (x i,y i ) (x i+1,y i+1 )

Active Contours Example – circle deforming to concave object

Active Contours Example – effects of varying  weight in total energy functional

Active Contours Example – fitting one active contour around two adjacent objects

Active Contours Additional constraints can be added to active contours to introduce stiffness (minimize second derivatives) and other external forces (user pulling curve at selected points) Calculus of variations is typically used to solve the Euler-Legrange equations associated with these active contour energy functionals

Conclusions This section described how boundary tracking and active contours can be used to obtain the boundary of objects of interest in an image Both methods require user interaction to select a starting point/curve –Boundary tracking fast and easy to implement –Active contours will result in smoother boundary