Nonrigid Shape Correspondence using Landmark Sliding, Insertion, and Deletion Theodor Richardson

2 Overview Statistical Shape Analysis (SSA) is growing in usage (mainly to develop models for better image segmentation) Accurate SSA methods depend upon an accurate shape correspondence. To address this problem, a novel, nonrigid, landmark- based method to correspond a set of 2D shape instances is presented. Unlike prior methods, the proposed method combines three important factors in measuring the shape- correspondence error: landmark-correspondence error, shape-representation error, and shape-representation compactness.

3Theodor Richardson Statistical Shape Analysis (SSA) Most anatomical structures possess a unique shape. This shape is often used in medical imaging for purposes of automated diagnosis. SSA can build models of such shapes for use in guiding shape extraction/image segmentation.

4Theodor Richardson Shape Correspondence SSA relies upon an accurate mapping across a set of shape instances. Constructing this mapping is the shape correspondence problem. A shape is defined as a continuous curve, also referred to as a contour. SSA utilizes a finite sampling of each curve called a landmark set.

5Theodor Richardson The (Landmark-Based) Point- Correspondence Problem The discrete form of shape correspondence is often called the point- correspondence problem. The desired outcome of this correspondence is a mapping from any point along one shape instance to an equivalent point along all other shape instances. Human vision can solve this problem for high curvature points.

6Theodor Richardson The Three Factors Affecting Correspondence Accuracy There are three main factors that determine the accuracy of shape correspondence: Landmark-correspondence error – it is necessary to measure the accuracy of the landmark mapping, Landmark-correspondence error – it is necessary to measure the accuracy of the landmark mapping, Shape-representation error – only when a set of landmarks well-represents the underlying contour does shape-correspondence equate to landmark- correspondence, Shape-representation error – only when a set of landmarks well-represents the underlying contour does shape-correspondence equate to landmark- correspondence, Shape-representation compactness – a sparse sampling of landmarks is desirable for current SSA methods, meaning the fewest number of landmarks required is desirable. Shape-representation compactness – a sparse sampling of landmarks is desirable for current SSA methods, meaning the fewest number of landmarks required is desirable.

7Theodor Richardson Fixed Landmark Methods Many prior methods construct a mapping based on a set of pre-sampled landmarks along each shape instance. These methods tend to use either local or global methods of matching one landmark to another. Global methods may not utilize local shape features to capture the underlying contour Global methods may not utilize local shape features to capture the underlying contour Local methods may catch local feature information but they tend to overlook global positioning Local methods may catch local feature information but they tend to overlook global positioning

8Theodor Richardson Nonfixed Landmark Methods The fixed landmark methods have a major drawback; there is no way to overcome a poor initialization of the landmark points. Nonfixed landmark methods allow landmarks to travel from their original position to an optimal location. The machine learning techniques for correspondence are a subset of this group, including MDL

9Theodor Richardson The Landmark Sliding Methods The work most closely related to the method of correspondence applied in the proposed method is landmark sliding. Bookstein first proposed the idea of sliding landmarks along their tangent directions to relocate them to ideal positions to minimize thin-plate spline bending energy*. *F. L. Bookstein. Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Trans. PAMI, 11(6):567–585, June *F. L. Bookstein. Landmark methods for forms without landmarks: Morphometrics of group differences in outline shape. Medical Image Analysis, 1(3):225–243, 1997.

10Theodor Richardson Landmark Correspondence Error The model chosen for representing the landmark correspondence error is the thin-plate spline bending energy proposed by Bookstein. Bending energy is invariant to affine transformations.

11Theodor Richardson Shape Representation Error Shape representation error is the measure of data loss in representing a continuous curve with a finite number of landmarks.

12Theodor Richardson Shape Representation Compactness Shape representation compactness simply requires that the landmark set be as small as possible while still upholding the criteria of the other two factors. This will increase shape representation error, so a balance must be found to prevent both supersampling and undersampling

13Theodor Richardson An Algorithmic Solution Choose one shape instance as the template Vt Initialize the landmark sets Vq, q = 1, 2, … n //Main loop Repeat while max sliding distance > 0 Repeat while alpha > epsilonH Landmark insertion Update the template Vt Loop over each shape instance Landmark sliding Update the template Vt Repeat while alpha < epsilonL Landmark deletion Update the template Vt End

14Theodor Richardson Detecting High Curvature Points High curvature points are easily detected by human vision; they generally represent mathematically critical points to defining a curve These points decrease representation error Retaining high curvature points emulates human vision shape correspondence These points also act as an edge case to the sliding algorithm used herein

15Theodor Richardson High Curvature Case The local maxima for the curvature plot are subjected to a threshold of the maximum difference in unsigned curvature. Points above this threshold are retained as critical correspondence landmarks (CCLs) CCLs are prevented from sliding and maintain equivalent points in all shape instances to preserve correspondence If a CCL is not present in all shape instances, the placeholder for the CCL is allowed to slide to conform to the shape instances that have the fixed CCL.

16Theodor Richardson Landmark Sliding Algorithm The landmark sliding algorithm addresses the landmark- correspondence accuracy. Landmarks slide along their estimated tangent directions. The offset landmarks are then projected back onto the original curve to preserve shape representation*. Allowable landmark sliding distance is determined by the curvature at the starting position for each landmark. Sliding is optimized by quadratic programming (minimizing a quadratic function) *S.Wang, T. Kubota, and T. Richardson. Shape correspondence through landmark sliding. In Proc. Conf. Computer Vision and Pattern Recog., pages 143–150, 2004.

17Theodor Richardson Topology Preservation For the landmark correspondence to represent the underlying shape correspondence, the topology of the underlying shape must be preserved. This means that landmarks should not be allowed to slide past each other or move in a way that breaks the flow of the underlying shape contour. This is accomplished by a constraint bounding the allowed sliding length.

18Theodor Richardson Landmark Insertion/Deletion Landmark insertion: When the mean alpha value is above epsilon, the representation error is too high; to counter this, a new landmark is inserted in the gap between landmarks contributing most to the representation error. Landmark deletion: When the mean alpha value is below epsilon, the representation error is too low; therefore a landmark is deleted from the span of the curve contributing the least amount of representation error. These processes are opposites and require two separate epsilon values to prevent oscillation.

19Theodor Richardson Comparison Study Our method was compared to the implementation of the Minimum Description Length (MDL) method over five data sets. Each data set was run with three initializations for each algorithm to compare the statistical results.

20Theodor Richardson Visual Comparison

21Theodor Richardson D1 – Corpus Callosum

22Theodor Richardson D1 – Corpus Callosum

23Theodor Richardson D2 - Cerebellum

24Theodor Richardson D3 - Cardiac

25Theodor Richardson D4 - Kidney

26Theodor Richardson D5 - Femur

27Theodor Richardson Conclusion This method considers three important factors in modeling the shape- correspondence error: landmark-correspondence error, representation error, and representation compactness. These three factors are explicitly handled by the landmark sliding, insertion, and deletion operations, respectively. The performance of the proposed method was evaluated on five shape-data sets that are extracted from medical images and the results were quantitatively compared with an implementation of the MDL method. Within a similar allowed representation error, the proposed method has a performance that is comparable to or better than MDL in terms of (a) average bending energy, (b) principal variances in SSA, (c) representation compactness, and (d) algorithm speed.