Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence Yusuf Sahillioğlu and Yücel Yemez Computer Eng. Dept., Koç University, Istanbul, Turkey

Problem Definition & Apps 2 / 24 Shape interpolation Shape registration Shape matching Time-varying recon. Statistical shape analysis Goal: Find a mapping between two isometric shapes Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11. Attribute transfer

Contributions Avoid embedding C2F joint sampling of evenly-spaced salient vertices geodesic curvature integral Euclidean embedding Non-Euclidean embedding 3 / 24 O(NlogN) time complexity for dense correspondence Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Isometry Our method is purely isometric Intrinsic global property Similar shapes have similar metric structures Metric: geodesic distance 4 / 24 Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Isometric Distortion Given, measure its isometric distortion: 5 / 24 in the most general setting. : normalized geodesic distance b/w two vertices. Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Isometric Distortion 6 / 24 g g gg g g g g average for. in action: Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Minimizing Isometric Distortion N = |S| = |T| for perfectly isometric shapes. N! different mappings; intractable. 7 / 24 Solution: Patch-by-patch matching to reduce search space. Optimal mapping maps nearby vertices in source to nearby vertices in target. Recursively subdivide matched patches into smaller patches (C2F sampling) to be matched (combinatorial search). Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Coarse-to-Fine Sampling : set of base vertices sampled from at level. Sampling radii s.t. for k=0,1,..,K. at level defines patch : all vertices within a distance from the base. 8 / 24 greens inherited from level k−1 blues are all vertices ( ) patches being defined ( ) blacks + greens = Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Correspondence Algorithm Correspondence at level k is obtained in two steps: Match level k bases inside the patch pairs matched at level k−1. Merge patch-based local correspondences into one global correspondence over whole surface. 9 / 24 Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Patch-based Matching ( ) Ensure base vertices fall into each patch to allow combinatorial matching. Patch radius to select for such an : 10 / 24, area of the largest patch at level k−1. M=5 samples with circular patches to cover blue area (enlarge a bit to cover whites) Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Patch-based Matching ( ) 11 / 24 Combinatorial matching greens inherited from level k−1 blacks + greens = Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Correspondence Merging ( ) Merge patch-to-patch correspondences into one global correspondence that covers the whole surface. 12 / 24 Multi-graph  single graph. Also, d iso values made available. 1 st pass over source samples to keep only one match per sample, the one with the min d iso. 2 nd pass over target samples to assign one match per isolated sample, the one with the min d iso. Trim matches with d iso > 2D iso, i.e., outliers. Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Insight to the Algorithm 13 / 24 Conditions for the algorithm to work correctly High-resolution sampling on two perfectly isometric surfaces Evenly-spaced sampling s.t. every vertex is in at least one patch Distortion is a slowly changing convex function around optimum One optimal solution (no symmetric flips) Optimal mapping assigns s i to t j which is as nearest to the ground-truth t i as possible Inclusion assertion is then expected to apply: Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Inclusion assertion (demonstration) 14 / 24 Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Computational Complexity 15 / 24 Saliency sorting C2F sampling Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Computational Complexity 16 / 24 Patch-based combinatorial matching Merging Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Computational Complexity 17 / 24 Overall Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Experimental Results 18 / 24 Details captured, smooth flow Many-to-one Two meshes at different resolutions red line: the worst match w.r.t. isometric distortion Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11. 6K vs. 16K

Experimental Results 19 / 24 red line: the worst match w.r.t. isometric distortion Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Experimental Results 20 / 2 for four more pairs: red line: the worst match w.r.t. isometric distortion green line: the worst match w.r.t. ground-truth distortion Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Experimental Results 21 / 24 Comparisons GMDS O(N 2 logN) [Bronstein et al.] Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11. Spectral O(N 2 logN) [Jain et al.] Nonrigid world dataset Our method O(NlogN)

Future Work 22 / 24 Symmetric flip issue Purely isometry-based methods naturally fail at symmetric inputs Not intrinsically symmetric  only one optimal solution Our method may still occasionally fail to find the optimum due to initial coarse sampling Solution suggested A solution for symmetric flips due to initial coarse sampling: Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Conclusion 23 / 24 Computationally efficient C2F dense isometric shape correspondence algorithm ( O(NlogN) ). Isometric distortion minimized in the original 3D Euclidean space wherein isometry is defined. Accurate for isometric and nearly isometric pairs. Different levels of detail thanks to the C2F joint sampling. No restriction on topology. Symmetric flips may occasionally occur due to initial coarse sampling (but can be healed as proposed). Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

People Assoc. Prof. Yücel Yemez, supervisor Yusuf, PhD student 24 / 24