1. EMANUELE RODOLÀ A Game-Theoretic Perspective on Registration and Recognition of 3D Shapes

2. Surface registration The aim is to rigidly align (“register”) two or more 3D surfaces so as to attain automatic assemblage of range data (demo) 2

3. Surface registration Typically a 2-step process:  Coarse motion estimation  Refinement 3

4. Coarse alignment Knowledge of the acquisition process Marker-based RANSAC-based DARCES PROSAC variants PCA / 4PCS / Genetic 4

5. Refinement Given a “good enough” initial alignment, it is possible to refine the registration iteratively. This is usually done by establishing pointwise correspondences among the two surfaces, and using them to estimate the rigid transformation. (demo chef) 5

6. A Game-Theoretic approach We cast the registration problem to an inlier selection scenario: We are given a set of candidate correspondences (strategies)  Then we look for a robust set of inlier correspondences wrt some notion of “rigidity” Finally we can estimate the rigid transformation between the two surfaces 6

7. Enforcing rigidity Given a model mesh M, a data mesh D, and a set of candidate correspondences (or strategies): We define a rigidity-enforcing payoff function giving a measure of compatibility among strategies: We wish to bring global information into the matching process by favoring sets of point-associations that are mutually compatible with a single rigid transformation. We do this by operating at a local level. 7

8. The evolutionary process The search for a solution is performed by simulating the evolution of a natural selection process. The choice of an actual selection process is not crucial and can be driven mostly by considerations of efficiency and simplicity. 8

9. Building the strategies set It is not practical to deal with all the surface points from both the surfaces, i.e. we restrict to Moreover, the isolation of interest points can help to avoid false correspondences 9

10. Surface descriptors Surface vertices can be described using information at the vertex and of a local patch around it. o Spin Images (ref. axis) o Integral Invariants (no ref.) o Point Signatures (ref. frame) o Signatures of Histograms (stable ref. frame) o And many more 10

11. Surface Hashes To fully exploit our inlier selection method, we need descriptors with the following properties: o high repeatability o weak distinctiveness 11

12. Interest point detection Relevance-based sampling Clustering (via a Matching Game) Simple threshold on the Integral Hash 12

13. Enhancing the framework The set of strategies can now be greatly reduced: The descriptor prior gives better candidates The least likely correspondences are pruned The selection process converges more rapidly Much lower memory requirements Memory is the bottleneck! 13

14. Object-in-clutter Focus is on recognition rather than alignment We now have a known, usually complete model to match against an incomplete and cluttered scene. (demo) 14

15. Object-in-clutter A good point selection strategy Robust descriptors wrt to occlusion and clutter 15

16. Object-in-clutter 16

17. Future directions Scale invariance could be attained by taking into account the geodesic path between strategies Non-rigid registration 17

18. Questions? 18