Probabilistic Fingerprints for Shapes Niloy J. MitraLeonidas Guibas Joachim GiesenMark Pauly Stanford University MPII SaarbrückenETH Zurich

Introduction Shape Analysis and Comparison shape retrieval, shape clustering, feature selection, correspondence, compression, re-use, etc Question: Are two shapes similar? ≈?

Introduction More general: Are two shapes similar in parts? relative size of overlap region partially matching under rigid motion scan alignment context-based editing shape recognition, etc. Efficient tests require compact signatures database query network setting fast pre-filtering, etc.

Background Methods for global registration Gelfand, Mitra, Guibas and Pottmann, Robust Global Registration, SGP 2005 Li and Guskov, Multi-scale Features for Approximate Alignment of Point- based Surfaces, SGP 2005 Huber and Hebert, Fully Automatic Registration of Multiple 3D Data Sets, CVBVS 2001 Global shape descriptors Kazhdan, Funkhouser and Rusinkiewicz, Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors, SGP 2003 Osada, Funkhouser, Chazelle and Dobkin, Shape Distributions, ACM TOG 2002 Reuter and Wolter, Laplace-Spectra as fingerprints for shape matching, SPM 2005

Background Geometric Hashing Wolfson, Rigoutsos. Geometric Hashing: An Overview, IEEE Computational Science and Engineering, 4(4), 1997 Gal and Cohen-Or, Salient geometric features for partial shape matching and similarity, ACM TOG 2006 File matching Broder, Glassman, Manasse, Zweig. Syntactic Clustering of the Web, World Wide Web Conference, 1997 Broder, On the Resemblance and Containment of Documents, Sequences 1997 Schleimer, Wilkerson and Alex Aiken, Winnowing: local algorithms for document fingerprinting, Sigmod, ’03

Probabilistic Fingerprints Function such that Given two shapes S 1 and S 2, with high probability if f(S 1 ) ≠ f(S 2 ) then S 1 and S 2 are dissimilar if f(S 1 ) = f(S 2 ) then S 1 and S 2 are similar f is efficiently computable compact, i.e., output sensitive localized (partial matching) robust to sampling and articulated motion

Pre-Processing Input Sample Shingles Signatures Descriptors Fingerprint

Pre-Processing Input Sample Uniform random sample guarantee δ -coverage avoid arbitrarily dense sampling [Turk 92] such that

Sample Pre-Processing Shingles Local surface patches intersection with ρ -balls create sufficient overlap for robust signature estimation, i.e.,

Pre-Processing ShinglesSignatures Local signatures should be invariant to rigid transforms sampling & local perturbations Examples: Spin images, shape histograms, integral descriptors, etc.

Pre-Processing Descriptors Signatures Optional: Compressed descriptors e.g., Rabin’s hashing Signature set multi-set of points in high-dimensional space spatial relation of shingles not preserved

Resemblance

Resemblance

Probabilistic Fingerprint Let be random permutations Estimate of resemblance indicator function

Probabilistic Fingerprint Estimate of resemblance Example: m =

Probabilistic Fingerprint Estimate of resemblance

Probabilistic Fingerprint Estimate of resemblance

Probabilistic Fingerprint Estimate of resemblance

Probabilistic Fingerprint Estimate of resemblance

Probabilistic Fingerprint Estimate of resemblance

Probabilistic Fingerprint Estimate of resemblance

Pre-Processing FingerprintDescriptors Probabilistic Fingerprint reduce using min-hashing based on random permutations of universe set of ‘random experts’ consistent for all models

Min-Hashing Feature selection by random experts reduces set comparison to element-wise comparison estimate resemblance using m permutations = perform m coin tosses to estimate bias of coin Analysis probabilistic bounds using Markov inequality & strong Chernoff bound relates size of the fingerprint to confidence in estimated resemblance

Data Reduction Shingles Signatures Descriptors Fingerprint quantizationmin hashing set size remains constant 100k 1k set reduction

Applications Resemblance between partial scans

Applications Adaptive feature selection

Applications Alignment using adaptive feature selection scan Ascan Bfinal alignment

Applications Multiple scans greedy alignment using priority queue fingerprint matching determines score advanced alignment method for verification merging fingerprints requires no re- computation

Applications Articulated motion effect of shingle size

Applications Shape distributions

Applications Database retrieval

Statistics Pre-processing time in seconds: Query time: ~ 15 msec on average Fingerprint size ~10kb model#vts. uniform sampl. spin image Rabin hash min-hash skull54k Caesar65k bunny121k horse8k

Remarks & Insights Resemblance defined as set operation on signature sets → quantization is crucial Random experts effectively extract consistent set of features → requires no explicit correspondence Fingerprints do not preserve spatial relation of shingles → false positives are possible Few parameters that are easy to tune

Remarks & Insights Accumulate local evidence for global inference Spatial structure vs. unordered signature set? Semantic features vs. random experts? Thank You!