Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval Philip Shilane and Thomas Funkhouser

Large Databases of 3D Shapes Mechanical CAD (National Design Repository) Molecular Biology (Protein Databank) Computer Graphics (Princeton Shape Benchmark)

Shape Retrieval 3D Model Model Database Best Matches

Local Matches for Retrieval 3D Model Model Database Best Matches

Local Matches for Retrieval 3D Model Model Database Best Matches Cost Function

Local Matches for Retrieval 3D Model Model Database Best Matches Cost Function Using many local descriptors is slow.

Local Matches for Retrieval 3D Model Model Database Best Matches Cost Function Using many local descriptors is slow. Many descriptors do not represent distinguishing parts.

Local Matches for Retrieval 3D Model Model Database Best Matches Cost Function Focusing on the distinctive regions improves retrieval time and accuracy.

Related Work Selecting Local Descriptors Random Mori 2001 Frome 2004

Related Work Selecting Local Descriptors Random Salient Gal 2005 Lee 2005 Frintrop 2004

Related Work Selecting Local Descriptors Random Salient Likelihood Johnson 2000 Shan 2004

Distinction = Retrieval Performance Query Descriptors The distinction of each local descriptor is based on how well it retrieves shapes of the correct class. Retrieval Results

Distinction = Retrieval Performance Query Descriptors The distinct descriptors that distinguish between classes are classification dependent. Retrieval Results

Approach DescriptorsDistinction We want a predicted distinction score for each descriptor on the model.

Approach We map descriptors into a 1D space where we learn distinction from a training set. Distinction 1D Parameterization Descriptors Distinction

Approach Descriptors Distinction Likelihood of shape descriptors is a 1D function that groups descriptors with similar distinction. Likelihood Parameterization

System Overview Likelihood Retrieval Evaluation Training Query Shape DB Local Descriptors Descriptor DB Likelihood Evaluate Distinction Local Descriptors Classification Shape Distinction Function Match Retrieval List Select Descriptors

System Overview Likelihood Retrieval Evaluation Training Query Shape DB Local Descriptors Descriptor DB Likelihood Evaluate Distinction Local Descriptors Classification Shape Distinction Function Match Retrieval List Select Descriptors

System Overview Likelihood Retrieval Evaluation Training Query Shape DB Local Descriptors Descriptor DB Likelihood Evaluate Distinction Local Descriptors Classification Shape Distinction Function Match Retrieval List Select Descriptors

System Overview Likelihood Retrieval Evaluation Training Query Shape DB Local Descriptors Descriptor DB Likelihood Evaluate Distinction Local Descriptors Classification Shape Distinction Function Match Retrieval List Select Descriptors

Multi-dimensional normal density [Johnson 2000] Likelihood of Descriptors

The likelihood function is proportional to the descriptor density.

Map from Descriptors to Likelihood Flat regions are the most common while wing tips and the cockpit area are rarer. Less Likely More Likely

System Overview Likelihood Retrieval Evaluation Training Query Shape DB Local Descriptors Descriptor DB Likelihood Evaluate Distinction Local Descriptors Classification Shape Distinction Function Match Retrieval List Select Descriptors

Measuring Distinction 0.33 Query Descriptors Evaluation Metric Evaluate the retrieval performance of every query descriptor. Retrieval Results

Measuring Distinction Query Descriptors Evaluation Metric Some descriptors are better for retrieval than others. Retrieval Results

System Overview Likelihood Retrieval Evaluation Training Query Shape DB Local Descriptors Descriptor DB Likelihood Evaluate Distinction Local Descriptors Classification Shape Distinction Function Match Retrieval List Select Descriptors

Build Distinction Function Measure likelihood and retrieval performance of each descriptor.

Build Distinction Function Measure likelihood and retrieval performance of each descriptor.

Build Distinction Function Measure likelihood and retrieval performance of each descriptor.

Build Distinction Function Retrieval performance is averaged within each likelihood bin.

Descriptor Distinction A likelihood mapping separates descriptors with different retrieval performance. Less Likely More Likely

Less Likely More Likely Descriptor Distinction The most common features are the worst for retrieval.

Predicting Distinction Distinction Function DescriptorsDistinction The likelihood mapping predicts descriptor distinction.

System Overview Likelihood Retrieval Evaluation Training Query Shape DB Local Descriptors Descriptor DB Likelihood Evaluate Distinction Local Descriptors Classification Shape Distinction Function Match Retrieval List Select Descriptors

Selecting Distinctive Descriptors The k most distinctive descriptors with a minimum distance constraint are selected. MeshDescriptorsDistinction Scores 3 Selected Descriptors

Matching with Selected Descriptors 3D Model Model Database Best Matches

Results Examples of Distinctive Descriptors Evaluation for Retrieval

Distinctive Descriptor Examples Descriptors on the head and neck represent consistent regions of the models.

Distinctive Descriptor Examples Descriptors on the front of the jet are consistent as opposed to on the wings.

Challenge The wheels are consistent features for cars.

Shape Database 100 Models in 10 Classes from the Princeton Shape Benchmark Models come from different branches of the hierarchical classification

Shape Descriptors Mass per Shell Shape Histogram (SHELLS) Ankerst 1999 Spherical Harmonics of the Gaussian Euclidean Distance Transform (SHD) Kazhdan Radius of Descriptors Considered

Local vs. Global Descriptors Using local descriptors improves retrieval relative to global descriptors.

Focus on Distinctive Descriptors Using a small number of distinct descriptors maintains retrieval performance while improving retrieval time.

Alternative Selection Techniques

Distinction improves retrieval more than other techniques.

Conclusion Method to select distinctive descriptors Distinctive descriptors can improve retrieval Mapping descriptors through likelihood and learned retrieval performance to distinction is better than other alternatives Distinction is independent of type of descriptor

Future Work Explore other definitions of likelihood including mixture models

Future Work Explore other definitions of likelihood including mixture models Consider non-likelihood parameterizations

Future Work Explore other definitions of likelihood including mixture models Consider non-likelihood parameterizations Combine descriptors while accounting for deformation [Funkhouser and Shilane, SGP]

Acknowledgements Szymon Rusinkiewicz Joshua Podolak Princeton Graphics Group Funding Sources: National Science Foundation Grant CCR and Grant 11S Air Force Research Laboratory Grant FA

The End