1. Paretian similarity for partial comparison of non-rigid objects
(or how to compare a centaur to a horse)
Michael M. Bronstein
Department of Computer Science
Technion – Israel Institute of Technology

2. (Riemannian 2-manifolds)
Non-rigid world
3D OBJECTS
(Riemannian 2-manifolds)
2D OBJECTS
(silhouettes)

3. Rock, scissors, paper
ROCK
PAPER
SCISSORS

4. Extrinsic vs intrinsic similarity
EXTRINSIC SIMILARITY
INTRINSIC SIMILARITY
Are the shapes congruent?
Invariant only to rigid
transformations
Classical solution: ICP
Is the metric structure of the
shape similar?
Invariant to isometric
deformations

5. Intrinsic similarity
Metric structure described by the geodesic distances
Isometry – deformation that preserves the geodesic distances
Intrinsically similar = isometric
Approximate similarity: and are -isometric if

6. Gromov-Hausdorff distance
where:
M. Gromov, 1981, F. Memoli, G. Sapiro, 2005, BBK, PNAS, 2006

7. Gromov-Hausdorff distance
where:
M. Gromov, 1981, F. Memoli, G. Sapiro, 2005, BBK, PNAS, 2006

8. Gromov-Hausdorff distance (cont)
Metric of the space of non-rigid shapes (up to isometry)
If , then and are isometric
If and are -isometric, then
iff and are isometric
Efficient computation using generalized multidimensional scaling
M. Gromov, 1981, F. Memoli, G. Sapiro, 2005, BBK, PNAS, 2006

9. If it doesn’t fit, you must acquit
OJ Simpson measuring the glove that
appeared as an evidence in the court

10. Is a centaur similar to a horse or a man?
Example: Jacobs et al.

11. Horse is not similar to man
Partial similarity
Horse is similar
to centaur
Man is similar
to centaur
Horse is not similar to man
Partial similarity is an intransitive relation
Non-metric (no triangle inequality)
Weaker than full similarity (shapes may be partially but not fully similar)

12. Recognition by parts Divide the shapes into meaningful parts and
Compare each part separately using full similarity criterion
Merge the partial similarities,
Pentland, et al., Basri et al.

13. Problems Problem 1: how to divide the shapes into meaningful parts?
Problem 2: are all parts equally important?
Solution 1: find the most similar pair out of the sets and of all
possible parts of shapes and :
Solution 2: define partiality measuring how significant the
selected parts are w.r.t. entire shapes (larger parts = smaller partiality)

14. Multicriterion optimization
Minimize the vector objective function over
Competing criteria – impossible to minimize and simultaneously
ATTAINABLE CRITERIA
DISSIMILARITY
UTOPIA
PARTIALITY
BBBK, IJCV, submitted

15. Minimum of scalar function
Pareto optimality
Minimum of scalar function
Pareto optimum
Pareto optimum: a point at which no criterion can be improved without compromising the other
V. Pareto, 1901

16. Pareto distance
Pareto distance: set of all Pareto optima (Pareto frontier), acting as a
set-valued criterion of partial dissimilarity
Only partial order relation exists between set-valued distances: not
always possible to compare
Infinite possibilities to convert Pareto distance into a scalar-valued one
One possibility: select a point on the
Pareto frontier closest to the utopia
point,
BBBK, IJCV, submitted

17. Fuzzy approximation
Optimization over subsets is an NP-hard combinatorial problem
Relaxed problem (fuzzy approximation): optimize over membership
functions
Crisp part
Fuzzy part
BBBK, IJCV, submitted

18. Example I – mythological creatures
Large Gromov-Hausdorff distance
Small partial dissimilarity
Large Gromov-Hausdorff distance
Large partial dissimilarity
BBBK, IJCV, submitted

19. Example I – mythological creatures (cont.)
BBBK, IJCV, submitted

20. Gromov-Hausdorff distance
Example I – mythological creatures (cont.)
Gromov-Hausdorff distance
Partial similarity
BBBK, IJCV, submitted

22. Pareto frontiers, representing partial dissimilarities between objects
Example II – 3D partially missing objects
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Pareto frontiers, representing partial dissimilarities between objects
BBBK, SSVM

23. Partial dissimilarities between objects
Example II – 3D partially missing objects
Partial dissimilarities between objects
BBBK, SSVM

24. Conclusions
Intrinsic similarity of non-rigid shapes based on the Gromov-Hausdorff
distance
Generic definition of partial similarity and set-valued Pareto distance
Other applications beyond shape recognition (e.g. text sequences)

25. Grazie