1 Numerical geometry of non-rigid shapes Partial similarity Partial Similarity Alexander Bronstein, Michael Bronstein © 2008 All rights reserved. Web: tosca.cs.technion.ac.il The whole is more than the sum of its parts. Aristotle, Metaphysica

2 Numerical geometry of non-rigid shapes Partial similarity Greek mythology, bis

3 Numerical geometry of non-rigid shapes Partial similarity I am a centaur.Am I human?Am I equine? Yes, I’m partially human. Yes, I’m partially equine. Partial similarity Partial similarity is a non-metric relation

4 Numerical geometry of non-rigid shapes Partial similarity Human vision example ?

5 Numerical geometry of non-rigid shapes Partial similarity Visual agnosia Oliver Sacks The man who mistook his wife for a hat

6 Numerical geometry of non-rigid shapes Partial similarity Recognition by parts Divide the shapes into parts Compare each part separately using a full similarity criterion Merge the part similarities

7 Numerical geometry of non-rigid shapes Partial similarity Partial similarity X and Y are partially similar = X and Y have significantsimilar parts Illustration: Herluf Bidstrup

8 Numerical geometry of non-rigid shapes Partial similarity Significance Problem: how to select significant parts? Significance of a part is a semantic definition Different shapes may have different definition of significance

9 Numerical geometry of non-rigid shapes Partial similarity Significance INSIGNIFICANCE Straightforward definition: significant = large Measure insignificance as the area remaining after cropping the parts

10 Numerical geometry of non-rigid shapes Partial similarity Multicriterion optimization Simultaneously minimize dissimilarity and insignificance over all the possible pairs of parts This type of problems is called multicriterion optimization Vector objective function How to solve a multicriterion optimization problem?

11 Numerical geometry of non-rigid shapes Partial similarity Pareto optimality Traditional optimizationMulticriterion optimization A solution is said to be a global optimum of an optimization problem if there is no other such that A solution is said to be a Pareto optimum of a multicriterion optimization problem if there is no other such that Optimum is a solution such that there is no other better solution In multicriterion case, better = all the criteria are better

12 Numerical geometry of non-rigid shapes Partial similarity Pareto frontier Vilfredo Pareto ( ) INSIGNIFICANCE DISSIMILARITY UTOPIA Attainable set Pareto frontier

13 Numerical geometry of non-rigid shapes Partial similarity Set-valued partial similarity The entire Pareto frontier is a set-valued distance The dissimilarity at coincides with full similarity

14 Numerical geometry of non-rigid shapes Partial similarity Scalar- vs. set-valued similarity INSIGNIFICANCE DISSIMILARITY Use intrinsic similarity as

15 Numerical geometry of non-rigid shapes Partial similarity Is a crocodile green or ? What is better: (1,1) or (0.5,0.5)? What is better: (1,0.5) or (0.5,1)? Order relations

16 Numerical geometry of non-rigid shapes Partial similarity Order relations There is no total order relation between vectors As a result, two Pareto frontiers can be non-commeasurable INSIGNIFICANCE DISSIMILARITY BLUE < RED GREEN ? RED

17 Numerical geometry of non-rigid shapes Partial similarity Scalar-valued partial similarity In order to compare set-valued partial similarities, they should be scalarized Fix a value of insignificance INSIGNIFICANCE DISSIMILARITY Can be written as using Lagrange multiplier Define scalar-valued partial similarity

18 Numerical geometry of non-rigid shapes Partial similarity Scalar-valued partial similarity INSIGNIFICANCE DISSIMILARITY Fix a value of dissimilarity Distance from utopia point Area under the curve

19 Numerical geometry of non-rigid shapes Partial similarity Characteristic functions Problem: optimization over all possible parts A part is a subset of the shape Can be represented using a characteristic function Still a problem: optimization over binary-valued variables (combinatorial optimization)

20 Numerical geometry of non-rigid shapes Partial similarity Fuzzy sets Relax the values of the characteristic function to the continuous range The value of is the membership of the point in the subset is called a fuzzy set

21 Numerical geometry of non-rigid shapes Partial similarity Fuzzy partial similarity Compute partial similarity using fuzzy parts Fuzzy insignificance Fuzzy dissimilarity where is the fuzzy stress

22 Numerical geometry of non-rigid shapes Partial similarity Partial similarity computation Decouple parts and correspondence computation (alternating minimization)

23 Numerical geometry of non-rigid shapes Partial similarity Start with, Fix and, find correspondence and Fix and, find the parts and Repeat for a different value of Partial similarity computation Until convergence

24 Numerical geometry of non-rigid shapes Partial similarity Intrinsic partial similarity Intrinsic partial similarity is obtained by defining the fuzzy stress e.g. as the L 2 weighted stress

25 Numerical geometry of non-rigid shapes Partial similarity Example of intrinsic partial similarity Partial similarity Full similarity Scalar partial similarity Human Human- like Horse-like

26 Numerical geometry of non-rigid shapes Partial similarity Extrinsic partial similarity How to make ICP compare partially similar shapes? Introduce weights into the shape-to-shape distance to reject points with bad correspondence Possible selection of weights: where is a threshold on normals is a threshold on distance and are the normals to shapes and

27 Numerical geometry of non-rigid shapes Partial similarity Extrinsic partial similarity Without rejectionWith rejection

28 Numerical geometry of non-rigid shapes Partial similarity Extrinsic partial similarity Parts obtained by matching of man and centaur using ICP with rejection for different thresholds Problem: there is no explicit influence of the rejection thresholds and the size of the resulting parts Pareto framework allows to control the size of the selected parts!

29 Numerical geometry of non-rigid shapes Partial similarity Extrinsic partial similarity Correspondence stage (fixed and ) Closest point correspondence Weighted rigid alignment Correspondence

30 Numerical geometry of non-rigid shapes Partial similarity Extrinsic partial similarity Part selection stage (fixed and )

31 Numerical geometry of non-rigid shapes Partial similarity Extrinsic partial similarity Controllable part size by changing the value of

32 Numerical geometry of non-rigid shapes Partial similarity Not only size matters What is better?... Many small parts……or one large part?

33 Numerical geometry of non-rigid shapes Partial similarity Boundary regularization Large irregularitySmall irregularity Not only size matters: take into consideration the regularity of the parts Define shape irregularity as the boundary length Problem: what is the boundary of a fuzzy part?

34 Numerical geometry of non-rigid shapes Partial similarity Mumford-Shah functional Salvation comes from image segmentation [Mumford&Shah]: given image, find the segmented region replace by a membership function The two problems are equivalent

35 Numerical geometry of non-rigid shapes Partial similarity Mumford-Shah functional In our problem, we need only the integral along the boundary which becomes in the fuzzy setting

36 Numerical geometry of non-rigid shapes Partial similarity Regularized partial similarity INSIGNIFICANCE DISSIMILARITY IRREGULARITY