1. The Earth Mover's Distance
9/17/2018
The Earth Mover’s Distance as a Metric for Image Retrieval and as a Metric for Distributions with Applications to Image Databases Ranjini Swaminathan
University of Arizona
17 September September 2018

2. The Problem
To employ EMD as a method to find dissimilarities between images.
Image Parameters :
Images are described by features such as
color (3-D representation)
Texture (a distribution of energy in the frequency domain)
17 September September 2018

3. Need for methods
Image Retrieval-based on dissimilarity between two images.
Steps:
- Compute histograms/signatures
- Implement EMD /other method
- Navigation
17 September September 2018

4. Histograms
A histogram {hi} is a mapping from a set of d-dimensional integer vectors i to the set of nonnegative reals.
An example:
- Consider a grey-level histogram where d=1
- split possible grey values into N intervals
- image pixels fall into respective bins
17 September September 2018

5. Histograms contd. Disadvantage:
-fail to strike a balance between expressiveness and efficiency.
Adaptive Binning of Histograms:
-Need prior knowledge of distribution of features for all images.
-significant information restricted to few bins.
17 September September 2018

6. Signatures Signature {sj=(mj,wj) } - a set of clusters
mj - d-dimensional mode
Wj - number of pixels belonging to the cluster
A histogram is a special case of a signature.
17 September September 2018

7. Signatures contd.
Clusters
17 September September 2018

8. Methods Bin-By-Bin Dissimilarity measures - Minkowski-Form Distance:
dLr(H,K) = (hi-kir)1/r
- Histogram Intersection:
d(H,K) = 1 - i min(hi,ki)/i ki
17 September September 2018

9. Methods contd. - K-L Divergence and Jeffrey Divergence:
dKL(H,K)= i hi log hi/ki
dJ(H,K)= i (hi log hi/mi+ ki log ki/mi)
- X2 Statistics:
d2(H,K) = i(hi-mi)2/mi
17 September September 2018

10. Methods contd. Cross-Bin Dissimilarity measures
Quadratic-form distance
Match distance
- Kolmogorov-Smirnov distance
17 September September 2018

11. Signatures vs Histograms
- fixed size structures
- efficiency vs expressiveness
Signatures
- representative of features
- less info yet better retrieval
17 September September 2018

12. EMD
-measures the amount of work done in moving mass over holes in the same space.
-not a matching problem but a transportation problem
-increased efficiency
17 September September 2018

13. The Earth Mover's Distance
9/17/2018
Flow equations
WORK(P,Q,F)= ijdijfij
with the constraints
fij  0
j fij  wpi
i fij  wqj
ijfij = min(i wpi , j wqj )
where 1  i  m , 1  j  n
17 September September 2018

14. EMD-Features EMD(P,Q) = ijdijfij/ijfij Features:
-applies to signatures which subsume histograms
- avoids quantization problems
-allows partial matches & variable length descriptions
-metric
17 September September 2018

15. MDS Multi Dimensional Scaling as a Perceptual Evaluation Tool:
ij is the original distance between objects i and j
dij is the distance between the points in a low
dimensional space
STRESS = (i,j(dij - ij )2/ i,j ij 2)1/2
Zero Stress =perfect fit
17 September September 2018

16. Color
Courtesy :
17 September September 2018

17. Color Distributions Given :Set of images Do : cluster
obtain signatures
compute EMD with a stress value
Result : images aligned along lightness and chroma
MDS displays more subtle similarities between closely related images
17 September September 2018

18. Color distributions contd.
Test results show that EMD with signatures outperforms all other methods.
Navigation :
- Embeddings are adaptive
- Zooming into query
- Use of “don’t cares”
17 September September 2018

19. Efficiency EMD efficiency depends on: Transportation algorithm
Easy to compute lower bounds
However, EMD is :
Computationally intensive
Works better for smaller sample sizes
17 September September 2018

20. Notes Metric : (Merriam-Webster) Proof :
“a mathematical function that associates with each pair of elements of a set a real nonnegative number with the general properties of distance such that the number is zero only if the two elements are identical, the number is the same regardless of the order in which the two elements are taken, and the number associated with one pair of elements plus that associated with one member of the pair and a third element is equal to or greater than the number associated with the other member of the pair and the third element.”
Proof :
Triangle Inequality
17 September September 2018

21. Future Work
Other vision problems –classification, recognition and segmentation
Outside computer vision????
-in speech recognition
-in biology
17 September September 2018