1. Sparse RecoveryAlgorithmResults  Original signal x = x k + u, where x k has k large coefficients and u is noise.  Acquire measurements Ax = y. If |x|=n, A is an m x n matrix, and usually m = O(k log n) << n.  Recover vector x* from y. The usual guarantee is ||x* – x|| 1 < C ||u|| = C min k-sparse x’ ||x' – x|| 1, for some constant C. In other words, x* is close to the best possible k-sparse approximation of x. High Level Overview  Use a distance preserving embedding to map the problem under EMD to a problem under the L 1 norm.  Solve as a standard sparse recovery problem in the L 1 norm. The Pyramid Mapping  Use a distance preserving embedding to map the problem under EMD to a problem under the L 1 norm.  For each j = 0, 1, 2,..., form a grid with cell length d j = 2 j, and a mapping P j, where each entry of P j x corresponds to the total mass of a cell of the grid.  The mapping Px is the concatenation [d 0 P 0 x, d 1 P 1 x, d 2 P 2 x,...].  If the grids are shifted by a random vector, E[||Px 1 – Px 2 || 1 ] < O(log n) ||x 1 – x 2 || EMD, and ||Px 1 – Px 2 || 1 > ||x 1 – x 2 || EMD  for images x 1 and x 2 [3].  Hence P is a linear, distance preserving embedding from the L 1 norm to the EMD norm. Also, if x is k-sparse, Px is k log(n)-sparse. Full Algorithm  Let A be any matrix enabling k log(n)-sparse recovery, eg. from [1].  Measurements y = APx, where P is the pyramid mapping from above.  Recovery: Compute “A -1 y” using a sparse recovery algorithm, and then compute x* = P -1 A -1 y using an “inverse” pyramid mapping.  In our experiments, we use the following constraints during the sparse recovery portion of the algorithm to reduce the number of measurements |y|:  x is non-negative for real life images.  The large coefficients of Px form a “tree”. Theoretical  We find a set of m x n matrices B, m = O(k log n/k), such that for any x, given Bx, we can compute x* such that ||x* - x|| EMD < C min k-sparse x’ ||x' – x|| EMD, with high probability, in O(k log n) time, for a constant C. Experimental  We use Sequential Sparse Matching Pursuit (SSMP) [1] for the Sparse Recovery portion of the algorithm.  We test our algorithm on the images such as the one below to the left. After recovery, we attempt to locate five star-like objects in x*.  Each point on the graph on the right is the median over 15 trials of the average distance from recovered stars to the actual stars. If fewer than 5 stars are found, the distance is assumed to be infinite and is not displayed.  The EMD algorithm recovers the stars using substantially fewer measurements than standard SSMP. Earth Movers Distance (EMD)  x represents a 2 dimensional image.  If x 1 (green) and x 2 (purple) are binary images such that ||x 1 || 1 = ||x 2 || 1, then ||x 1 – x 2 || EMD is the minimum cost matching shown to the right.  There are natural generalizations to non-binary vectors with unequal L 1 mass. Problem Statement  We want to perform sparse recovery as above, but we would like to recover x* such that x* is close to x under EMD rather than L 1.  EMD corresponds well with our perceptual notion of image similarity, and is used in computer vision algorithms. By contrast, even small translations in an image result in almost maximal L 1 difference [4]. References [1]Berinde, Indyk. Sequential sparse matching pursuit. Allerton 2009. [2]Gupta, Indyk, Price. Sparse recovery for earth mover distance. Allerton 2010. [3]Indyk, Thaper. Fast image retrieval via embeddings. ICCV 2003. [4]Rubner, Tomasi, Guibas. The earth mover's distance as a metric for image retrieval. IJCV 2000. Sparse Recovery for Earth Mover Distance MADALGO – Center for Massive Data Algorithmics, a Center of the Danish National Research Foundation Eric Price MIT Piotr Indyk MIT Rishi Gupta MIT