On the Impossibility of Dimension Reduction for Doubling Subsets of L p Yair Bartal Lee-Ad Gottlieb Ofer Neiman

Embedding and Distortion

JL Lemma Lemma: Any n points in L 2 can be embedded into L 2 k, k=O((log n)/ε 2 ) with 1+ε distortion Extremely useful for many applications: – Machine learning – Compressive sensing – Nearest Neighbor search – Many others… Limitations: specific to L 2, dimension depends on n – There are lower bounds for dimension reduction in L 1, L ∞

Lower bounds on Dimension Reduction

Doubling Dimension Doubling constant: The minimal λ so that every ball of radius 2r can be covered by λ balls of radius r Doubling dimension: log 2 λ A measure for dimensionality of a metric space Generalizes the dimension for normed space: L p k has doubling dimension Θ(k) The volume argument holds only for metrics with high doubling dimension

Overcoming the Lower Bounds? One could hope for an analogous version of the JL- Lemma for doubling subsets Question: Does every set of points in L 2 of constant doubling dimension, embeds to constant dimensional space with constant distortion? More ambitiously: Any subset of L 2 with doubling constant λ, can be embedded into L 2 k, k=O((log λ)/ε 2 ) with 1+ε distortion

Our Result

Implications

The Laakso Graph A recursive graph, G i+1 is obtained from G i by replacing every edge with a copy of G 1 A series-parallel graph Has doubling constant 6 G0G0 G1G1 G2G2

Simple Case: p=∞

v Consider a single iteration The pair a,b is an edge of the previous iteration Let f j be the j-th coordinate There is a natural embedding that does not increase stretch... But then u,v may be distorted ab s t u f j (a) f j (b)

Simple Case: p=∞ For simplicity (and w.l.o.g) assume – f j (s)=(f j (b)-f j (a))/4 – f j (t)=3(f j (b)-f j (a))/4 – f j (v)=(f j (b)-f j (a))/2 Let Δ j (u) be the difference between f j (u) and f j (v) The distortion D requirement imposes that for some j, Δ j (u)>1/D (normalizing so that d(u,v)=1) v ab s t u f j (a) f j (b) Δ j (u)

Simple Case: p=∞ v ab s t f j (a) f j (b) Δ j (u) u v ab s t f h (a) f h (b) -Δ h (u) u

Simple Case: p=∞ v ab s t f j (a) f j (b) Δ j (u) u

Simple Case: p=∞ u s a b t v

Going Beyond Infinity For p<∞, we cannot use the Laakso graph – Requires high distortion to embed it into L p Instead, we build an instance in L p, inspired by the Laakso graph The new points u,v will use a new dimension Parameter ε determines the (scaled) u,v distance as u v t b ε

Going Beyond Infinity Problem: the u,s distance is now larger than 1, roughly 1+ε p Causes a loss of ≈ ε p in the stretch of each level Since u,v are at distance ε, the increase to the stretch is now only (ε/D) 2 When p>2, there is a choice of ε for which the increase overcomes the loss as u v t b ε

Conclusion We show a strong lower bound against dimension reduction for doubling subsets of L p, for any p>2 Can our techniques be extended to 1<p<2 ? – The u,s distance when p<2 is quite large, ≈ 1+(p-1)ε 2, so a different approach is required General doubling metrics embed to L p with distortion O(log 1/p n) (for p≥2) – Can this distortion bound be obtained in constant dimension?