1. The Intrinsic Dimension of Metric Spaces
Anupam Gupta Carnegie Mellon University
3ème cycle romand de Recherche Opérationnelle, 2010 Seminar, Lecture #4

2. y z x Metric space M = (V, d) set V of points
symmetric non-negative distances d(x,y)
triangle inequality
d(x,y) ≤ d(x,z) + d(z,y)
d(x,x) = 0 y z x

3. in the previous two lectures…
We saw: every metric is (almost) a (random) tree every metric is (almost) an Euclidean metric every Euclidean metric is (almost) a low-dimensional metric Today: how to measure the complexity of a metric space and doing “better” dimension reduction

4. “better” dimension reduction
Can we do “better” than random projections?

5. tight bounds on dimension reduction
Theorem [1984] Given any n-point subset V of Euclidean space, one can map into O(log n/²2)-dimensional space while incurring a distortion at most (1+²). Theorem (Alon): Lower bound of (log n/(²2 log ² -1)) again for the “uniform/equidistant” metric Un

6. so what do we do now?
Note that these are uniform results. (“All n-point Euclidean metrics have property blah(n) …”) Can we give a better “per-instance” guarantee? (“For any n-point Euclidean metric M, we get property blah(M) …”) Note: If the metric contains a copy of equidistant metric Ut we know we need (log t) dimensions if we want O(1) distortion Similar lower bounds if our metric contains almost equidistant metrics But are these the only obstructions to low-dimensionality?

7. more nuanced results To paraphrase the results recent proven:
If a Euclidean metric embeds into Rk for some dimension k
with distortion O(1)
then we can find an embedding into RO(k)
with distortion O(log n)
[Chan G. Talwar ’08, Abraham Bartal Neiman ’08]
JL says: we can find an embedding into RO(log n)

8. here’s how we do it
come up with convenient ways of capturing the property that there are no large near-equidistant metrics formally: define a notion of “intrinsic dimension” of a point set in Euclidean space in fact, our definitions will hold for any metric space, not just for point sets in Euclidean space. And we will make use of this generality to get good algos…

9. rest of this talk
Give two definitions of metric dimension: a) pointwise dimension b) doubling dimension Develop good algorithms for metrics with low pointwise/doubling dimension. Also, get “better” dimension reduction for metrics with low doubling dimension

10. pointwise dimension The pointwise dimension of a metric is at most k
if for all points x, number of points within distance 2r of point x
≤ 2k (Number of points within distance r of point x) x r

11. pointwise dimension
captures sets of regularly spaced points in Rk

12. using it for near-neighbor searching
Given a metric space M = (X, d) and a subset S of X
preprocess it so that
given query point q
we can return point in S (approximately) closest to q.

13. using it for near-neighbor searching
Query point q. Suppose I know a point x at distance d(q,x) = r.
If I find a point y at distance r/2 from q,
 have made progress.
Allowed operations:
Can sample points from “balls” around any point in the original point set S.
(sample from points in S  Ball(point x, radius r))

14. algorithm by picture
Query point q. Suppose I know a point x at distance d(q,x) = r.
Want to find a point y at distance r/2 from q.
3r/2 x r q
r/2

15. algorithm by picture
Query point q. Suppose I know a point x at distance d(q,x) = r. Want to find a point y at distance r/2 from q.
Suppose we sample
from B(x, 3r/2) to find
such a point y
3r/2
Bad case:
Most points in this
ball lie close to x x r
Not possible! This
would cause high
“growth rate”!!! q
r/2

16. algorithm by picture |B(x,3r/2)| ≤ |B(q, 5r/2)| ≤ |B(q, 4r)|
≤ 23k |B(q,r/2)|
 Probability of hitting a good point ≥ 1/23k
3r/2
Near-neighbor Algorithm:
1. Let x = closest point seen so far, with distance r = d(x,q)
2. Pick 23k samples from B(x, 5r/2)
3. If see some point at distance ≤ r/2 from q, go to line1
else output closest point seen so far x r q
r/2

17. pointwise dimension used widely
Used to model “tractable/simple” networks by [Kamoun Kleinrock], [Kleinberg Tardos], [Plaxton Rajaraman Richa], [Karger Ruhl], [Beygelzimer Kakade Langford]… x r

18.  but… Drawback: the definition is not closed under taking subsets
(A subset of a low-dimensional metric can be high dimensional.) r 
O(lg r2)
2-dimensional set

19. and that’s not all
Pointwise dimension is a somewhat restrictive notion
(unclear if “real” metrics have low pointwise dimension)
Would like something a bit more robust and general

20. new notion: doubling dimension
Dimension dimD(M) is at most k such that
for any set S, if S has diameter DS
it can be covered by 2k sets of diameter ½DS D

21. new notion: doubling dimension
Dimension dimD(M) is at most k such that
for any set S, if S has diameter DS
it can be covered by 2k sets of diameter ½DS D

22. doubling generalizes geometric dimension
Take k-dim Euclidean space Rk Claim: dimD(Rk) ≈ Θ(k) Easy to see for boxes Argument for spheres a bit more involved.
23 boxes to cover
larger box in R3

23. “doubling metrics”
Dimension at most k if
every set S with diameter DS can be covered by 2k sets of diameter ½DS
A family of metric spaces is called “doubling” if there exists a constant k such that the doubling dimension of these metrics is bounded by k

24. what is not a doubling metric?
The equidistant metric Ut on t points has dimension (log t) Hence low doubling dimension captures the fact that the metric does not have large equidistant metrics.

25. doubling generalizes pointwise dim
Fact: If a metric has pointwise dimension k, it has doubling dimension O(k)

26. the picture thus far…

27. rest of this talk
Give two definitions of metric dimension: a) pointwise dimension b) doubling dimension Develop good algorithms for metrics with low pointwise/doubling dimension. Also get “better” dimension reduction for metrics with low doubling dimension

28. some useful properties of doubling dimension

29. small near-equidistant metrics (restated) 
this 2-dim set has (/d)2 points
Fact: Suppose a metric (X,d) has doubling dimension k.
If any subset S µ X of points has
all inter-point distances lying between ± and ¢
there are at most (¢/±)O(k) points in S.

30. the (simple) proof

31. advantages of this fact
Thm: Doubling metrics admit O(dim(M))-padded decompositions Useful wherever padded decompositions are useful E.g.: can prove that all doubling metrics embed into ℓ2 with distortion

32. btw, just to check
Natural Q: Do all doubling metrics embed into ℓ2 with distortion O(1)?

33. a substantial generalization
Many geometric algorithms can be extended to doubling spaces…
Near neighbor search
Compact routing
Distance labeling
Network triangulation
Sensor placements
Small-world networks
Traveling Salesman
Sparse Spanners
Approx. inference
Network Design
Clustering problems
Well-separated pair decomposition
Data structures
Learnability

34. example application
Assign labels L(x) to each host x in a metric space Looking just at L(x) and L(y), can infer distance d(x,y)
Results
labels with (O(1)/ε)dim × log n bits
estimates within (1 + ε) factor y
010001
010001
Contrast with
lower bound of n bit labels
in general for any factor < 2
110001
110001 x
f( , )
≈ d(x,y)

35. another example
[Arora 95] showed that TSP on Rk was (1+²)-approximable in time
[Talwar 04] extended this result to metrics with doubling dimension k

36. example in action: sparse spanners for doubling metrics [Chan G
example in action: sparse spanners for doubling metrics [Chan G. Maggs Zhou]

37. spanners
Given a metric M = (V, d), a graph G = (V, E) is an (m, ²)-spanner if 1) number of edges in G is m 2) d(x,y) ≤ dG(x,y) ≤ (1 + ²) d(x,y) A reasonable goal: (?) ² = 0.1, m = O(n) Fact: For the equidistant metric Un, if ² < 1 then G = Kn

38. spanners for doubling metrics
Theorem: [Chan G. Maggs Zhou] Given any metric M, and any ² < ½, we can efficiently find an (m, ²)-spanner G with m = n (1 + 1/²) dimD(M) Hence, for doubling metrics, linear-sized spanners! Independently proved by [HarPeled Mendel], they give better runtime Generalizes a similar theorem for Euclidean metrics due to [Arya Das Narasimhan]

39. standard tool: nets Nets: A set of points N is an r-net of a set S if
d(u,v) ≥ r for any u,v 2 N
For every w 2 S \ N, there is a u 2 N with d(u,w) < r r

40. standard tool: nets Nets: A set of points N is an r-net of S if
d(u,v) ≥ r for any u,v 2 N
For every w 2 S \ N, there is a u 2 N with d(u,w) < r
Fact: If a metric has doubling dim k and N is an r-net
) B(x,2r) Å N · O(1)k

41. recursive nets 16 8 4 2
Suppose all the points were at least unit distance apart
so you take a 2-net N1 of these points
Now you can take a 4-net N2 of this net
And so on…

42. recursive nets Nt is a 2t-net of the set Nt-1
N0 = V
Nt is a 2t-net of the set Nt-1
 Nt is a 2t+1-net of the set V (almost)

43. the spanner construction
N0 = V
Nt is a 2t-net of the set Nt-1
 Nt is a 2t+1-net of the set V (almost)

44. the sparsity

45. the stretch

46. rest of this talk
Give two definitions of metric dimension: a) pointwise dimension b) doubling dimension Develop good algorithms for metrics with low pointwise/doubling dimension. Also get “better” dimension reduction for metrics with low doubling dimension

47. want to improve on J-L
Theorem [1984] Given any n-point subset V of Euclidean space, one can map it into O(log n/²2)-dimensional space while incurring distortion at most (1+²). Want map to depend on the metric space. (JL just computes a random projection.) Fact: we need a non-linear map.

48. back to dimensionality reduction
To paraphrase the best currently-known results: If a Euclidean metric embeds into Rk for some dimension k with distortion O(1) the metric has doubling dimension O(k) we can find an embedding into RO(k) with distortion O(log n)  

49. new: “per-instance” bounds
Theorem: [Chan G. Talwar] Any metric with doubling dimension k embeds into Euclidean space with T dimensions with distortion (where T 2 [ k log log n, log n]) Independently, [Abraham Bartal Neiman] get similar results.

50. special cases of interest
Distortion
on using O(dimD (M))
Euclidean dimensions
Distortion on using O(log n)
General metrics
Euclidean
If the metric is doubling, this quantity is sqrt{log n}.
In general, this is never more than O(log n).
Again generalizes the previous result.
This generalizes result we talked about in Lecture #2:
any metric embeds into Euclidean space with O(log n) distortion
This is just the Johnson-Lindenstrauss lemma.

51. thank you!