Graph Sparsifiers Nick Harvey Joint work with Isaac Fung TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A

What is the max flow from s to t? st

The answer in this graph is the same: it’s a Gomory-Hu tree. What is capacity of all edges incident on u? st 15 u

Can any dense graph be “approximated” by a sparse graph? Approximating pairwise distances – Spanners: number of edges = O(n 1+2/ ® ), distance approximated to within ®. [ADDJS’93] – Low-stretch trees: number of edges = n-1, “most” distances approximated to within log n. [FRT’04] Approximating all cuts – Sparsifiers: number of edges = O(n log n / ² 2 ), every cut approximated within 1+ ². [BK’96] Spectral approximation – Spectral sparsifiers: number of edges = O(n log n / ² 2 ), entire spectrum approximated within 1+ ². [SS’08] [BSS’09] n = # vertices

What is the point of all this? Approximating pairwise distances – Spanners: Network routing, motion planning, etc. – Low-stretch trees: Approximating metrics by simpler metrics, approximation algorithms

What is the point of all this? Approximating all cuts – Sparsifiers: fast algorithms for cut/flow problem ProblemApproximationRuntimeReference Min st Cut1+ ² O~(n 2 )BK’96 Sparsest CutO(log n)O~(n 2 )BK’96 Max st Flow1O~(m+nv)KL’02 Sparsest CutO~(n 2 )AHK’05 Sparsest CutO(log 2 n)O~(m+n 3/2 )KRV’06 Sparsest CutO~(m+n 3/2+ ² )S’09 Perfect Matching in Regular Bip. Graphs n/aO~(n 1.5 )GKK’09 Sparsest CutO~(m+n 1+ ² )M’10 v = flow value n = # vertices m = # edges

What is the point of all this? Spectral approximation – Spectral sparsifiers: solving diagonally-dominant linear systems in nearly linear time! Dimensionality reduction in L 1 ProblemRuntimeReference Computing Fiedler VectorO~(m)ST’04 Computing Effective ResistancesO~(m)SS’08 Sampling Random Spanning TreesKM’09 Max st FlowO~(m 4/3 )CKMST’10 Min st CutO~(m+n 4/3 )CKMST’10

Graph Sparsifiers: Formal problem statement Design an algorithm such that Input: An undirected graph G=(V,E) Output: A weighted subgraph H=(V,F,w), where F µ E and w : F ! R Goals: | | ± G (U)| - w( ± H (U)) | · ² | ± G (U)| 8 U µ V |F| = O(n log n / ² 2 ) Running time = O~( m / ² 2 ) # edges between U and V\U in G weight of edges between U and V\U in H n = # vertices m = # edges | | ± (U)| - w( ± (U)) | · ² | ± (U)| 8 U µ V

Why should sparsifiers exist? Example: G = Complete graph K n Sampling: Construct H by sampling every edge of G with probability p=100 log n/n Properties of H: # sampled edges = O(n log n) Standard fact: H is connected Stronger fact: p| ± G (U)| ¼ | ± H (U)| 8 U µ V Output: – H with each edge given weight 1/p – By this, H is a sparsifier of G

Chernoff Bound: Let X 1,X 2,... be {0,1} random variables. Let X =  i X i and let ¹ = E[ X ]. For any ± 2 [0,1], Pr[ |X- ¹ | ¸ ±¹ ] · 2 exp( - ± 2 ¹ / 3 ). Consider any cut ± G (U) with |U|=k. Then | ± G (U)| ¸ kn/2. Let X e = 1 if edge e is sampled. Let X =  e 2 C X e = | ± H (U)|. Then ¹ = p | ± (U)| ¸ 50 k log n. Say cut fails if |X- ¹ | ¸ ¹ /2. So Pr[ cut fails ] · 2 exp( - ¹ /12 ) · n -4k. # of cuts with |U|=k is. So Pr[ any cut fails ] ·  k n -4k <  k n -3k < n -2. So, whp, every U has || ± H (U)| - p | ± (U)|| < p | ± (U)|/2. Chernoff Bound Bound on # small cuts Key Ingredients Union bound

Generalize to arbitrary G? Can’t sample edges with same probability! Idea [BK’96] Sample low-connectivity edges with high probability, and high-connectivity edges with low probability Keep this Eliminate most of these

Non-uniform sampling algorithm [BK’96] Input: Graph G=(V,E), parameters p e 2 [0,1] Output: A weighted subgraph H=(V,F,w), where F µ E and w : F ! R For i=1 to ½ For each edge e 2 E With probability p e, Add e to F Increase w e by 1/( ½ p e ) Main Question: Can we choose ½ and p e ’s to achieve sparsification goals? Clearly running time is O( ½ m) (so want ½ · polylog(n)) Clearly |F| = O( ½  e p e ) (so want  e p e = O(n))

Non-uniform sampling algorithm [BK’96] Claim: H perfectly approximates G in expectation! For any e 2 E, E[ w e ] = 1 ) For every U µ V, E[ w( ± H (U)) ] = | ± G (U)| Goal: Show every w( ± H (U)) is tightly concentrated Input: Graph G=(V,E), parameters p e 2 [0,1] Output: A weighted subgraph H=(V,F,w), where F µ E and w : F ! R For i=1 to ½ For each edge e 2 E With probability p e, Add e to F Increase w e by 1/( ½ p e )

Prior Work Benczur-Karger ‘96 – Set ½ = O(log n), p e = 1/“strength” of edge e (max k s.t. e is contained in a k-edge-connected vertex-induced subgraph of G) – All cuts are preserved –  e p e · n ) |F| = O(n log n) – Running time is O(m log 3 n) Spielman-Srivastava ‘08 – Set ½ = O(log n), p e = “effective resistance” of edge e (view G as an electrical network where each edge is a 1-ohm resistor) – H is a spectral sparsifier of G ) all cuts are preserved –  e p e = n-1 ) |F| = O(n log n) – Running time is O(m log 50 n) – Uses powerful tools from Geometric Functional Analysis Assume ² is constant O(m log 3 n) [Peng et al. ‘10] See it here on Nov 22 nd.

Our Work Fung-Harvey ’10 (and independently Hariharan-Panigrahi ‘10) – Set ½ = O(log 2 n), p e = 1/edge-connectivity of edge e (min size of a cut that contains e) – Advantage: Edge connectivities easy to compute – All cuts are preserved –  e p e · n ) |F| = O(n log 2 n) – Running time is O(m log 2 n) Alternative Algorithm – Let H be union of ½ uniformly random spanning trees of G, where w e is 1/ ( ½ ¢ (effective resistance of e) ) – All cuts are preserved – |F| = O(n log 2 n) – Running time is Assume ² is constant

Notation: k uv = min size of a cut separating u and v Main ideas: – Partition edges into connectivity classes E = E 1 [ E 2 [... E log n where E i = { e : 2 i-1 · k e <2 i }

Notation: k uv = min size of a cut separating u and v Main ideas: – Partition edges into connectivity classes E = E 1 [ E 2 [... E log n where E i = { e : 2 i-1 · k e <2 i } – Prove weight of sampled edges that each cut takes from each connectivity class is about right – This yields a sparsifier U

Prove weight of sampled edges that each cut takes from each connectivity class is about right Notation: C = ± (U) is a cut C i = ± (U) Å E i is a cut-induced set Need to prove: C1C1 C2C2 C3C3 C4C4

Notation: C i = ± (U) Å E i is a cut-induced set C1C1 C2C2 C3C3 C4C4 Prove 8 cut-induced set C i Key Ingredients Chernoff bound: Prove small Bound on # small cuts: Prove #{ cut-induced sets C i induced by a small cut |C| } is small. Union bound: sum of failure probabilities is small, so probably no failures.

Counting Small Cut-Induced Sets Theorem: Let G=(V,E) be a graph. Fix any B µ E. Suppose k e ¸ K for all e in B. (k uv = min size of a cut separating u and v) Then, for every ® ¸ 1, |{ ± (U) Å B : | ± (U)| · ® K }| < n 2 ®. Corollary: Counting Small Cuts [K’93] Let G=(V,E) be a graph. Let K be the edge-connectivity of G. (i.e., global min cut value) Then, for every ® ¸ 1, |{ ± (U) : | ± (U)| · ® K }| < n 2 ®.

Comparison Theorem: Let G=(V,E) be a graph. Fix any B µ E. Suppose k e ¸ K for all e in B. (k uv = min size of a cut separating u and v) Then |{ ± (U) Å B : | ± (U)| · c }| < n 2c/K 8 c ¸ 1. Corollary [K’93]: Let G=(V,E) be a graph. Let K be the edge-connectivity of G. (i.e., global min cut value) Then, |{ ± (U) : | ± (U)| · c }| < n 2c/K 8 c ¸ 1. How many cuts of size 1? Theorem says < n 2, taking K=c=1. Corollary, says < 1, because K=0. (Slightly unfair)

Algorithm For Finding Needle in Haystack Input: A haystack Output: A needle (maybe) While haystack not too small – Pick a random handful – Throw it away End While Output whatever is left

Algorithm for Finding a Min Cut [K’93] Input: A graph Output: A minimum cut (maybe) While graph has  2 vertices “Not too small” – Pick an edge at random “Random Handful” – Contract it “Throw it away” End While Output remaining edges Claim: For any min cut, this algorithm outputs it with probability ¸ 1/n 2. Corollary: There are · n 2 min cuts.

Finding a Small Cut-Induced Set Input: A graph G=(V,E), and B µ E Output: A cut-induced subset of B While graph has  2 vertices – If some vertex v has no incident edges in B Split-off all edges at v and delete v – Pick an edge at random – Contract it End While Output remaining edges in B Claim: For any min cut-induced subset of B, this algorithm outputs it with probability > 1/n 2. Corollary: There are < n 2 min cut-induced subsets of B

Sparsifiers from Random Spanning Trees Let H be union of ½ =log 2 n uniform random spanning trees, where w e is 1/( ½ ¢ (effective resistance of e)) Then all cuts are preserved and |F| = O(n log 2 n) Why does this work? – Pr T [ e 2 T ] = effective resistance of edge e – Similar to usual independent sampling algorithm, with p e = effective resistance of e – Key difference: edges in a random spanning tree are not independent. – But, they are negatively correlated! That is enough to make Chernoff bounds work.

Conclusions Graph sparsifiers important for fast algorithms and some combinatorial theorems. Sampling by edge-connectivities gives a sparsifier with O(n log 2 n) edges Also true for sampling by effective resistances. ) sampling O(log 2 n) random spanning trees gives a sparsifier. Questions Improve log 2 n to log n? Sampling o(log n) random trees gives a sparsifier?

Fix some min cut. Say it has k edges. If algorithm doesn’t contract any edge in this cut, then the algorithm outputs this cut – When contracting edge uv, both u & v are on same side of cut So what is probability that this happens? While graph has  2 vertices “Not too small” – Pick an edge uv at random “Random Handful” – Contract it “Throw it away” End While Output remaining edges Analysis of Min Cut Algorithm

Initially there are n vertices. Claim 1: # edges in min cut=k  every vertex has degree  k  total # edges  nk/2 Pr[random edge is in min cut] = # edges in min cut / total # edges  k / (nk/2) = 2/n

Now there are n-1 vertices. Claim 2: min cut in remaining graph is  k Why? Every cut in remaining graph is also a cut in original graph. So, Pr[ random edge is in min cut ]  2/(n-1)

In general, when there are i vertices left Pr[ random edge is in min cut ]  2/i So Pr[ alg never contracts an edge in min cut ]