Graph Sparsifiers by Edge-Connectivity and Random Spanning Trees Nick Harvey U. Waterloo Department of Combinatorics and Optimization Joint work with Isaac Fung TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A

What is the max flow from s to t? st

The answer in this graph is the same: it’s a Gomory-Hu tree. What is capacity of all edges incident on u? st 15 u

Can any dense graph be “approximated” by a sparse graph? Approximating pairwise distances – Spanners: number of edges = O(n 1+2/ ® ), distance approximated to within ®. [ADDJS’93,...,P’09] – Low-stretch trees: number of edges = n-1, “most” distances approximated to within log n. [FRT’04] Approximating all cuts – Sparsifiers: number of edges = O(n log n / ² 2 ), every cut approximated within 1+ ². [BK’96] Spectral approximation – Spectral sparsifiers: number of edges = O(n log n / ² 2 ), entire spectrum approximated within 1+ ². [SS’08] [BSS’09] n = # vertices

What is the point of all this? Approximating pairwise distances – Spanners: Network routing Motion planning Etc. – Low stretch / congestion trees: Approximating metrics by simpler metrics Approximation algorithms Online algorithms

What is the point of all this? Approximating all cuts – Sparsifiers: fast algorithms for cut/flow problem ProblemApproximationRuntimeReference Min st Cut1+ ² O~(n 2 )BK’96 Sparsest CutO(log n)O~(n 2 )BK’96 Max st Flow1O~(m+nv)KL’02 Sparsest CutO~(n 2 )AHK’05 Sparsest CutO(log 2 n)O~(m+n 3/2 )KRV’06 Sparsest CutO~(m+n 3/2+ ² )S’09 Perfect Matching in Regular Bip. Graphs n/aO~(n 1.5 )GKK’09 Sparsest CutO~(m+n 1+ ² )M’10 v = flow value n = # vertices m = # edges

What is the point of all this? Spectral approximation – Spectral sparsifiers: solving diagonally-dominant linear systems in nearly linear time! Connections to metric embeddings, Banach spaces... ProblemRuntimeReference Computing Fiedler VectorO~(m)ST’04 Computing Effective ResistancesO~(m)SS’08 Sampling Random Spanning TreesKM’09 Max st FlowO~(m 4/3 )CKMST’10 Min st CutO~(m+n 4/3 )CKMST’10

Graph Sparsifiers: Formal problem statement Design an algorithm such that Input: An undirected graph G=(V,E) Output: A weighted subgraph H=(V,F,w), where F µ E and w : F ! R Goals: | | ± G (U)| - w( ± H (U)) | · ² | ± G (U)| 8 U µ V |F| = O(n log n / ² 2 ) Running time = O~( m / ² 2 ) # edges between U and V\U in G weight of edges between U and V\U in H n = # vertices m = # edges | | ± (U)| - w( ± (U)) | · ² | ± (U)| 8 U µ V

Why should sparsifiers exist? Example: G = Complete graph K n Sampling: Construct H by sampling every edge of G with probability p=100 log n/n Properties of H: # sampled edges = O(n log n) Standard fact: H is connected Stronger fact: p| ± G (U)| ¼ | ± H (U)| 8 U µ V Output: – H with each edge given weight 1/p – By this, H is a sparsifier of G

Chernoff Bound: Let X 1,X 2,... be {0,1} random variables. Let X =  i X i and let ¹ = E[ X ]. For any ± 2 [0,1], Pr[ |X- ¹ | ¸ ±¹ ] · 2 exp( - ± 2 ¹ / 3 ). Consider any cut ± G (U) with |U|=k. Then | ± G (U)| ¸ kn/2. Let X e = 1 if edge e is sampled. Let X =  e 2 C X e = | ± H (U)|. Then ¹ = E[X] = p | ± (U)| ¸ 50 k log n. Say cut fails if |X- ¹ | ¸ ¹ /2. So Pr[ cut fails ] · 2 exp( - ¹ /12 ) · n -4k. # of cuts with |U|=k is. So Pr[ any cut fails ] ·  k n -4k <  k n -3k < n -2. So, whp, every U has || ± H (U)| - p | ± (U)|| < p | ± (U)|/2. Chernoff Bound Bound on # small cuts Key Ingredients Union bound

Generalize to arbitrary G? Can’t sample edges with same probability! Idea [BK’96] Sample low-connectivity edges with high probability, and high-connectivity edges with low probability Keep this Eliminate most of these

Non-uniform sampling algorithm [BK’96] Input: Graph G=(V,E), parameters p e 2 [0,1] Output: A weighted subgraph H=(V,F,w), where F µ E and w : F ! R For i=1 to ½ For each edge e 2 E With probability p e, Add e to F Increase w e by 1/( ½ p e ) Main Question: Can we choose ½ and p e ’s to achieve sparsification goals?

Non-uniform sampling algorithm [BK’96] Claim: H perfectly approximates G in expectation! For any e 2 E, E[ w e ] = 1 ) For every U µ V, E[ w( ± H (U)) ] = | ± G (U)| Goal: Show every w( ± H (U)) is tightly concentrated Input: Graph G=(V,E), parameters p e 2 [0,1] Output: A weighted subgraph H=(V,F,w), where F µ E and w : F ! R For i=1 to ½ For each edge e 2 E With probability p e, Add e to F Increase w e by 1/( ½ p e )

Prior Work Benczur-Karger ‘96 – Set ½ = O(log n), p e = 1/“strength” of edge e (max k s.t. e is contained in a k-edge-connected vertex-induced subgraph of G) – All cuts are preserved –  e p e · n ) |F| = O(n log n) – Running time is O(m log 3 n) Spielman-Srivastava ‘08 – Set ½ = O(log n), p e = “effective resistance” of edge e (view G as an electrical network where each edge is a 1-ohm resistor) – H is a spectral sparsifier of G ) all cuts are preserved –  e p e = n-1 ) |F| = O(n log n) – Running time is O(m log 50 n) – Uses matrix-valued concentration inequalities Assume ² is constant O(m log 3 n) [Koutis-Miller-Peng ’10] Similar to edge connectivity.

Our Work Fung-Harvey ’10 (and independently Hariharan-Panigrahi ‘10) – Set ½ = O(log 2 n), p e = 1/edge-connectivity of edge e (min size of a cut that contains e) – All cuts are preserved –  e p e · n ) |F| = O(n log 2 n) – Running time is O(m log 2 n) – Advantages: Edge connectivities natural, easy to compute, Faster than previous algorithms, Also implies sampling by strength / effective resistances works. – Disadvantages: Extra log factor, no spectral sparsification. Panigrahi ’10 – A sparsifier with O(n log n / ² 2 ) edges, with running time O(m) in unwtd graphs and O(m)+O~(n/ ² 2 ) in wtd graphs Assume ² is constant

Our Work Alternative Algorithm – Let H be union of ½ uniformly random spanning trees of G, where w e is 1/ ( ½ ¢ (effective resistance of e) ) – All cuts are preserved – |F| = O(n log 2 n) – Running time is Motivation – Perhaps using random spanning trees leads to a sparsifier with O(n) edges? Negative result –  ( log n ) spanning trees needed Assume ² is constant

Notation: k uv = min size of a cut separating u and v Main ideas: – Partition edges into connectivity classes E = E 1 [ E 2 [... E log n where E i = { e : 2 i-1 · k e <2 i }

Notation: k uv = min size of a cut separating u and v Main ideas: – Partition edges into connectivity classes E = E 1 [ E 2 [... E log n where E i = { e : 2 i-1 · k e <2 i } – Prove weight of sampled edges that each cut takes from each connectivity class is about right – This yields a sparsifier U

Prove weight of sampled edges that each cut takes from each connectivity class is about right Notation: C = ± (U) is a cut C i = ± (U) Å E i is a cut-induced set Need to prove: C1C1 C2C2 C3C3 C4C4

Notation: C i = ± (U) Å E i is a cut-induced set C1C1 C2C2 C3C3 C4C4 Prove 8 cut-induced set C i Key Ingredients Chernoff bound: Prove small Bound on # small cuts: Prove #{ cut-induced sets C i induced by a small cut |C| } is small. Union bound: sum of failure probabilities is small, so probably no failures.

Counting Small Cut-Induced Sets Theorem: Let G=(V,E) be a graph. Fix any B µ E. Suppose k e ¸ K for all e in B. (k uv = min size of a cut separating u and v) Then, for every ® ¸ 1, |{ ± (U) Å B : | ± (U)| · ® K }| < n 2 ®. Corollary: Counting Small Cuts [K’93] Let G=(V,E) be a graph. Let K be the edge-connectivity of G. (i.e., global min cut value) Then, for every ® ¸ 1, |{ ± (U) : | ± (U)| · ® K }| < n 2 ®.

Comparison Theorem: Let G=(V,E) be a graph. Fix any B µ E. Suppose k e ¸ K for all e in B. (k uv = min size of a cut separating u and v) Then |{ ± (U) Å B : | ± (U)| · c }| < n 2c/K 8 c ¸ 1. Corollary [K’93]: Let G=(V,E) be a graph. Let K be the edge-connectivity of G. (i.e., global min cut value) Then, |{ ± (U) : | ± (U)| · c }| < n 2c/K 8 c ¸ 1. How many cuts of size 1? Theorem says < n 2, taking K=c=1. Corollary, says < 1, because K=0. (Slightly unfair)

Algorithm For Finding Needle in Haystack Input: A haystack Output: A needle (maybe) While haystack not too small – Pick a random handful – Throw it away End While Output whatever is left

Algorithm for Finding a Min Cut [K’93] Input: A graph Output: A minimum cut (maybe) While graph has  2 vertices “Not too small” – Pick an edge at random “Random Handful” – Contract it “Throw it away” End While Output remaining edges Claim: For any min cut, this algorithm outputs it with probability ¸ 1/n 2. Corollary: There are · n 2 min cuts.

Finding a Small Cut-Induced Set Input: A graph G=(V,E), and B µ E Output: A cut-induced subset of B While graph has  2 vertices – If some vertex v has no incident edges in B Split-off all edges at v and delete v – Pick an edge at random – Contract it End While Output remaining edges in B Claim: For any min cut-induced subset of B, this algorithm outputs it with probability > 1/n 2. Corollary: There are < n 2 min cut-induced subsets of B

Conclusions Graph sparsifiers important for fast algorithms and some combinatorial theorems Sampling by edge-connectivities gives a sparsifier with O(n log 2 n) edges in O(m log 2 n) time – Improvements: O(n log n) edges in O(m) + O~(n) time [Joint with Hariharan and Panigrahi] Sampling by effective resistances also works ) sampling O(log 2 n) random spanning trees gives a sparsifier. Questions Improve log 2 n to log n? Other ways to get sparsifiers with O(n) edges?