Randomness in Computation and Communication Part 1: Randomized algorithms Lap Chi Lau CSE CUHK

Making Decision Flip a coin.

Making Decision Flip a coin! An algorithm which flip coins is called a randomized algorithm.

Why Randomness? A randomized algorithm is often simpler and faster. Making good decisions could be complicated and expensive.  Minimum spanning trees A linear time randomized algorithm, but no known linear time deterministic algorithm.  Primality testing A randomized polynomial time algorithm, but it takes thirty years to find a deterministic one.  Volume estimation of a convex body A randomized polynomial time approximation algorithm, but no known deterministic polynomial time approximation algorithm.

Minimum Cut Minimum cut: Given an undirected multi-graph G with n vertices, find a cut of minimum cardinality (a min-cut). A cut is a set of edges whose removal disconnects the graph.

Minimum Cut Minimum cut: Given an undirected multi-graph G with n vertices, find a cut of minimum cardinality (a min-cut). A cut is a set of edges whose removal disconnects the graph. A minimum s-t cut can be solved in polynomial time by a max-flow algorithm. s t Minimum s-t cut = maximum number of edge-disjoint s-t paths

Minimum Cut Naive: To compute the (global) min-cut, an algorithm is to compute minimum s-t cuts for all pairs of s and t, and take the minimum one. Running time: O(n 2 · max-flow) = O(n 3 · m) Observation: fix s and consider minimum s-t cuts for t. Running time: O(n · max-flow) = O(n 2 · m) [Nagamochi-Ibaraki 92, Stoer-Wagner 97]O(nm + n 2 log n) new ideas that lead to faster randomized algorithms

Edge Contraction Contraction of an edge e = (u,v): Merge u and v into a single vertex “uv”. For any edge (w,u) or (w,v), this edge becomes (w,“uv”) in the resulting graph. Delete any edge which becomes a “self-loop”, i.e. an edge of the form (v,v) ,2 5 3

Edge Contraction Observation: an edge contraction would not decrease the min-cut size. This is because every cut in the graph at any intermediate stage is a cut in the original graph ,2 5 3

A Randomized Algorithm Algorithm Contract: Input: A multigraph G=(V,E) Output: A cut C 1.H <- G. 2.while H has more than 2 vertices do 2.1 choose an edge (x,y) uniformly at random from the edges in H. 2.2 H <- H/(x,y). (contract the edge (x,y)) 3. C <- the set of vertices corresponding to one of the two meta-vertices in H. Running time: Each iteration takes O(n) time. Each iteration has one fewer vertex. So, there are at most n iteration. Total running time O(n 2 ).

Analysis Obviously, this algorithm will not always work. How should we analyze the success probability? Let C be a minimum cut, and E(C) be the edges crossing C. Claim 1: C is produced as output if and only if none of the edges in E(C) is contracted by the algorithm. So, we want to estimate the probability that an edge in E(C) is picked. What is this probability?

Analysis Let k be the number of edges in E(C). How many edges are there in the initial graph? Claim 2: The graph has at least nk/2 edges. Because each vertex has degree at least k. So, the probability that an edge in E(C) is picked at the first iteration is at most 2/n.

Analysis In the i-th iteration, there are n-i+1 vertices in the graph. Observation: an edge contraction would not decrease the min-cut size. So the min-cut still has at least k edges. Hence there are at least (n-i+1)k/2 edges in the i-th iteration. The probability that an edge in E(C) is picked is at most 2/(n-i+1).

Analysis So, the probability that a particular min- cut is output by Algorithm Contract with probability at least Ω(n -2 ).

Boosting the Probability To boost the probability, we repeat the algorithm for n 2 log(n) times. What is the probability that it still fails to find a min-cut? So, high probability to find a min-cut in O(n 4 log(n)) time.

Reflecting It is a simpler algorithm for minimum cut, but is slower and could make mistakes… This algorithm was proposed by Karger in New way of thinking, leading to a revolution. We first present the improvement by Karger and Stein in 1993.

Ideas to an Improved Algorithm Key: The probability that an edge in E(C) is picked is at most 2/(n-i+1). Observation: at early iterations, it is not very likely that the algorithm fails. Idea: “share” the random choices at the beginning! In the previous algorithm, we boost the probability by running the Algorithm Contract (from scratch) independently many times. G n contractions n 2 leaves

Ideas to an Improved Algorithm Idea: “share” the random choices at the beginning! Previous algorithm. G n contractions n 2 leaves G ……………… n 2 leaves n/2 contractions n/4 contractions n/8 contractions log(n) levels New algorithm.

A Fast Randomized Algorithm Algorithm FastCut: Input: A multigraph G=(V,E) Output: A cut C 1.n <- |V|. 2.if n <= 6, then compute min-cut by brute-force enumeration else 2.1 t <- (1 + n/√2). 2.2 Using Algorithm Contract, perform two independent contraction sequences to obtain graphs H1 and H2 each with t vertices. 2.3 Recursively compute min-cuts in each of H1 and H return the smaller of the two min-cuts.

A Surprise Theorem 1: Algorithm Fastcut runs in O(n 2 log(n)) time. Theorem 2: Algorithm Fastcut succeeds with probability Ω(1/log(n)). By a similar boosting argument, repeating Fastcut for log 2 (n) times would succeed with high probability. Total running time = O(n 2 log 3 (n))! It is much faster than the best known deterministic algorithm, which has running time O(n 3 ). Min-cut is faster than Max-Flow!

Complexity G n 2 leaves n/2 contractions n/4 contractions n/8 contractions log(n) levels First level complexity = 2 x (n 2 /2) = n 2. Second level complexity = 4 x (n 2 /4) = n 2. Total time = n 2 log(n). The i-th level complexity = 2 i x (n 2 /2 i ) = n 2. Theorem 1: Algorithm Fastcut runs in O(n 2 log(n)) time.

Theorem 2: Algorithm Fastcut succeeds with probability Ω(1/log(n)). Analysis G n 2 leaves n/2 contractions n/4 contractions n/8 contractions log(n) levels Claim: for each “branch”, the survive probability is at least ½.

Theorem 2: Algorithm Fastcut succeeds with probability Ω(1/log(n)). Analysis Proof: Let k denote the depth of recursion, and p(k) be a lower bound on the success probability. Solving the recurrence will give:

Remarks Fastcut is fast, simple and cute. Combinatorial consequence: # of min cut is O(n 2 ). [Karger 96] Near linear time algorithm for minimum cut, O(m log 3 n). Open problem 1: deterministic o(mn) min-cut algorithm? Open problem 2: fast randomized min s-t cut algorithm? Error probability is independent of the problem instance.

Bipartite Matching The bipartite matching problem: Find a matching with the maximum number of edges. A matching M is a subset of edges so that every vertex has degree at most one in M. [Hopcroft-Karp 73] Bipartite matching solved in O(m√n) time.

Edmonds Matrix A perfect matching is a matching in which every vertex is matched. We will present a “fast” randomized algorithm for finding a perfect matching. The algorithm is based on some algebraic techniques. u1u1 u2u2 u3u3 u4u4 v1v1 v2v2 v3v3 v4v4 u1u1 u2u2 u3u3 u4u4 v1v1 v2v2 v3v3 v4v4 Step 1: construct a matrix with one variable for each edge. x 11 x x 22 0 x 23 0 x 32 0 x 33 x 42 0 x 43 x 41

Edmonds Matrix u1u1 u2u2 u3u3 u4u4 v1v1 v2v2 v3v3 v4v4 u1u1 u2u2 u3u3 u4u4 v1v1 v2v2 v3v3 v4v4 Step 2: consider the determinant of the matrix. x 11 x x 22 0 x 23 0 x 32 0 x 33 x 42 0 x 43 x 41 [Edmonds] The determinant is non-zero if and only if there is a perfect matching

Edmonds Matrix u1u1 u2u2 u3u3 u4u4 v1v1 v2v2 v3v3 v4v4 u1u1 u2u2 u3u3 u4u4 v1v1 v2v2 v3v3 v4v4 Step 2: consider the determinant of the matrix. x 11 x x 22 0 x 23 0 x 32 0 x 33 x 42 0 x 43 x 41 [Edmonds] The determinant is non-zero if and only if there is a perfect matching

Complexity u1u1 u2u2 u3u3 u4u4 v1v1 v2v2 v3v3 v4v4 x 11 x x 22 0 x 23 0 x 32 0 x 33 x 42 0 x 43 x 41 There is no way to compute this determinant efficiently… [Edmonds] The determinant is non-zero if and only if there is a perfect matching

Algebraic Algorithm u1u1 u2u2 u3u3 u4u4 v1v1 v2v2 v3v3 v4v4 x 11 x x 22 0 x 23 0 x 32 0 x 33 x 42 0 x 43 x 41 [Edmonds] The determinant is non-zero if and only if there is a perfect matching Step 3 [Lovasz]: pick a large field, substitute random values into the variables. u1u1 u2u2 u3u3 u4u4 v1v1 v2v2 v3v3 v4v Step 4: Compute this determinant, say yes if and only if it is nonzero.

Analysis u1u1 u2u2 u3u3 u4u4 v1v1 v2v2 v3v3 v4v If there is no perfect matching, then det(M) must be equal to zero. But if there is a perfect matching, then det(M) may be equal to zero. (due to cancellation) How to analyze the probability of failure?

Analysis u1u1 u2u2 u3u3 u4u4 v1v1 v2v2 v3v3 v4v Problem: if there is a perfect matching, then det(M) may be equal to zero. How to analyze the probability of failure? Observation: det(M) is a degree n multivariate polynomial. (low degree polynomial, not so many roots) [Schwartz Zippel] If the field size is |S|, then failure probability <= n/|S|. So if the field size is large (say n 3 ), then the error probability is very small.

Complexity [Coppersmith Winogrod 90] Determinant can be computed in O(n ) time So we can use it to test if there is a perfect matching in the graph. [Mucha Sankowski 04] [Harvey 06] A maximum matching can be constructed in O(n ) time Work also for general graph matching, and some other optimization problems.

Remarks Polynomial identify testing is an important problem that can be solved in randomized polynomial time but not known to be solvable in deterministic polynomial time. Red-Black matching: Given a bipartite graph with edges of 2 colors, red and black, decide if there is a perfect matching with exactly k red edges and n-k black edges. Open problem 1: Deterministic polynomial time algorithm for red-black matching? Open problem 2: Weighted matching in O(n ) time? Open problem 3: Fast algebraic algorithms for other optimization problems?

Concluding Remarks Fast randomized algorithms for minimum s-t cut? Fast algebraic algorithms for weighted problems? Fast randomized algorithms for maximum flow? There are still many basic open problems in this area. See one interesting recent result in the problem set.