Approximation Algorithms Greedy Strategies

I hear, I forget. I learn, I remember. I do, I understand! Max and Min min f is equivalent to max –f. However, a good approximation for min f may not be a good approximation for max –f. For example, consider a graph G=(V,E). C is a minimum vertex cover of G iff V \ C is a maximum independent set of G. The minimum vertex cover has a polynomial-time 2-approximation, but the maximum independent set has no constant-bounded approximation unless NP=P. Another example: Minimum Connected Dominating Set and Minimum Spanning Tree with Maximum Number of Leaves I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Greedy for Max and Min Max --- independent system Min --- submodular potential function I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Independent System I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Independent System Consider a set E and a collection C of subsets of E. (E,C) is called an independent system if The elements of C are called independent sets I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Maximization Problem I hear, I forget. I learn, I remember. I do, I understand!

Greedy Approximation MAX I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Theorem I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Proof I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

Maximum Weight Hamiltonian Cycle Given an edge-weighted complete graph, find a Hamiltonian cycle with maximum total weight. I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Independent sets E = {all edges} A subset of edges is independent if it is a Hamiltonian cycle or a vertex-disjoint union of paths. C = a collection of such subsets I hear, I forget. I learn, I remember. I do, I understand!

Maximal Independent Sets Consider a subset F of edges. For any two maximal independent sets I and J of F, |J| < 2|I| I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Theorem: For the maximum Hamiltonian cycle problem, the greedy algorithm MAX produces a polynomial time approximation with performance ratio at most 2. I hear, I forget. I learn, I remember. I do, I understand!

Maximum Weight Directed Hamiltonian Path Given an edge-weighted complete digraph, find a Hamiltonian path with maximum total weight. I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Independent sets E = {all edges} A subset of edges is independent if it is a vertex-disjoint union of directed paths. I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Tightness The rest of all edges have a cost ε ε 1 1 1 1+ε I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! A Special Case If c satisfies the following quadrilateral condition: For any 4 vertices u, v, u’, v’ in V, Then the greedy approximation for maximum weight Hamiltonian path has the performance ratio 2. I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! simple path I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Superstring Given n strings s1, s2, …, sn, find a shortest string s containing all s1, s2, …, sn as substrings. No si is a substring of another sj. I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! An Example Given S = {abcc, efaab, bccef} Some possible solutions: Concatenate all substrings = abccefaabbccef (14 chars) A shortest superstring is abccefaab (9 chars) I hear, I forget. I learn, I remember. I do, I understand!

Relationship to Set Cover? How to “transform” the shortest superstring (SS) to the Set Cover (SC) problem? Need to identify U Need to identify S Need to define the cost function The SC instance is an SS instance Let U = S (a set of n strings). How to define S ? I hear, I forget. I learn, I remember. I do, I understand!

Relationship to SC (cont) | | | | Let M be the set that consists of the strings σijk k si sj σijk I hear, I forget. I learn, I remember. I do, I understand!

Relationship to SC (cont) Now, define S Define cost of Let C is the set cover of this constructed SC, then the concatenation of all strings in C is a solution of SS. Note that C is a collection of I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Algorithm 1 for SS I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Approximation Ratio Lemma 1: Let opt be length of the optimal solution of SS and opt’ be the cost of the optimal solution of SC, we have: opt ≤ opt’ ≤ 2opt Proof: I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Proof of Lemma 1 (cont) I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Approximation Ratio Theorem1: Algorithm 1 has an approximation ratio within a factor of 2Hn Proof: We know that the approximation ratio of Set Cover is Hn. From Lemma 1, it follows directly that Algorithm 1 is a 2Hn factor algorithm for SS I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Prefix and Overlap For two string s1 and s2, we have: Overlap(s1,s2) = the maximum between the suffix of s1 and the prefix of s2. pref(s1,s2) = the prefix of s1 that remains after chopping off overlap(s1,s2) Example: s1 = abcbcaa and s2 = bcaaca, then overlap(s1,s2) = bcaa pref(s1,s2) = abc Note: overlap(s1,s2) ≠ overlap(s2, s1) I hear, I forget. I learn, I remember. I do, I understand!

Is there any better approach? Now, suppose that in the optimal solution, the strings appear from the left to right in the order: s1, s2, …, sn Define: opt = |pref(s1,s2)| + …+|pref(sn-1,sn)| + |pref(sn,s1)| + |overlap(sn,s1)| Why overlap(sn,s1)? Consider this example S={agagag, gagaga}. If we just consider the prefix only, the result would be ag whereas the correct result is agagaga I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Prefix Graph Define the prefix graph as follows: Complete weighted directed graph G=(V,E) V is a set of vertices, labeled from 1 to n (each vertex represents each string si) For each edge i→j, i ≠ j, assign a weight of |pref(si, sj)| Example: S={abc, bcd, dab} 1( ) ( )3 2( ) I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Cycle Cover Cycle Cover: a collection of disjoint cycles covering all vertices (each vertex is in exactly one cycle) Note that the tour 1 → 2 → … → n → 1 is a cycle cover Minimum weight cycle cover: sum of weights is minimum over all covers Thus, we want to find a minimum weight cycle cover I hear, I forget. I learn, I remember. I do, I understand!

How to find a min. weight cycle cover Corresponding to the prefix graph, construct a bipartite graph H=(X,Y;E) such that: X = {x1, x2, …, xn} and Y = {y1, y2, …, yn} For each i, j (in 1…n), add edge (xi, yj) of weight |pref(si,sj)| Each cycle cover of the prefix graph ↔ a perfect matching of the same weight in H. (Perfect matching is a matching which covers all the vertices) Finding a minimum weight cycle cover = finding a minimum weight perfect matching (which can be found in poly-time) I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! How to break the cycle s11 s12 s13 I hear, I forget. I learn, I remember. I do, I understand!

A constant factor algorithm I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Approximation Ratio Lemma 2: Let C be the minimum weight cycle cover of S. Let c and c’ be two cycles in C, and let r, r’ be representative strings from these cycles. Then |overlap(r, r’)| < w(c) + w(c’) Proof: Exercise I hear, I forget. I learn, I remember. I do, I understand!

Approximation Ratio (cont) Theorem 2: Algorithm 2 has an approximation ratio of 4. Proof: (see next slide) I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Proof I hear, I forget. I learn, I remember. I do, I understand!

Modification to 3-Approximation I hear, I forget. I learn, I remember. I do, I understand!

3-Approximation Algorithm I hear, I forget. I learn, I remember. I do, I understand!

Superstring via Hamiltonian path |ov(u,v)| = max{|w| | there exist x and y such that u=xw and v=wy} Overlap graph G is a complete digraph: V = {s1, s2, …, sn} |ov(u,v)| is edge weight. Suppose s* is the shortest supper string. Let s1, …, sn be the strings in the order of appearance from left to right. Then si, si+1 must have maximum overlap in s*. Hence s1, …, sn form a directed Hamiltonian path in G. I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

The Algorithm (via Hamiltonian) I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! A special property u’ v u v’ I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Theorem The Greedy approximation MAX for maximum Hamiltonian path in overlapping graph has performance ratio 2. Conjecture: This greedy approximation also give the minimum superstring an approximation solution within a factor of 2 from optimal. Example: S={abk, bk+1, bka}. s* = abk+1a. Our obtained solution: abkabk+1. I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Submodular Function I hear, I forget. I learn, I remember. I do, I understand!

What is a submodular function? Consider a finite set E, (called ground set), and a function f : 2E →Z. The function f is said to be submodular if for any two subsets A and B in 2E : Example: f(A) = |A| is submodular. I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Set-Cover Given a collection C of subsets of a set E, find a minimum subcollection C’ of C such that every element of E appears in a subset in C’ . I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Greedy Algorithm Return C’ Here: f(C) = # of elements in C Basically, the algorithm pick up the set that cover the most uncovered elements at each step I hear, I forget. I learn, I remember. I do, I understand!

Analyze the Approximation Ratio I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Alternative Analysis I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! What do we need? I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! What’s we need? I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Actually, this inequality holds if and only if f is submodular and (monotone increasing) I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Proof I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Proof of (1) I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Proof of (2) I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Theorem Greedy Algorithm produces an approximation within ln n +1 from optimal for the set cover problem The same result holds for weighted set-cover. I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Weighted Set Cover Given a collection C of subsets of a set E and a weight function w on C, find a minimum total-weight subcollection C’ of C such that every element of E appears in a subset in C’ . I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Greedy Algorithm I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! A General Problem I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Greedy Algorithm I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! A General Theorem Remark (Normalized): I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Proof I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Proof (cont) We will prove these following claims: I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Show the First Claim I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Show the Second Claim For any integers p > q > 0, we have: $(p – q)/p = \sum_{j=q+1}^p 1/p \le \sum_{j=q+1}^p 1/j$ I hear, I forget. I learn, I remember. I do, I understand!

Connected Vertex-Cover Given a connected graph, find a minimum vertex-cover which induces a connected subgraph. I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! For any vertex subset A, p(A) is the number of edges not covered by A. For any vertex subset A, q(A) is the number of connected component of the subgraph induced by A. -p is submodular. -q is not submodular. Note that when A is a connected vertex cover, the q(A) = 1 and p(A) = 0. I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! -p-q Define f(A)= -p(A) –q(A). Then f(A) is submodular and monotone increasing I hear, I forget. I learn, I remember. I do, I understand!

I hear, I forget. I learn, I remember. I do, I understand! Theorem Connected Vertex-Cover has a (1+ln Δ)-approximation where Δ is the maximum degree. -p(Ø)=-|E|, -q(Ø)=0. |E|-p(x)-q(x) < Δ-1 I hear, I forget. I learn, I remember. I do, I understand!

Weighted Connected Vertex-Cover Given a vertex-weighted connected graph, find a connected vertex-cover with minimum total weight. Theorem Weighted Connected Vertex-Cover has a (1+ln Δ)-approximation. I hear, I forget. I learn, I remember. I do, I understand!