1. Approximation Algorithms
Greedy Strategies

2. I hear, I forget. I learn, I remember. I do, I understand!
Max and Min
min f is equivalent to max –f.
However, a good approximation for min f may not be a good approximation for max –f.
For example, consider a graph G=(V,E). C is a minimum vertex cover of G iff V \ C is a maximum independent set of G. The minimum vertex cover has a polynomial-time 2-approximation, but the maximum independent set has no constant-bounded approximation unless NP=P.
Another example: Minimum Connected Dominating Set and Minimum Spanning Tree with Maximum Number of Leaves
I hear, I forget. I learn, I remember. I do, I understand!

3. I hear, I forget. I learn, I remember. I do, I understand!
Greedy for Max and Min
Max --- independent system
Min submodular potential function
I hear, I forget. I learn, I remember. I do, I understand!

4. I hear, I forget. I learn, I remember. I do, I understand!
Independent System
I hear, I forget. I learn, I remember. I do, I understand!

5. I hear, I forget. I learn, I remember. I do, I understand!
Independent System
Consider a set E and a collection C of subsets of E. (E,C) is called an independent system if
The elements of C are called independent sets
I hear, I forget. I learn, I remember. I do, I understand!

6. I hear, I forget. I learn, I remember. I do, I understand!
Maximization Problem
I hear, I forget. I learn, I remember. I do, I understand!

7. Greedy Approximation MAX
I hear, I forget. I learn, I remember. I do, I understand!

8. I hear, I forget. I learn, I remember. I do, I understand!
Theorem
I hear, I forget. I learn, I remember. I do, I understand!

9. I hear, I forget. I learn, I remember. I do, I understand!
Proof
I hear, I forget. I learn, I remember. I do, I understand!

10. I hear, I forget. I learn, I remember. I do, I understand!

11. I hear, I forget. I learn, I remember. I do, I understand!

12. Maximum Weight Hamiltonian Cycle
Given an edge-weighted complete graph, find a Hamiltonian cycle with maximum total weight.
I hear, I forget. I learn, I remember. I do, I understand!

13. I hear, I forget. I learn, I remember. I do, I understand!
Independent sets
E = {all edges}
A subset of edges is independent if it is a Hamiltonian cycle or a vertex-disjoint union of paths.
C = a collection of such subsets
I hear, I forget. I learn, I remember. I do, I understand!

14. Maximal Independent Sets
Consider a subset F of edges. For any two maximal independent sets I and J of F,
|J| < 2|I|
I hear, I forget. I learn, I remember. I do, I understand!

15. I hear, I forget. I learn, I remember. I do, I understand!

16. I hear, I forget. I learn, I remember. I do, I understand!
Theorem: For the maximum Hamiltonian cycle problem, the greedy algorithm MAX produces a polynomial time approximation with performance ratio at most 2.
I hear, I forget. I learn, I remember. I do, I understand!

17. Maximum Weight Directed Hamiltonian Path
Given an edge-weighted complete digraph, find a Hamiltonian path with maximum total weight.
I hear, I forget. I learn, I remember. I do, I understand!

18. I hear, I forget. I learn, I remember. I do, I understand!
Independent sets
E = {all edges}
A subset of edges is independent if it is a vertex-disjoint union of directed paths.
I hear, I forget. I learn, I remember. I do, I understand!

19. I hear, I forget. I learn, I remember. I do, I understand!

20. I hear, I forget. I learn, I remember. I do, I understand!
Tightness
The rest of all edges have a cost ε ε 1 1 1
1+ε
I hear, I forget. I learn, I remember. I do, I understand!

21. I hear, I forget. I learn, I remember. I do, I understand!
A Special Case
If c satisfies the following quadrilateral condition:
For any 4 vertices u, v, u’, v’ in V,
Then the greedy approximation for maximum weight Hamiltonian path has the performance ratio 2.
I hear, I forget. I learn, I remember. I do, I understand!

22. I hear, I forget. I learn, I remember. I do, I understand!
simple path
I hear, I forget. I learn, I remember. I do, I understand!

23. I hear, I forget. I learn, I remember. I do, I understand!

24. I hear, I forget. I learn, I remember. I do, I understand!

25. I hear, I forget. I learn, I remember. I do, I understand!

26. I hear, I forget. I learn, I remember. I do, I understand!
Superstring
Given n strings s1, s2, …, sn, find a shortest string s containing all s1, s2, …, sn as substrings.
No si is a substring of another sj.
I hear, I forget. I learn, I remember. I do, I understand!

27. I hear, I forget. I learn, I remember. I do, I understand!
An Example
Given S = {abcc, efaab, bccef}
Some possible solutions:
Concatenate all substrings = abccefaabbccef (14 chars)
A shortest superstring is abccefaab (9 chars)
I hear, I forget. I learn, I remember. I do, I understand!

28. Relationship to Set Cover?
How to “transform” the shortest superstring (SS) to the Set Cover (SC) problem?
Need to identify U
Need to identify S
Need to define the cost function
The SC instance is an SS instance
Let U = S (a set of n strings).
How to define S ?
I hear, I forget. I learn, I remember. I do, I understand!

29. Relationship to SC (cont)
| |
| |
Let M be the set that consists of the strings σijk k si sj
σijk
I hear, I forget. I learn, I remember. I do, I understand!

30. Relationship to SC (cont)
Now, define S
Define cost of
Let C is the set cover of this constructed SC, then the concatenation of all strings in C is a solution of SS. Note that C is a collection of
I hear, I forget. I learn, I remember. I do, I understand!

31. I hear, I forget. I learn, I remember. I do, I understand!
Algorithm 1 for SS
I hear, I forget. I learn, I remember. I do, I understand!

32. I hear, I forget. I learn, I remember. I do, I understand!
Approximation Ratio
Lemma 1: Let opt be length of the optimal solution of SS and opt’ be the cost of the optimal solution of SC, we have: opt ≤ opt’ ≤ 2opt
Proof:
I hear, I forget. I learn, I remember. I do, I understand!

33. I hear, I forget. I learn, I remember. I do, I understand!
Proof of Lemma 1 (cont)
I hear, I forget. I learn, I remember. I do, I understand!

34. I hear, I forget. I learn, I remember. I do, I understand!
Approximation Ratio
Theorem1: Algorithm 1 has an approximation ratio within a factor of 2Hn
Proof: We know that the approximation ratio of Set Cover is Hn. From Lemma 1, it follows directly that Algorithm 1 is a 2Hn factor algorithm for SS
I hear, I forget. I learn, I remember. I do, I understand!

35. I hear, I forget. I learn, I remember. I do, I understand!
Prefix and Overlap
For two string s1 and s2, we have:
Overlap(s1,s2) = the maximum between the suffix of s1 and the prefix of s2.
pref(s1,s2) = the prefix of s1 that remains after chopping off overlap(s1,s2)
Example:
s1 = abcbcaa and s2 = bcaaca, then
overlap(s1,s2) = bcaa
pref(s1,s2) = abc
Note: overlap(s1,s2) ≠ overlap(s2, s1)
I hear, I forget. I learn, I remember. I do, I understand!

36. Is there any better approach?
Now, suppose that in the optimal solution, the strings appear from the left to right in the order: s1, s2, …, sn
Define: opt = |pref(s1,s2)| + …+|pref(sn-1,sn)| + |pref(sn,s1)| + |overlap(sn,s1)|
Why overlap(sn,s1)?
Consider this example S={agagag, gagaga}.
If we just consider the prefix only, the result would be ag whereas the correct result is agagaga
I hear, I forget. I learn, I remember. I do, I understand!

37. I hear, I forget. I learn, I remember. I do, I understand!
Prefix Graph
Define the prefix graph as follows:
Complete weighted directed graph G=(V,E)
V is a set of vertices, labeled from 1 to n (each vertex represents each string si)
For each edge i→j, i ≠ j, assign a weight of |pref(si, sj)|
Example:
S={abc, bcd, dab}
1( )
( )3
2( )
I hear, I forget. I learn, I remember. I do, I understand!

38. I hear, I forget. I learn, I remember. I do, I understand!
Cycle Cover
Cycle Cover: a collection of disjoint cycles covering all vertices (each vertex is in exactly one cycle)
Note that the tour 1 → 2 → … → n → 1 is a cycle cover
Minimum weight cycle cover: sum of weights is minimum over all covers
Thus, we want to find a minimum weight cycle cover
I hear, I forget. I learn, I remember. I do, I understand!

39. How to find a min. weight cycle cover
Corresponding to the prefix graph, construct a bipartite graph H=(X,Y;E) such that:
X = {x1, x2, …, xn} and Y = {y1, y2, …, yn}
For each i, j (in 1…n), add edge (xi, yj) of weight |pref(si,sj)|
Each cycle cover of the prefix graph ↔ a perfect matching of the same weight in H. (Perfect matching is a matching which covers all the vertices)
Finding a minimum weight cycle cover = finding a minimum weight perfect matching (which can be found in poly-time)
I hear, I forget. I learn, I remember. I do, I understand!

40. I hear, I forget. I learn, I remember. I do, I understand!
How to break the cycle
s11
s12
s13
I hear, I forget. I learn, I remember. I do, I understand!

41. A constant factor algorithm
I hear, I forget. I learn, I remember. I do, I understand!

42. I hear, I forget. I learn, I remember. I do, I understand!
Approximation Ratio
Lemma 2: Let C be the minimum weight cycle cover of S. Let c and c’ be two cycles in C, and let r, r’ be representative strings from these cycles. Then
|overlap(r, r’)| < w(c) + w(c’)
Proof: Exercise
I hear, I forget. I learn, I remember. I do, I understand!

43. Approximation Ratio (cont)
Theorem 2: Algorithm 2 has an approximation ratio of 4.
Proof: (see next slide)
I hear, I forget. I learn, I remember. I do, I understand!

44. I hear, I forget. I learn, I remember. I do, I understand!
Proof
I hear, I forget. I learn, I remember. I do, I understand!

45. Modification to 3-Approximation
I hear, I forget. I learn, I remember. I do, I understand!

46. 3-Approximation Algorithm
I hear, I forget. I learn, I remember. I do, I understand!

47. Superstring via Hamiltonian path
|ov(u,v)| = max{|w| | there exist x and y
such that u=xw and v=wy}
Overlap graph G is a complete digraph:
V = {s1, s2, …, sn}
|ov(u,v)| is edge weight.
Suppose s* is the shortest supper string. Let s1, …, sn be the strings in the order of appearance from left to right. Then si, si+1 must have maximum overlap in s*. Hence s1, …, sn form a directed Hamiltonian path in G.
I hear, I forget. I learn, I remember. I do, I understand!

48. I hear, I forget. I learn, I remember. I do, I understand!

49. I hear, I forget. I learn, I remember. I do, I understand!

50. The Algorithm (via Hamiltonian)
I hear, I forget. I learn, I remember. I do, I understand!

51. I hear, I forget. I learn, I remember. I do, I understand!
A special property u’ v u v’
I hear, I forget. I learn, I remember. I do, I understand!

52. I hear, I forget. I learn, I remember. I do, I understand!
Theorem
The Greedy approximation MAX for maximum Hamiltonian path in overlapping graph has performance ratio 2.
Conjecture: This greedy approximation also give the minimum superstring an approximation solution within a factor of 2 from optimal.
Example: S={abk, bk+1, bka}. s* = abk+1a. Our obtained solution: abkabk+1.
I hear, I forget. I learn, I remember. I do, I understand!

53. I hear, I forget. I learn, I remember. I do, I understand!
Submodular Function
I hear, I forget. I learn, I remember. I do, I understand!

54. What is a submodular function?
Consider a finite set E, (called ground set), and a function f : 2E →Z. The function f is said to be submodular if for any two subsets A and B in 2E :
Example: f(A) = |A| is submodular.
I hear, I forget. I learn, I remember. I do, I understand!

55. I hear, I forget. I learn, I remember. I do, I understand!
Set-Cover
Given a collection C of subsets of a set E, find a minimum subcollection C’ of C such that every element of E appears in a subset in C’ .
I hear, I forget. I learn, I remember. I do, I understand!

56. I hear, I forget. I learn, I remember. I do, I understand!
Greedy Algorithm
Return C’
Here: f(C) = # of elements in C
Basically, the algorithm pick up the set that cover the most uncovered elements at each step
I hear, I forget. I learn, I remember. I do, I understand!

57. Analyze the Approximation Ratio
I hear, I forget. I learn, I remember. I do, I understand!

58. I hear, I forget. I learn, I remember. I do, I understand!

59. I hear, I forget. I learn, I remember. I do, I understand!

60. I hear, I forget. I learn, I remember. I do, I understand!
Alternative Analysis
I hear, I forget. I learn, I remember. I do, I understand!

61. I hear, I forget. I learn, I remember. I do, I understand!
What do we need?
I hear, I forget. I learn, I remember. I do, I understand!

62. I hear, I forget. I learn, I remember. I do, I understand!
What’s we need?
I hear, I forget. I learn, I remember. I do, I understand!

63. I hear, I forget. I learn, I remember. I do, I understand!
Actually, this inequality holds if and only if f is submodular and (monotone increasing)
I hear, I forget. I learn, I remember. I do, I understand!

64. I hear, I forget. I learn, I remember. I do, I understand!

65. I hear, I forget. I learn, I remember. I do, I understand!
Proof
I hear, I forget. I learn, I remember. I do, I understand!

66. I hear, I forget. I learn, I remember. I do, I understand!
Proof of (1)
I hear, I forget. I learn, I remember. I do, I understand!

67. I hear, I forget. I learn, I remember. I do, I understand!
Proof of (2)
I hear, I forget. I learn, I remember. I do, I understand!

68. I hear, I forget. I learn, I remember. I do, I understand!
Theorem
Greedy Algorithm produces an approximation within ln n +1 from optimal for the set cover problem
The same result holds for weighted set-cover.
I hear, I forget. I learn, I remember. I do, I understand!

69. I hear, I forget. I learn, I remember. I do, I understand!
Weighted Set Cover
Given a collection C of subsets of a set E and a weight function w on C, find a minimum total-weight subcollection C’ of C such that every element of E appears in a subset in C’ .
I hear, I forget. I learn, I remember. I do, I understand!

70. I hear, I forget. I learn, I remember. I do, I understand!
Greedy Algorithm
I hear, I forget. I learn, I remember. I do, I understand!

71. I hear, I forget. I learn, I remember. I do, I understand!
A General Problem
I hear, I forget. I learn, I remember. I do, I understand!

72. I hear, I forget. I learn, I remember. I do, I understand!
Greedy Algorithm
I hear, I forget. I learn, I remember. I do, I understand!

73. I hear, I forget. I learn, I remember. I do, I understand!
A General Theorem
Remark (Normalized):
I hear, I forget. I learn, I remember. I do, I understand!

74. I hear, I forget. I learn, I remember. I do, I understand!
Proof
I hear, I forget. I learn, I remember. I do, I understand!

75. I hear, I forget. I learn, I remember. I do, I understand!
Proof (cont)
We will prove these following claims:
I hear, I forget. I learn, I remember. I do, I understand!

76. I hear, I forget. I learn, I remember. I do, I understand!
Show the First Claim
I hear, I forget. I learn, I remember. I do, I understand!

77. I hear, I forget. I learn, I remember. I do, I understand!

78. I hear, I forget. I learn, I remember. I do, I understand!

79. I hear, I forget. I learn, I remember. I do, I understand!
Show the Second Claim
For any integers p > q > 0, we have:
$(p – q)/p = \sum_{j=q+1}^p 1/p \le \sum_{j=q+1}^p 1/j$
I hear, I forget. I learn, I remember. I do, I understand!

80. Connected Vertex-Cover
Given a connected graph, find a minimum vertex-cover which induces a connected subgraph.
I hear, I forget. I learn, I remember. I do, I understand!

81. I hear, I forget. I learn, I remember. I do, I understand!
For any vertex subset A, p(A) is the number of edges not covered by A.
For any vertex subset A, q(A) is the number of connected component of the subgraph induced by A.
-p is submodular.
-q is not submodular.
Note that when A is a connected vertex cover, the q(A) = 1 and p(A) = 0.
I hear, I forget. I learn, I remember. I do, I understand!

82. I hear, I forget. I learn, I remember. I do, I understand!
-p-q
Define f(A)= -p(A) –q(A). Then f(A) is submodular and monotone increasing
I hear, I forget. I learn, I remember. I do, I understand!

83. I hear, I forget. I learn, I remember. I do, I understand!
Theorem
Connected Vertex-Cover has a (1+ln Δ)-approximation where Δ is the maximum degree.
-p(Ø)=-|E|, -q(Ø)=0.
|E|-p(x)-q(x) < Δ-1
I hear, I forget. I learn, I remember. I do, I understand!

84. Weighted Connected Vertex-Cover
Given a vertex-weighted connected graph,
find a connected vertex-cover with minimum
total weight.
Theorem Weighted Connected Vertex-Cover
has a (1+ln Δ)-approximation.
I hear, I forget. I learn, I remember. I do, I understand!