Exponential time algorithms Algorithms and networks

Exponential time algorithms2 Today Exponential time algorithms: introduction Techniques 3-coloring 4-coloring Coloring Maximum Independent Set TSP

Exponential time algorithms3 What to do if a problem is NP-complete? Solve on special cases Heuristics and approximations Algorithms that are fast on average Good exponential time algorithms …

Exponential time algorithms4

5 Good exponential time algorithms Algorithms with a running time of c n.p(n) –c a constant –p() a polynomial –Notation: O*(c n ) Smaller c helps a lot!

Exponential time algorithms6 Important techniques Dynamic programming Branch and reduce –Measure and conquer (and design) Divide and conquer Clever enumeration Local search Inclusion-exclusion

Exponential time algorithms7 Running example Graph coloring –Several applications: scheduling, frequency assignment (usually more complex variants) –Different algorithms for small fixed number of colors (3-coloring, 4-coloring, …) and arbitrary number of colors –2-coloring is easy in O(n+m) time

Exponential time algorithms8 3-coloring O*(3 n ) is trivial Can we do this faster?

Exponential time algorithms9 3-coloring in O*(2 n ) time G is 3-colorable, if and only if there is a set of vertices S with –S is independent –G[V-S] is 2-colorable Algorithm: enumerate all sets, and test these properties (2 n tests of O(n+m) time each)

Exponential time algorithms10 3-coloring Lawler, 1976: –We may assume S is a maximal independent set –Enumerating all maximal independent sets in O*(3 n/3 ) = O*( n ) time There are O*(3 n/3 ) maximal independent sets (will be proved later.) –Thus O*( n ) time algorithm for 3-coloring Schiermeyer, 1994; O*(1.398 n ) time Beigel, Eppstein, 1995: O*( n ) time

Exponential time algorithms11 4-coloring in O*(2 n ) time Lawler, 1976 G is 4-colorable, if and only if we can partition the vertices in two sets X and Y such that G[X] and G[Y] are both 2- colorable Enumerate all partitions –For each, check both halves in O(n+m) time

Exponential time algorithms12 4-coloring Using 3-coloring –Enumerate all maximal independent sets S –For each, check 3-colorability of G[V-S] – n * n = n Better: there is always a color with at least n/4 vertices –Enumerate all m.i.s. S with at least n/4 vertices –For each, check 3-colorability of G[V-S] – n * n/4 = n Byskov, 2004: O*( n ) time

Exponential time algorithms13 Coloring Arbitrary number of colors With polynomial memory: Björklund, Husfeld, 2005 Divide and conquer approach Set some constant r, 0 < r < 1 There is a color with at least rn vertices, or we can divide the colors in two groups, such that for each group, there are between (1- r)/2 n and (1+r)/2 n vertices with a color in the group

Exponential time algorithms14 Algorithm Enumerate all sets S –If S is a m.i.s. with at least rn vertices, compute chromatic number of G[V-S] recursively; add 1 –If (1-r)/2 n  |S|  (1+r)/2 n then recursively compute chromatic number of G[S] and recursively compute chromatic number of G[V- S], and add these numbers Output the best over all |S|.

Exponential time algorithms15 Setting r Using Maple or Mathematica for finding the value of r that gives the best solution r= gives an algorithm using O*(5.283 n ) time –Analysis of number of m.i.s. of certain sizes, etc. Other technique (2005): O*( n )

Exponential time algorithms16 Coloring with dynamic programming Lawler, 1976: using DP for solving graph coloring.  (G) = min S is m.i.s. in G 1+X(G[V-S]) Tabulate chromatic number of G[W] over all subsets W –In increasing size –Using formula above –2 n * n = n

Exponential time algorithms17 Coloring Lawler 1976: n (improved analysis) Eppstein, 2003: n Byskov, 2004: n –All using O*(2 n ) memory –Improvements on DP method Björklund, Husfeld, 2005: n 2006 ….

Exponential time algorithms18 Inclusion-exclusion Björklund and Husfeld, 2006, and independently Koivisto, 2006 O*(2 n ) time algorithm for coloring Expression: number of ways to cover all vertices with k independent sets

Exponential time algorithms19 First formula Let c k (G) be the number of ways we can cover all vertices in G with k independent sets, where the stable sets may be overlapping, or even the same –Sequences (V 1,…,V k ) with the union of the V i ’s = V, and each V i independent Lemma: G is k-colorable, if and only if c k (G) > 0

Exponential time algorithms20 Counting independent sets Let s(X) be the number of independent sets that do not intersect X, i.e., the number of independent sets in G(V-X). We can compute all values s(X) in O*(2 n ) time. –s(X) = s(X  {v}) + s(X  {v}  N(v)) for v  X Count IS’s with v and IS’s without v –Now use DP and store all values –Polynomial space slower algorithm also possible, by computing s(X) each time again

Exponential time algorithms21 Expressing c k in s s(X) k counts the number of ways to pick k independent sets from V-X If a pick covers all vertices, it is counted in s(  ) If a pick does not cover all vertices, suppose it covers all vertices in V-Y, then it is counted in all X that are a subset in Y –With a +1 if X is even, and a -1 if X is odd –Y has equally many even as odd subsets: total contribution is 0

Exponential time algorithms22 Explanations Consider the number of k-tuples (W(1), …, W(k)) with each W(i) an independent set in G If we count all these k-tuples, we count all colourings, but also some wrong k-tuples: those which avoid some vertices So, subtract from this number all k-tuples of independent sets that avoid a vertex v, for all v However, we now subtract too many, as k-tuples that avoid two or more vertices are subtracted twice So, add for all pairs {v,w}, the number of k-tuples that avoid both v and w But, then what happens to k-tuples that avoid 3 vertices??? Continue, and note that the parity tells if we add or subtract… This gives the formula of the previous slide

Exponential time algorithms23 The algorithm Tabulate all s(X) Compute values c k (G) with the formula Take the smallest k for which c k (G) > 0

Exponential time algorithms24 Maximum independent set Branch and reduce algorithm (folklore) Uses: –Branching rule –Reduction rules

Exponential time algorithms25 Two simple reduction rules Reduction rule 1: if v has degree 0, put v in the solution set and recurse on G-v Reduction rule 2: if v has degree 1, then put v in the solution set. Suppose v has neighbor w. Recurse on G – {v,w}. –If v has degree 1, then there is always a maximum independent set containing v

Exponential time algorithms26 Idea for branching rule Consider some vertex v with neighbors w 1, w 2, …, w d. Suppose S is a maximum independent set. One of the following cases must hold: 1.v  S. Then w 1, w 2, …, w d are not in S. 2.For some i, 1  i  d, v, w 1, w 2, …, w i-1 are not in S and w i  S. Also, no neighbor of w i is in S.

Exponential time algorithms27 Branching rule Take a vertex v of minimum degree. Suppose v has neighbors w 1, w 2, …, w d. Set best to 1 + what we get when we recurse on G – {v, w 1, w 2, …, w d }. (Here we put v in the solution set.) For i = 1 to d do –Recurse on G – {v, w 1, w 2, …, w i } – N(w i ). Say, it gives a solution of value x. (N(w i ) is set of neighbors of w i. Here we put w i in S.) –Set best = max (best, x+1). Return best Using some bookkeeping gives the corresponding set S

Exponential time algorithms28 Analysis Say T(n) is the number of leaves in the search tree when we have a graph with n vertices. If v has degree d, then we have T(n)  (d+1) T(n- d-1). –Note that each w i has degree d as v had minimum degree, so we always recurse on a graph with at least d- 1 fewer vertices. d > 1 (because of reduction rules). With induction: T(n)  3 n/3. Total time is O*( 3 n/3 ) = O*( n ).

Exponential time algorithms29 Number of maximal independent sets Suppose M(n) is maximum number of m.i.s.’s in graph with n vertices Choose v of minimum degree d. If v has degree 0: number of m.i.s.’s is at most 1* M(n) If v has degree 1: number of m.i.s.’s is at most 2* M(n-2) If v has degree d>1: number of m.i.s’s is at most (d+1)* M(n – d – 1) M(n)  3 n/3 with induction

Exponential time algorithms30 Some remarks Can be done without reduction step Bound on number of m.i.s.’s sharp: consider a collection of triangles

Exponential time algorithms31 A faster algorithm Reduction rule 3: if all vertices of G have degree at most two, solve problem directly. (Easy in O(n+m) time.) New branching rule: –Take vertex v of maximum degree –Take best of two recursive steps: v not in solution: recurse of G – {v} v in solution: recurse on G – {v} – N(v); add 1.

Exponential time algorithms32 Analysis Time on graph with n vertices T(n). We have T(n)  T(n – 1) + T(n – 4) + O(n+m) –As v has degree at least 3, we loose in the second case at least 4 vertices Induction: T(n) = O*( n ) –Solve (with e.g., Maple or Mathematica) x 4 = x 3 + 1

Exponential time algorithms33 Maximum Independent Set Final remarks More detailed analysis gives better bounds Current best known: O( n ) (Robson, 2001) –Extensive, computer generated case analysis! –Includes memorization (DP) 2005: Fomin, Grandoni, Kratsch: the measure and conquer technique for better analysis of branch and reduce algorithms –Much simpler and only slightly slower compared to Robson

Exponential time algorithms34 Analysis More complicated measure (measure and conquer) Very simplified example here: Suppose algorithm costs p(n)* a q * b r time if we have a graph with q vertices of degree three and r vertices with degree at least four Suppose a  b Case analysis: …

Exponential time algorithms35 If largest degree is five or more If we do not select, r decreases by 1 If we select, r decreases by at least 6 For T(q,r)  p(n)* a q * b r to hold as induction hypothesis, we want: –a q * b r  a q * b r-1 + a q * b r-6 or –b 6  b 5 + 1

Exponential time algorithms36 If largest degree is four (1) If we do not select the vertex, we loose one vertex of degree four, and for each of its neighbors, we get a vertex that gets a degree one smaller –If the degree was 2, it has degree one, and is removed, and then its neighbor gets a degree one smaller –So, 4 vertices of degree 3 or 4 get a degree one smaller –If all neighbors have degree 4, then we get r decreases by 5, and q increases by 3 –Other cases similar

Exponential time algorithms37 If largest degree is four (2) If we select the vertex, then it, and its neighbors will be removed. The possible case is that these form a clique, and no other vertices are affected. –Here you see a possibility to improve the algorithm… –If all neighbors have degree four, then we get that we want that a q * b r  a q+3 * b r-4 + a q * b r-4 which gives b 4  a 3 +1 –Other cases …

Exponential time algorithms38 If largest degree is 3 Exercise set 8, and similar to previous cases: We get in both cases that q drops by four, and r stays the same (namely, 0). As condition, we get that a 4  2

Exponential time algorithms39 Analysis Now, solve the set of equations Example: with Excel We get here as optimum n

Exponential time algorithms40 Polynomial space algorithm for coloring Using the current best polynomial space algorithm to compute s(X) for each X, we can get an algorithm that uses O*( n ) time and polynomial space for coloring

Exponential time algorithms41 Remarks on measure and conquer More complicated manner to count number of subproblems For each degree d, we have a cost c d, e.g., vertices with degree 2 cost 0.7 (c 2 = 0.7), etc. To an instance, associate the sum over all vertices of their cost (depending on their degree) From the rules, get a recurrence on these values, and bound it To select the costs that give the best bounds, a complicated problem has to be solved (quadratic programming) 2006, van Rooij: design by measure and conquer

Exponential time algorithms42 Dominating set Now: exact algorithm for Dominating Set –Given: Graph G=(V,E) –Question: find a minimum size set S  V, such that for all v  V: v  S or v has a neighbor in S Translate to Set Cover: –Given: Collection Z of subsets of V –Question: find a minimum subcollection such that each v  V is in at least one set of the subcollection Note: Z is set of sets

Exponential time algorithms43 Reduction rules for set cover: 1 If S  R, S and R both in Z, then there is an optimal solution without S –Replace S by R and we cover V with equally many sets Rule 1: If S  R, S and R both in Z, remove S from Z

Exponential time algorithms44 Reduction rules for set cover: 2 If, for v  V, there is only one set in the collection Z that contains v, say S, then any set cover must include S Rule 2: If, for v  V, there is only one set in Z that contains v, say S, then –Make a new instance, where we set all R  Z to R-S ={w  R | w  S} –Solve new instance recursively, and add 1

Exponential time algorithms45 All sets of size 2 If all sets have size at most 2, then the problem can be solved in polynomial time –After rules 1 and 2 cannot be applied, all sets have size 2 –The problem can be solved by maximum matching A maximum matching gives the largest number of sets that cover two vertices each All vertices not in the matching cost 1 entire set

Exponential time algorithms46 Algorithm while rule 1 or rule 2 can be applied do –Apply rule 1 or 2 if all sets have size at most 2 –Solve problem exactly by matching else –Take a set S from collection of maximum size –Recurse: put S in the solution (remove S from Z, set each R  Z to R-S, add 1 to result) –Recurse: remove S from Z (S not in solution) –Output the best of these two

Exponential time algorithms47 Simple analysis (1) Take as size: |V| + |Z| Size starts at 2n Rules 1 and 2, and all sets of degree 2 cost polynomial time Branching: two subproblems with sizes decreased by 1 and by at least 4 Let T(n) be the time when size is n T(n)  T(n – 1) + T(n – 4)

Exponential time algorithms48 Solving T(n) = T(n – 1) + T(n – 4) c n = c n-1 + c n-4 Divide by c n-4 c 4 = c has a unique positive solution: namely c = … Use e.g., Mathematica, Maple or Excel So T(n)  n Algorithm for Dominating set uses at most O*( n )  O*( n ) time

Exponential time algorithms49 More complicated cost measure (1) Idea: smaller sets are nicer than larger sets –Closer to reduction rules –Similarly: elements that appear in few sets are better than elements that appear in many sets Frequency of element v: number of sets R  Z with v  R Write n i = number of sets in Z with size i Write m i = number of elements with frequency i

Exponential time algorithms50 More complicated cost measure (2) For some values v i and w j, that will be chosen in a clever way later, we take as cost measure The values will be between 0 and 1, and increasing (larger is worse) Elements of frequency 1 and sets of size 1 are all removed by rules 1 and 2 Set v 1 = w 1 = 0

Exponential time algorithms51 Analysis If the largest set S has size k, then –Branch 1: choosing S in the solution, reduces n k by 1and k vertices are removed We gain w k for removing S We gain k times a value v j (different js depending on frequency of vertices in S, but each j>1) Each v in S with frequency j means that j sets will have frequency one smaller –Branch 2: not choosing S in the solution, reduces n k by 1, and k vertices have frequency 1 smaller We gain w k for removing S If one of these vertices had frequency 2, then it gets now frequency 1 and will be removed

Exponential time algorithms52 Details, details Different cases for frequencies and sizes give a large collection of recurrences To simplify, all values for 6 and larger are set to 1 (appears not to contribute anyhow to better analysis) For a choice of v i s and w j s we get a collection of recurrences A program in Mathematica or Maple (or for simpler instances Excell) can give the values where these recurrences give the largest value

Exponential time algorithms53 (Near) Optimal values w 2 = w 3 = w 4 = w 5 = w k = 1 if k > 5 v 2 = v 3 = v 4 = v 5 = v k = 1 if k > 5

Exponential time algorithms54 And then From the recurrences, we see that a cost n instance solves at most n subproblems Initial cost is at most 2n (n elements, and n sets, each of cost at most 1) So, total time of this algorithm for Dominating Set is bounded by O*( n ) = O*( n )

Exponential time algorithms55 Improvement van Rooij, 2006 Design by measure and conquer method: –Analyze which of the recurrences is sharp –Look at that case, and try to build a new rule that reduces that case For DS: one addition rule gives O*( n ) and, using DP and storing some subproblems O*( n )

Exponential time algorithms56 Held-Karp algorithm for TSP O*(2 n ) algorithm for TSP Uses Dynamic programming Take some starting vertex s For set of vertices R (s  R), vertex w  R, let –B(R,w) = minimum length of a path, that Starts in s Visits all vertices in R (and no other vertices) Ends in w

Exponential time algorithms57 TSP: Recursive formulation B({s},s) = 0 If |S| > 1, then –B(S,w) = min v  S – {w} B(S-{w}, v} If we have all B(V,v) then we can solve TSP. Gives requested algorithm using DP- techniques.

Exponential time algorithms58

Exponential time algorithms59 Final remarks Techniques for designing exponential time algorithms Other techniques, e.g., local search Combination of techniques Several interesting open problems