1. Advanced Algorithms NP-hard and NP-Complete Problems Instructor: Saeid Abrishami Ferdowsi University of Mashhad

2. Introduction NP-completeness is a form of bad news! ▫Evidence that many important problems can not be solved quickly. Knowing that they are hard lets you stop beating your head against a wall trying to solve them… ▫Use a heuristic: come up with a method for solving a reasonable fraction of the common cases. ▫Solve approximately: come up with a solution that you can prove that is close to right. ▫Use an exponential time solution: if you really have to solve the problem exactly and stop worrying about finding a better solution.

3. Introduction Problems ▫Tractable: have polynomial time algorithms, i.e., O(n k ) for some k. ▫Intractable: have no polynomial time algorithm, such as O(2 n ). NP-complete problems ▫Whose status is unknown. ▫No polynomial-time algorithm has yet been discovered for an NP-complete problem, nor has anyone yet been able to prove that no polynomial- time algorithm can exist for any one of them.

4. Nondeterministic Algorithms Deterministic Algorithm ▫The result of every operation is uniquely defined. Nondeterministic Algorithms ▫Contain operations whose outcomes are not uniquely defined but limited to specified sets of possibilities. ▫The machine executing such operations is allowed to choose any one of these outcomes subject to a termination condition.

5. Nondeterministic Algorithms Nondeterministic algorithms uses three functions: ▫Choice(S): arbitrarily chooses one of the elements of set S. ▫Failure(): signals an unsuccessful completion. ▫Success(): signals a successful completion. All of these functions are O(1). x = Choice(1,n) could result in x being assigned any one of the integers in the range [1,n]. A nondeterministic algorithm terminates unsuccessfully if and only if there exists no set of choices leading to a success signal.

6. Example: nondeterministic search 1.j := Choice(1,n); 2.if A[j] = x then {write (j); Success();} 3.write (0); Failure(); Time Complexity is O(1)

7. Example: nondeterministic sort 1.Algorithm NSort(A, n) 2.{ 3. for i:=1 to n do B[i]:= 0; 4. for i:=1 to n do 5. { 6. j:=Choice(1,n); 7. if B[j] <> 0 then Failure(); 8. B[j] = A[i]; 9. } 10. for i:=1 to n-1 do 11. if B[i] > B[i+1] then Failure(); 12. write (B[1:n]); 13. success(); 14.}

8. Nondeterministic Machine Whenever successful termination is possible, a nondeterministic machine makes a sequence of choices that is a shortest sequence leading to a successful termination. In case there is no sequence of choices leading to a successful termination, we assume that the algorithm terminates in one unit of time with output "unsuccessful computation." Since, the machine is fictitious, it is not necessary for us to concern ourselves with how the machine can make a correct choice at each step.

9. Decision vs. Optimization Problems Decision Problems ▫Any problem for which the answer is either zero or one is called a decision problem. ▫An algorithm for a decision problem is termed a decision algorithm. Optimization Problems ▫Any problem that involves the identification of an optimal (either minimum or maximum) value of a given cost function is known as an optimization problem. ▫An optimization algorithm is used to solve an optimization problem.

10. Decision vs. Optimization Problems Many optimization problems can be recast into decision problems with the property that the decision problem can be solved in polynomial time if and only if the corresponding optimization problem can. In other cases, we can at least make the statement that if the decision problem cannot be solved in polynomial time, then the optimization problem cannot either.

11. Example: Maximum Clique A maximal complete sub-graph of a graph G = (V,E) is a clique. The size of the clique is the number of vertices in it. The max clique problem is an optimization problem that has to determine the size of a largest clique in G. The corresponding decision problem is to determine whether G has a clique of size at least k for some given k.

12. Example: Maximum Clique Let DClique(G, k) be a deterministic decision algorithm for the clique decision problem. If the number of vertices in G is n, the size of a max clique in G can be found by making several applications of DClique. DClique is used once for each k, k = n, n-1, n-2,..., until the output from DClique is 1. If the time complexity of DClique is f(n), then the size of a max clique can be found in time less than or equal to n*f(n).

13. Example: Maximum Clique Also, if the size of a max clique can be determined in time g(n), then the decision problem can be solved in time g(n). Hence, the max clique problem can be solved in polynomial time if and only if the clique decision problem can be solved in polynomial time.

14. Example: 0/1 Knapsack The knapsack decision problem is to determine whether there is a 0/1 assignment of values to x i, 1 < i < n, such that  p i x i  r and  w i x i  m. The r is a given number. If the knapsack decision problem cannot be solved in deterministic polynomial time, then the optimization problem cannot either.

15. Time Complexity of Nondeterministic Machines Definition ▫The time required by a nondeterministic algorithm performing on any given input is  the minimum number of steps needed to reach a successful completion if there exists a sequence of choices leading to such a completion.  O(1) if successful completion is not possible ▫A nondeterministic algorithm is of complexity O(f(n)) if for all inputs of size n, n > n o, that result in a successful completion, the time required is at most cf(n) for some constants c and n o

16. Nondeterministic Knapsack Alg. 1.Algorithm DKP(p, w, n, m, r, x) 2.{ 3. W := 0; P := 0; 4. for i := 1 to n do 5. { 6. x[i] := Choice(0,1); 7. W := W + x[i] * w[i]; P := P + x[i] * p[i] 8. } 9. if ((W > m) or (P < r)) then Failure(); 10. else Success(); 11.} Time Complexity = O(n)

17. Nondeterministic Clique Alg. 1.Algorithm DCP(G, n, k) 2.{ 3. S :=  ; 4. for i := 1 to k do 5. { 6. t := Choice(1,n); 7. if t  S then Failure(); 8. S := S  {t} 9. } 10. for all pairs (i,j) such that i  S, j  S, and i  j do 11. if (i, j) is not an edge of G then Failure(); 12. Success(); 13.}

18. Classes P and NP P is the set of all decision problems solvable by deterministic algorithms in polynomial time. NP is the set of all decision problems solvable by nondeterministic algorithms in polynomial time. Since deterministic algorithms are just a special case of nondeterministic ones P  NP. Perhaps the most famous unsolved problem in computer science, is whether P = NP or P  NP. Most computer scientists believe that P  NP, but we do not have a proof.

19. Another Definition for NP Nondeterministic algorithms are two stage procedures: ▫Guessing stage (nondeterministic):  Generate randomly an arbitrary string that can be thought of as a candidate solution (certificate) ▫Verification stage (deterministic)  Take the certificate and the instance to the problem and returns YES if the certificate represents a solution. The class NP consists of those problems that are verifiable in polynomial time. ▫we were somehow given a certificate of a solution, then we could verify that the certificate is correct in time polynomial in the size of the input to the problem.

20. NP-complete Problems Why theoretical computer scientists believe that P  NP? ▫The class of NP-complete problems. ▫The NP-complete Problems are the hardest problems in NP. ▫if any NP-complete problem can be solved in polynomial time, then every problem in NP has a polynomial-time solution, that is, P = NP. How to prove a problem is NP-complete? ▫Reduction ▫Cook theorem: Satisfiability is NP-complete.

21. Reduction Reduction is a way of saying that one problem is easier than another. Definition: Let L 1 and L 2 be problems. Problem L 1 reduces to L 2 (also written L 1  L 2 ) if and only if there is a way to solve L 1 by a deterministic polynomial time algorithm using a deterministic algorithm that solves L 2 in polynomial time. Idea: transform the inputs of L 1 to inputs of L 2 f Problem L 2  yes no yes no Problem L 1

22. Polynomial Reductions Given two problems L 1 and L 2, we say that L 1 is polynomially reducible to L 2 (L 1  p L 2 ) if: 1.There exists a function f that converts the input of L 1 to inputs of L 2 in polynomial time 2.L 1 (i) = YES  L 2 (f(i)) = YES

23. Polynomial Reductions We use polynomial-time reductions in the opposite way to show that a problem has no polynomial time algorithm. ▫Suppose we have a decision problem A for which we already know that no polynomial-time algorithm can exist. ▫Suppose further that we have a polynomial-time reduction transforming instances of A to instances of B. ▫Now we can use a simple proof by contradiction to show that no polynomial time algorithm can exist for B.

24. Classes NP-hard and NP-complete Satisfiability Problem: Let x 1, x 2 … denote boolean variables. A literal is either a variable or its negation. A formula in the propositional calculus is an expression that can be constructed using literals and the operations and and or. ▫Example: (x 1  x 2 )  (x 3  x 4 ) The satisfiability problem is to determine whether a formula is true for some assignment of truth values to the variables. Cook Theorem: Satisfiability is in P if and only if P = NP. (See Horowitz for a proof)

25. Classes NP-hard and NP-complete A problem L is NP-hard if and only if satisfiability reduces to L (satisfiability  L). A problem L is NP-complete if and only if L is NP-hard and L  NP. Another Definition: ▫A problem B is NP-complete if: B  NP A  p B for all A  NP

26. Classes NP-hard and NP-complete

27. To show that a problem L 2 is NP-hard, it is adequate to show L 1  L2, where L 1 is some problem already known to be NP-hard. ▫Since  is a transitive relation, it follows that if satisfiability  L 1 and L 1  L 2, then satisfiability  L2. To show that an NP-hard decision problem is NP- complete, we have just to exhibit a polynomial time nondeterministic algorithm for it. Theorem: If any NP-Complete problem can be solved in polynomial time  then P = NP.

28. How to show a problem is NP-hard The strategy we adopt to show that a problem L 2 is NP-hard is: 1.Pick a problem L 1 already known to be NP-hard. 2.Show how to obtain (in polynomial deterministic time) an instance I of L 2 from any instance I of L 1 such that from the solution of I we can determine (in polynomial deterministic time) the solution to instance I of L 1. 3.Conclude from step (2) that L 1  L 2 4.Conclude from steps (1) and (3) and the transitivity of  that L 2 is NP-hard.

29. Clique Decision Problem(CDP) CNF-satisfiability ▫CNF is a special case of SAT  is in “Conjuctive Normal Form” (CNF) ▫“AND” of expressions (i.e., clauses) ▫Each clause contains only “OR”s of the variables and their complements ▫ E.g.:  = (x 1  x 2 )  (x 1   x 2 )  (  x 1   x 2 ) satidfiability  CNF-satidfiability ▫See sec. 11.2 (Horowitz)

30. Clique Decision Problem(CDP) Clique Problem: ▫Undirected graph G = (V, E) ▫Clique: a subset of vertices in V all connected to each other by edges in E (i.e., forming a complete graph) ▫Size of a clique: number of vertices it contains Optimization problem: ▫Find a clique of maximum size Decision problem: ▫Does G have a clique of size k? Clique(G, 2) = YES Clique(G, 3) = NO Clique(G, 3) = YES Clique(G, 4) = NO

31. Clique Decision Problem(CDP) Theorem: CNF-satidfiability  CDP Proof: Let be a propositional formula in CNF. Let x i, 1  i  n, be the variables in F. ▫We show how to construct from F a graph G = (V, E) such that G has a clique of size at least k if and only if F is satisfiable. ▫If the length of F is m, then G is obtainable from F in 0(m) time. ▫Hence, if we have a polynomial time algorithm for CDP, then we can obtain a polynomial time algorithm for CNF-satisfiability using this construction.

32. Clique Decision Problem(CDP) For any F, G = {V,E) is defined as follows: ▫V = { |  is a literal in clause C i } ▫E = {(, ) | i  j and  (bar)}.

33. Clique Decision Problem(CDP) Claim: F is satisfiable if and only if G has a clique of size  k. Proof: ▫If F is satisfiable, then there is a set of truth values for x i, 1  i  n, such that each clause is true with this assignment. ▫Thus, with this assignment there is at least one literal  in each C i such that  is true. ▫ Let S = { |  is true in C i } be a set containing exactly one for each i. ▫Between any two nodes and in S there is an edge in G, since i  j and both  and  have the value true. Thus, S forms a clique in G of size k.

34. Clique Decision Problem(CDP) Proof (continued): ▫Similarly, if G has a clique K = (V',E') of size at least k, then let S = { |  V'}.  |S| = k as G has no clique of size more than k.  If S' = {  |  S for some i}, then S' cannot contain both a literal  and its complement, as there is no edge connecting them in G.  Hence by setting x i =true if x i  S' and x i =false if  x i  S' and choosing arbitrary truth values for variables not in S' we can satisfy all clauses in F. ▫Hence, F is satisfiable iff G has a clique of size at least k.

35. Node Cover Decision Problem (NCDP) ▫A set S  V is a node cover for a graph G = (V, E) if and only if all edges in E are incident to at least one vertex in S. The size |S| of the cover is the number of vertices in S. Example: ▫S={2,4} is a node cover of size 2, and S={1,3,5} is a node cover of size 3.

36. Node Cover Decision Problem (NCDP) Theorem: ▫The clique decision problem  the node cover decision problem. Proof: ▫Let G = (V,E) and k define an instance of CDP. Assume that |V| = n. We construct a graph G such that G has a node cover of size at most n — k if and only if G has a clique of size at least k. ▫Graph G is the complement of G.

37. Node Cover Decision Problem (NCDP) Proof: ▫Let K be any clique in G. ▫Since there are no edges in connecting vertices in K, the remaining n — |K| vertices in G' must cover all edges in. ▫Similarly, if S is a node cover of G', then V-S must form a complete subgraph in G. Since G' can be obtained from G in polynomial time, CDP can be solved in polynomial deterministic time if we have a polynomial time deterministic algorithm for NCDP.

38. Node Cover Decision Problem (NCDP)

39. Directed Hamiltonian Cycle (DHC) A directed Hamiltonian cycle in a directed graph G = (V, E) is a directed cycle of length n = |V|. So, the cycle goes through every vertex exactly once and then returns to the starting vertex. The DHC problem is to determine whether G has a directed Hamiltonian cycle. Example: ▫1-2-3-4-5-1

40. Directed Hamiltonian Cycle (DHC) Theorem : ▫CNF-satisfiability  directed Hamiltonian cycle. Proof: ▫Let F be a propositional formula in CNF. ▫We show how to construct a directed graph G such that F is satisfiable if and only if G has a directed Hamiltonian cycle. ▫Example:

41. Directed Hamiltonian Cycle (DHC)

43. Proof (Continued): ▫If F is satisfiable, then let S be an assignment of truth values for which F is true. ▫A Hamiltonian cycle for G can start at v 1 and go to u 1, then to v 2 then to u 2, … and then to u n. ▫In going from v i to u i, this cycle uses the column corresponding to x i if x i is true in S. Otherwise it goes up the column corresponding to  x i. ▫From u n this cycle goes to A 1 and then through R 1,1, R 1,2, R 1,3, … R 1,ji, and B 1 to A 2 to … v 1.

44. Directed Hamiltonian Cycle (DHC) Proof (Continued): In going from R i,a to R i,a+1 in any subgraph, a diversion is made to a  subgraph in row i if and only if the vertices of that  subgraph are not already on the path from v i to R i,a Note that if C i has i j literals, then it allows a diversion to at most i j -1  subgraphs. At least one  subgraph must already have been traversed in C i So, if F is satisfiable, then G has a directed Hamiltonian cycle.

45. Directed Hamiltonian Cycle (DHC) Proof (continued): ▫If G has a directed Hamiltonian cycle, start at vertex v 1 on that cycle. ▫Such a cycle must proceed by going up exactly one column of each pair (x i,  x i ). ▫In addition, this part of the cycle must traverse at least one  subgraph in each row. ▫Hence the columns used in going from v i to u i, define a truth assignment for which F is true. See Coremen Book ▫Node Cover Decision Problem  DHC

46. Traveling Salesperson Decision Problem (TSP) Problem ▫Whether a complete directed graph G=(V,E) with edge costs c(u,v) has a tour of cost at most M. Theorem : ▫Directed Hamiltonian cycle  TSP Proof: ▫From the directed graph G = (V, E) construct the complete directed graph G' = (V,E'):  E' = { | i  j}  c(i,j)=1 if  E and c(i,j)=2 if i  j and  E. ▫Clearly, G' has a tour of cost at most n iff G has a directed Hamiltonian cycle.