Young CS 530 Ad. Algo. D&A NP-Completeness1 NP-Completeness Reference: Computers and Intractability: A Guide to the Theory of NP-Completeness by Garey and Johnson, W.H. Freeman and Company, 1979.

Young CS 530 Ad. Algo. D&A NP-Completeness2 General Problems, Input Size and Time Complexity Time complexity of algorithms : polynomial time algorithm ("efficient algorithm") v.s. exponential time algorithm ("inefficient algorithm") f(n) \ n n sec sec sec n5n5 0.1 sec24.3 sec5.2 mins 2n2n sec17.9 mins35.7 yrs

Young CS 530 Ad. Algo. D&A NP-Completeness3 “Hard” and “easy’ Problems Sometimes the dividing line between “easy” and “hard” problems is a fine one. For example –Find the shortest path in a graph from X to Y.(easy) –Find the longest path in a graph from X to Y.(with no cycles) (hard) View another way – as “yes/no” problems –Is there a simple path from X to Y with weight <= M? (easy) –Is there a simple path from X to Y with weight >= M? (hard) First problem can be solved in polynomial time. All known algorithms for the second problem (could) take exponential time.

Young CS 530 Ad. Algo. D&A NP-Completeness4 Decision problem: The solution to the problem is "yes" or "no". Most optimization problems can be phrased as decision problems (still have the same time complexity). Example : Assume we have a decision algorithm X for 0/1 Knapsack problem with capacity M, i.e. Algorithm X returns “Yes” or “No” to the question “is there a solution with profit  P subject to knapsack capacity  M?”

Young CS 530 Ad. Algo. D&A NP-Completeness5 We can repeatedly run algorithm X for various profits(P values) to find an optimal solution. Example : Use binary search to get the optimal profit, maximum of lg  p i runs. (where M is the capacity of the knapsack optimization problem) Min BoundOptimal Profit Max Bound 0  p i |___________________|_________________| Search for the optimal solution

Young CS 530 Ad. Algo. D&A NP-Completeness6 The Classes of P and NP The class P and Deterministic Turing Machine Given a decision problem X, if there is a polynomial time Deterministic Turing Machine program that solves X, then X is belong to P Informally, there is a polynomial time algorithm to solve the problem

Young CS 530 Ad. Algo. D&A NP-Completeness7 The class NP and Non-deterministic Turing Machine Given a decision problem X, if there is a polynomial time Non-deterministic Turing machine program that solves X, then X belongs to NP Given a decision problem X. For every instance I of X, (a) guess solution S for I, and (b) check “is S a solution to I?”. If (a) and (b) can be done in polynomial time, then X belongs to NP.

Young CS 530 Ad. Algo. D&A NP-Completeness8 Obvious : P  NP, i.e. A problem in P does not need “guess solution”. The correct solution can be computed in polynomial time. Some problems which are in NP, but may not in P : 0/1 Knapsack Problem PARTITION Problem : Given a finite set of positive integers Z. Question : Is there a subset Z' of Z such that Sum of all numbers in Z' = Sum of all numbers in Z-Z' ? i.e.  Z' =  (Z-Z') NP P

Young CS 530 Ad. Algo. D&A NP-Completeness9 One of the most important open problem in theoretical compute science : Is P=NP ? Most likely “No”. Currently, there are many known problems in NP, and there is no solution to show anyone of them in P.

Young CS 530 Ad. Algo. D&A NP-Completeness10 NP-Complete Problems Stephen Cook introduced the notion of NP- Complete Problems. This makes the problem “P = NP ?” much more interesting to study. The following are several important things presented by Cook :

Young CS 530 Ad. Algo. D&A NP-Completeness11 1.Polynomial Transformation ("  ") L1  L2 : There is a polynomial time transformation that transforms arbitrary instance of L1 to some instance of L2. If L1  L2 then L2 is in P implies L1 is in P (or L1 is not in P implies L2 is not in P) If L1  L2 and L2  L3 then L1  L3

Young CS 530 Ad. Algo. D&A NP-Completeness12 2.Focus on the class of NP – decision problems only. Many intractable problems, when phrased as decision problems, belong to this class. 3.L is NP-Complete if L  NP and for all other L'  NP, L'  L If a problem in NP-complete can be solved in polynomial time then all problems in NP can be solved in polynomial time. If a problem in NP cannot be solved in polynomial time then all problems in NP-complete cannot be solved in polynomial time. Note that an NP-complete problem is one of those hardest problems in NP. NP L

Young CS 530 Ad. Algo. D&A NP-Completeness13 So, if an NP-complete problem is in P then P=NP if P != NP then all NP-complete problems are in NP-P Question : how can we obtain the first NP-complete problem L? 4.Cook Theorem : SATISFIABILITY is NP- Complete. (The first NP-Complete problem) Instance : Given a set of variables, U, and a collection of clauses, C, over U. Question : Is there a truth assignment for U that satisfies all clauses in C?

Young CS 530 Ad. Algo. D&A NP-Completeness14 Example : U = {x 1, x 2 } C 1 = {(x 1, ¬ x 2 ), (¬ x 1, x 2 )} = (x 1 OR ¬ x 2 ) AND (¬ x 1 OR x 2 ) if x 1 = x 2 = True  C 1 = True C 2 = (x 1, x 2 ) (x 1, ¬ x 2 ) (¬ x 1 )  not satisfiable “¬ x i ” = “not x i ” “OR” = “logical or ” “AND” = “logical and ” This problem is also called “CNF-Satisfiability” since the expression is in CNF – Conjunctive Normal Form (the product of sums).

Young CS 530 Ad. Algo. D&A NP-Completeness15 With the Cook Theorem, we have the following property : Lemma : If L1 and L2 belong to NP, L1 is NP-complete, and L1  L2 then L2 is NP- complete. i.e. L1, L2  NP and for all other L'  NP, L'  L1 and L1  L2  L'  L2

Young CS 530 Ad. Algo. D&A NP-Completeness16 So now, to prove a problem L to be NP- complete problem, we need to show L is in NP select a known NP-complete problem L' construct a polynomial time transformation f from L' to L prove the correctness of f (i.e. L’ has a solution if and only if L has a solution) and that f is a polynomial transformation

Young CS 530 Ad. Algo. D&A NP-Completeness17 P:Problems solvable by deterministic algorithms in polynomial time NP: Problems solved by non-deterministic algorithms in polynomial time A group of problems, including all of the ones we have discussed (Satisfiability, 0/1 Knapsack, Longest Path, Partition) have an additional important property: If any of them can be solved in polynomial time, then they all can! NP P NP-Complete These problems are called NP-complete problems.

Young CS 530 Ad. Algo. D&A NP-Completeness18 Some NP-complete problems : SATISFIABILITY 0/1 Knapsack PARTITION Two-Processor Non-Preemptive Schedule Length CLIQUE : An undirected graph G=(V, E) and a positive integer J  |V| Question : Does G contain a clique (complete subgraph) of size J or more?

Young CS 530 Ad. Algo. D&A NP-Completeness19 Proving NP-Completeness Results Example 1 : Show that the PARTITION problem is NP-complete. Given a known NPC problem - Sum of Subset Problem (SS), show that PARTITION problem is NPC. SS Problem Instance : Let A = {a 1, a 2, …, a n } be a set of n positive numbers. Question : Given M, is there a subset A'  A such that  A' = M

Young CS 530 Ad. Algo. D&A NP-Completeness20 PARTITION Problem Instance : Given a finite set of m positive integers Z. Question : Is there a subset Z'  Z such that  Z' =  (Z-Z') PARTITION is in NP guess a subset Z'O(m) verify  Z' =  (Z-Z')?O(m) Total O(m)

Young CS 530 Ad. Algo. D&A NP-Completeness21 SS  PARTITION Given an arbitrary instance of SS, i.e. A = {a 1, a 2, …, a n } and M, Construct an instance of PARTITION as follows : B={b 1, b 2, …, b n, b n+1, b n+2 } of m = n+2 positive numbers where b i = a i for 1  i  n b n+1 = M + 1 b n+2 =  A + (1 – M) Note :  b i = 2  A + 2. Also, the transformation can be done in polynomial time (based on input size of A & M)

Young CS 530 Ad. Algo. D&A NP-Completeness22 To show the transformation is correct : The SS problem has a solution if and only if the PARTITION problem has a solution.  If SS problem has a solution, then the PARTITION problem has a solution assume A' is the solution for SS problem then Z' = A'  {b n+2 } and Z-Z' =A-A'  {b n+1 }  Z' = M +  A + (1 – M) =  A + 1 =  (Z-Z')

Young CS 530 Ad. Algo. D&A NP-Completeness23  If the PARTITION problem has a solution then the SS problem has a solution if Z' is the solution then  Z' =  A + 1  exactly one of b n+2 or b n+1  Z'  if b n+2  Z' then Z' – { b n+2 } = A' and  A' = M if b n+1  Z', then use Z - Z' to obtain A'

Young CS 530 Ad. Algo. D&A NP-Completeness24 Example 2 : Show that the Traveling Salesman (TS) Problem is NP-complete. Given a known NPC problem - Hamiltonian Circuit (HC), show that TS problem is NPC. Hamiltonian Circuit (HC) problem Instance : Give an undirected graph G=(V, E) Question : Does G contain a Hamiltonian circuit, i.e. a sequence of all vertices in V which is a simple cycle.

Young CS 530 Ad. Algo. D&A NP-Completeness25 Traveling Salesman (TS) Problem Instance : Give an undirected complete graph G=(V, E) with distance d(i,j)  0 for each edge (i,j) for i  j and a positive integer B. Question : Is there a tour of all cities (a simple cycle with all vertices) having total distance no more than B. TS is in NP guess a tour, i.e. sequence of all vertices O(|V|) verify that it is a cycle covering all vertices and total distance  B O(|V|)

Young CS 530 Ad. Algo. D&A NP-Completeness26 HC  TS Given arbitrary instance of HC, i.e. G=(V, E). Construct an instance of TS as follows : G’ = (V, E’ ), where (u,v)  E’ for all u, v  V and u  v d(u,v) = 0 if (u,v)  E d(u,v) = 1 if (u,v)  E and B = 0 Note : The transformation can be done in polynomial time (based on input size of V and E)

Young CS 530 Ad. Algo. D&A NP-Completeness27 To show the transformation is correct : The HC problem has a solution if and only if the TS problem has a solution.  If HC problem has a solution, then TS problem has a solution Assume a is the solution for HC  It is a simple cycle which contains all vertices  Each edge (u,v) in this cycle has d(u,v) = 0  Total distance is 0  Solution for TS  If TS problem has a solution then HC problem has a solution Obvious to see.

Young CS 530 Ad. Algo. D&A NP-Completeness28 Example 3 : Show that the Vertex Cover (VC) Problem is NP-complete. Given 3SAT problem is NPC, show that VC problem is NPC. 3SAT Problem Instance : Given a set of variables U = {u 1, u 2, …, u n } and a collection of clauses C = {c 1, c 2, …, c m } over U such that | c i | = 3 for 1  i  m. Question : Is there a truth assignment for U that satisfies all clauses in C? Note : 3SAT problem is a restricted problem of SATISFIABILITY problem.

Young CS 530 Ad. Algo. D&A NP-Completeness29 Vertex Cover (VC) Problem Instance : Given an undirected graph G=(V, E) and a positive integer K  |V| Question : Is there a vertex cover of size K or less for G, i.e. a subset V’  V such that |V’|  K and, for each (u,v)  E, at least one of u or v  V’. VC is in NP guess a set of vertices V’  V O(|V|) verify that |V’|  K and, for each (u,v)  E, u  V’ or v  V’ O(|V|+|E|)

Young CS 530 Ad. Algo. D&A NP-Completeness30 3SAT  VC Given arbitrary instance of 3SAT, i.e. U = {u 1, u 2, …, u n } and C = {c 1, c 2, …, c m }. Construct an instance of VC as follows :  G = (V, E ) and K = n+2m  V = V u  V c V u = {u 1t, u 1f, u 2t, u 2f, …, u nt, u nf } and V c = {a 11, a 12, a 13 }  {a 21, a 22, a 23 }  …  {a m1, a m2, a m3 }

Young CS 530 Ad. Algo. D&A NP-Completeness31  E = E u  E c  E uc E u = {(u 1t, u 1f ), (u 2t, u 2f ), …, (u nt, u nf )} E c = {(a 11, a 12 ), (a 12, a 13 ), (a 13, a 11 )}  …  {(a m1, a m2 ), (a m2, a m3 ), (a m3, a m1 )} Assume c i = (x i, y i, z i ) for 1  i  m, find the corresponding vertices, x i, y i, z i, in V u E uc = {(x 1, a 11 ), (y 1, a 12 ), (z 1, a 13 )}  …  {(x m, a m1 ), (y m, a m2 ), (z m, a m3 )}

Young CS 530 Ad. Algo. D&A NP-Completeness32  |V| = 2n+3m and |E| = n+3m+3m  The transformation can be done in polynomial time (based on input size n and m) Example : U = {u 1, u 2, u 3, u 4 } and C = {{u 1, ¬ u 3, ¬ u 4 }, {u 1, u 2, ¬ u 4 }} E  u 1t u 1f u 2t u 2f u 3t u 3f u 4t u 4f a 11 a 12 a 13 a 21 a 22 a 23 EuEu EcEc E uc

Young CS 530 Ad. Algo. D&A NP-Completeness33 Major Property : if there is a vertex cover set V’  K = n+2m, then a)|V’| = n+ 2m, and b)V’ must include exactly 1 vertex in {u it, u if } for 1  i  n from V u and at exactly 2 vertices in {a i1, a i2, a i3 } for 1  i  m from V c, i.e. n vertices from V u and 2m vertices from V c  Look at edges in E u and E c, a vertex cover set V’ must include at least 1 vertex {u it, u if } for 1  i  n, and at least 2 vertices from {a i1, a i2, a i3 } for 1  i  m. Since |V’|  K  |V’| = K

Young CS 530 Ad. Algo. D&A NP-Completeness34 To show the transformation is correct : The 3SAT problem has a solution if and only if the VC problem has a solution.  If VC problem has a solution then 3SAT problem has a solution From the above property, V’ contains n vertices from V u and 2m vertices from V c From V u, the truth assignment for {u 1, u 2, …, u n } in 3SAT is u i = T if u it  V’ u i = F if u if  V’ for 1  i  n

Young CS 530 Ad. Algo. D&A NP-Completeness35 To see that this is a solution for 3SAT : we must show that for each c i = (x i, y i, z i ), there is at least one variable  i  {x i, y i, z i }which set c i to TRUE, 1  i  n From the above property, exactly 2 vertices from {a i1, a i2, a i3 } in V’, for 1  i  m only cover 2 edges from {(x i, a i1 ), (y i, a i2 ), (z i, a i3 )} from E uc assume the edge (x i, a i1 ) is not covered by 2 vertices from {a i1, a i2, c i3 } then x i  V’ since V’ is a vertex cover set x i set the clause c i to True for 1  i  n

Young CS 530 Ad. Algo. D&A NP-Completeness36  If the 3SAT problem has a solution, then the VC problem has a solution The vertex cover set V’ with exactly n+2m vertices can be obtained as follows : From the truth assignment for {u 1, u 2, …, u n } in 3SAT, we get n vertices from V u, i.e. u it  V’ if u i = T; otherwise u if  V’ for 1  i  n This covers all edges in E u and at least one edge in {(x i, a i1 ), (y i, a i2 ), (z i, a i3 )} for 1  i  m From V c, include 2 vertices from each {a i1, a i2, a i3 } into V’, for 1  i  m. These 2 vertices cover all edges {(a i1, a i2 ), (a i2, a i3 ), (a i3, a i1 )} and also cover edges in E uc that are not covered previously.

Young CS 530 Ad. Algo. D&A NP-Completeness37 Example 4 : Show that the Square Packing (SP) Problem is NP-complete. Motivation : truck loading, processor allocation problem in a partitionable mesh connected system, the design of VLSI chips and etc. Square Packing Problem Given a packing square S and a set of packed squares L = {s 1, s 2,..., s n }. Question : is there and orthogonal packing of L into S? Note : orthogonal packing  the sides of squares are parallel to the vertical and horizontal axes

Young CS 530 Ad. Algo. D&A NP-Completeness38 3-Partition Problem Given a list A = {a 1, a 2,..., a 3z } of 3z positive integers such that sum of all numbers is zB and B/4 < a i < B/2 for each 1  i  3z. Question : Can A be partitioned into z groups such that the sum of all numbers in each group is B. Note : each group must have exactly 3 numbers Proof : Refer to the following research paper Leung, Tam, Wong, Young and Chin, “Packing Squares into a Square”, Journal of Parallel and Distributed Computing, 1990.

Young CS 530 Ad. Algo. D&A NP-Completeness39 Coping with NP-Completeness Problem NP-hard Problem Note : Refer to Chapter 5 of Garey and Johnson If L'  L and L' is an NP-complete problem then L is called NP-hard problem. All NPC problems are NP-hard.

Young CS 530 Ad. Algo. D&A NP-Completeness40 There are some NP-hard decision problems that are not in NP. Example : K th Largest Subset Problem is not in NP Instance : Given a set of positive integers A = {a 1, a 2, …, a n }, and two non-negative numbers, B   A and K  2 |A|. Question : Are there at least K distinct subsets A’  A such that each subset has total sum >= B. Note : PARTITION problem  K th Largest Subset Problem

Young CS 530 Ad. Algo. D&A NP-Completeness41 Pseudo-polynomial time algorithm Note : Refer to Chapter 4 of Garey and Johnson Some NP-complete problem may be solved in "polynomial" time (based on input size and magnitude). Example : PARTITION problem

Young CS 530 Ad. Algo. D&A NP-Completeness42 Dynamic Programming Algorithm : assume B = (sum of n integers)/2 construct a table of size (approx.) n x B fill in the table row by row for each row, add a new element mark sum of all possible subsets if there is a subset with sum = B, stop. Time Complexity : O(nB)

Young CS 530 Ad. Algo. D&A NP-Completeness43 NP-completeness in the strong sense Note : Refer to Chapter 4 of Garey and Johnson If L is NP-complete problem in the strong sense, then L cannot be solved by a pseudo-polynomial time algorithm unless P=NP L is NP-complete in the strong sense if L'  L, L' is NP-complete in the strong sense and L is in NP Example : 3-partition problem is NPC in the strong sense.

Young CS 530 Ad. Algo. D&A NP-Completeness44 Solving more restricted problems If we restricted the problem L to problem L' i.e. L' is a special/restricted case of L, then we may solve L' in polynomial time. For example : PARTITION problem if we assume that each input integer a i  n, where n is the number of input integers, then the pseudo polynomial time algorithm becomes polynomial time algorithm, i.e. O(n 3 )