CSC 413/513: Intro to Algorithms NP Completeness

Problems and Instances ● Problem: Sort(A) ● Problem instance: Sort([4,2,7,1,6,9]) ● Input encoding ■ Bin(4) concat Bin(2) concat Bin(7) … ■ Size: n lg x ○ n: size of A ○ lg x: max. number of bits required to represent A[i], for any i ○ We say size = (n), or simply n

NP-Completeness ● Some problems are intractable: as the input size increases, we are unable to solve them in reasonable time ● What constitutes reasonable time? Standard working definition: polynomial time ■ On an input of size n the worst-case running time is O(n k ) for some constant k ■ Polynomial time: O(n 2 ), O(n 3 ), O(1), O(n lg n) ■ Not in polynomial time: O(2 n ), O(n n ), O(n!)

Polynomial-Time Problems ● Are some problems solvable in polynomial time? ■ Of course: every algorithm we’ve studied provides polynomial-time solution to some problem ○ Except DP algorithm to 0-1 knapsack ■ We define P to be the class of problems solvable in polynomial time ○ That is, all instances of such problems are polynomially solvable (e.g., sorting) ■ These are the “easy” problems

Polynomial-Time Problems ● Are all problems solvable in polynomial time? ■ No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given ○ But we often detect infinite loops in a program by inspection, and hence “solve” the halting problem ○ However, there are instances that are not solvable, even by our brains ■ Such problems are clearly intractable, not in P ■ Similarly, there are problems that can be solved, but we haven’t found algorithms that will take any instance of the problem and solve it in polynomial time

NP-Complete Problems ● The NP-Complete problems are an interesting class of problems whose status is unknown ■ No polynomial-time algorithm has been discovered for an NP-Complete problem, for all instances of it ■ No suprapolynomial lower bound has been proved for some instance of any NP-Complete problem, either ● We call this the P = NP question ■ The biggest open problem in CS

An NP-Complete Problem: Hamiltonian Cycles ● An example of an NP-Complete problem: ■ A hamiltonian cycle of an undirected graph is a simple cycle that contains every vertex ■ The hamiltonian-cycle problem: given a graph G, does it have a hamiltonian cycle? Describe a naïve algorithm for solving the hamiltonian- cycle problem. Running time?

P and NP ● As mentioned, P is set of problems that can be solved in polynomial time ● NP (nondeterministic polynomial time) is the set of problems that can be solved in polynomial time by a nondeterministic computer ■ What the hell is that?

Nondeterminism ● Think of a non-deterministic computer as a computer that magically “guesses” a solution, and then has to verify that it is correct ■ If a solution exists, n.d. computer always guesses it ■ One way to imagine it: a parallel computer that can freely spawn an infinite number of processes ○ Have one processor work on each possible solution ○ All processors attempt to verify that their solution works, in parallel ○ If a processor finds it has a working solution ■ So: NP = problems verifiable in polynomial time ■ Note: P NP (think of verifying the output of sorting)

P and NP ● Summary so far: ■ P = problems that can be solved in polynomial time ■ NP = problems for which a solution can be verified in polynomial time ■ P NP ■ Unknown whether P = NP (most suspect not) ● Hamiltonian-cycle problem is in NP: ■ Don’t know how to solve in polynomial time ■ But, easy to verify solution in polynomial time (How?)

Intuition behind NP-Complete problems ● Many problems exhibit structure that is exploited ■ E.g., recursive structure in sorting, optimal substructure in problems solved by DP, greedy etc. ● But, some problems seem to have no exploitable structure ■ E.g., longest simple path in a graph ○ Contrast with shortest paths in graphs, which has the optimal substructure ■ Typically, these are the NP-Complete problems Does not have optimal substructure

What about 0-1 Knapsack? ● Has optimal substructure ■ But input contains a number (W), and the optimal substructure depends on the magnitude of this number, not its size ■ Algorithm turns out to be exponential in the size of W (although polynomial in the magnitude of W) ■ Its DP Algorithm is called pseudo-polynomial ● 0-1 knapsack is called weakly NP-complete

NP-Complete Problems ● We will see that NP-Complete problems are the “hardest” problems in NP: ■ If any one NP-Complete problem can be solved in polynomial time… ○ …then every NP-Complete problem can be solved in polynomial time… ○ …and in fact every problem in NP can be solved in polynomial time (which would show P = NP) ■ Thus: solve hamiltonian-cycle in O(n 100 ) time, you’ve proved that P = NP. Retire rich & famous.

A Simplification ● Henceforth, we will only talk about decision problems, with boolean answers ■ Most problems of interest can be posed as decision problems ■ E.g., SHORTEST_PATH(G,u,v) ■ Its decision problem is PATH(G,u,v,k): is there a path between u and v involving no more than k edges? ■ Can solve SHORTEST_PATH by calling PATH repeatedly with decreasing k, until the answer is “No” ● So, “solve” ≡ “decide”

How to Verify? ● How do you verify the answer to a decision problem (in polynomial time)? ● Need a certificate ■ E.g., certificate for PATH(G,u,v,k) would be the path: ● How do you verify the certificate? ■ Check length ≤ k ■ Check first vertex is u, last vertex is v ■ Check all path-edges actually exist ● How much time?

Why only Decision Problems? ● So that problems can be considered as languages ■ Then, “algorithm” ≡ Turing machine ■ Consider the binary encoding of a problem instance (i.e., the input) ■ Collect all input binary strings for which the answer/output is “Yes” ■ This set is a language ○ A machine that decides the language also solves the problem ● So, “decision problem” ≡ “language”

Reduction ● The crux of NP-Completeness is reducibility ■ Informally, a problem P can be reduced to another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of P ○ What do you suppose “easily” means? ○ This rephrasing is called transformation ■ Intuitively: If P reduces to Q, P is “no harder to solve” than Q Polynomial time

Polynomial Reduction Polynomial reduction can be used for polynomial time solution If A polynomially reduces to B, and we know a polynomial algorithm to B, then we can solve A in polynomial time as well.

Reducibility ● An example: ■ A: Given a set of Booleans, is at least one TRUE? ■ B: Given a set of integers, is their sum > 0? ■ Transformation: Given (x 1, x 2, …, x n ) construct (y 1, y 2, …, y n ) where y i = 1 if x i = TRUE, y i = 0 if x i = FALSE ● Another example: ■ Solving linear equations is reducible to solving quadratic equations ○ How can we easily use a quadratic-equation solver to solve linear equations?

NP-Hard and NP-Complete ● If P is polynomial-time reducible to Q, we denote this P  p Q ● Definition of NP-Hard and NP-Complete: ■ If all problems R  NP are reducible to P, then P is NP-Hard ■ We say P is NP-Complete if P is NP-Hard and P  NP ● If P  p Q and P is NP-Complete, Q is also NP- Complete

Why Prove NP-Completeness? ● Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractable ● Therefore it can be important to prove that a problem is NP-Complete ■ Don’t need to come up with an efficient algorithm ■ Can instead work on approximation algorithms

Proving NP-Completeness ● What steps do we have to take to prove a problem P is NP-Complete? ■ Pick a known NP-Complete problem Q ■ Reduce Q to P ○ Describe a transformation that maps an arbitrary instance of Q to some instance of P, s.t. “yes” for P = “yes” for Q ○ Prove the transformation works ○ Prove the transformation takes polynomial time ■ Oh yeah, prove P  NP (What if you can’t?)

Coming Up ● Given one NP-Complete problem, we can prove many interesting problems NP-Complete ■ Graph coloring (recall the wrestler-rivalry HW problem) ■ Hamiltonian cycle ■ Hamiltonian path ■ Knapsack problem ■ Traveling salesman ■ Job scheduling with penalties ■ Many, many more