CS 3343: Analysis of Algorithms Lecture 25: P and NP Some slides courtesy of Carola Wenk

Have seen so far Algorithms for various problems –Sorting, LCS, scheduling, MST, shortest path, etc –Running times O(n 2 ),O(n log n), O(n), O(nm), etc. –i.e., polynomial in the input size –Those are called tractable problems Can we solve all (or most of) interesting problems in polynomial time?

Example difficult problem Traveling Salesperson Problem (TSP) –Input: undirected graph with lengths on edges –Output: shortest tour that visits each vertex exactly once –An optimization problem Best known algorithm: O(n 2 n ) time. Intractable

Another difficult problem Maximum Clique: –Input: undirected graph G=(V,E) –Output: largest subset C of V such that every pair of vertices in C has an edge between them –An optimization problem Best known algorithm: O(n 2 n ) time

What can we do ? Spend more time designing algorithms for those problems –People tried for a few decades, no luck Prove there is no polynomial time algorithm for those problems –Would be great –Seems really difficult

What else can we do ? Show that those hard problems are essentially equivalent. i.e., if we can solve one of them in polynomial time, then all others can be solved in polynomial time as well. Works for at least hard problems

The benefits of equivalence Combines research efforts If one problem has polynomial time solution, then all of them do More realistically: Once an exponential lower bound is shown for one problem, it holds for all of them P1 P2 P3

Summing up If we show that a problem ∏ is equivalent to ten thousand other well studied problems without efficient algorithms, then we get a very strong evidence that ∏ is hard. We need to: –Identify the class of problems of interest –Define the notion of equivalence –Prove the equivalence(s)

Classes of problems: P and NP Implicitly, we mean decision problems: answer YES or NO –Other problems can be converted into decision problems –k-clique: is there a clique of size = k? P is set of problems that can be solved in polynomial time NP (nondeterministic polynomial time) is the set of problems that can be solved in polynomial time by a nondeterministic computer

Nondeterminism Think of a non-deterministic computer as a computer that magically “guesses” a solution, then has to verify that it is correct –If a solution exists, computer always guesses it –One way to imagine it: a parallel computer that can freely spawn an infinite number of processes Have one processor work on each possible solution All processors attempt to verify that their solution works If a processor finds it has a working solution –So: NP = problems verifiable in polynomial time

P and NP P = problems that can be solved in polynomial time NP = problems for which a solution can be verified in polynomial time K-clique problem is in NP: –Easy to verify solution in polynomial time Is sorting in NP? –Not a decision problem Is sortedness in NP? –Yes. Easy to verify

P and NP Is P = NP? –The biggest open problem in CS –Most suspect not –the Clay Mathematics Institute has offered a $1 million prize for the first proofClay Mathematics Institute NP P P = NP? or

NP-Complete Problems We will see that NP-Complete problems are the “hardest” problems in NP: –If any one NP-Complete problem can be solved in polynomial time… –…then every NP-Complete problem can be solved in polynomial time… –…and in fact every problem in NP can be solved in polynomial time (which would show P = NP) –Thus: solve TSP in O(n 100 ) time, you’ve proved that P = NP. Retire rich & famous.

Reduction The crux of NP-Completeness is reducibility –Informally, a problem  ’ can be reduced to another problem  if any instance of  ’ can be “easily rephrased” as an instance of , the solution to which provides a solution to the instance of  ’ What do you suppose “easily” means? This rephrasing is called transformation –Intuitively: If  ’ reduces to ,  ’ is “no harder to solve” than 

Reductions: ∏ ’ to ∏ A for ∏ YES NO x A’ for ∏’ YES NO x’

Reductions: ∏ ’ to ∏ A for ∏ YES NO f f(x’)= A’ for ∏’ x YES NO x’

Reduction: an example –  ’: Given a set of Booleans, is at least one TRUE? –  : Given a set of integers, is their sum positive? –Transformation: (x 1, x 2, …, x n ) = (y 1, y 2, …, y n ) where y i = 1 if x i = TRUE, y i = 0 if x i = FALSE

Reductions ∏ ’ is polynomial time reducible to ∏ ( ∏ ’ ≤ p ∏) iff there is a polynomial time function f that maps inputs x ’ for ∏ ’ into inputs x for ∏, such that for any x ’ ∏ ’ (x ’ )=∏(f(x ’ )) Fact 1: if ∏  P and ∏ ’ ≤ p ∏ then ∏ ’  P Fact 2: if ∏  NP and ∏ ’ ≤ p ∏ then ∏ ’  NP Fact 3 (transitivity): if ∏ ’’ ≤ p ∏ ’ and ∏ ’ ≤ p ∏ then ∏ ” ≤ p ∏

Recap We defined a large class of interesting problems, namely NP We have a way of saying that one problem is not harder than another (∏ ’ ≤ p ∏) Our goal: show equivalence between hard problems

Showing equivalence between difficult problems TSP P3 P4 Clique P5 Options: –Show reductions between all pairs of problems –Reduce the number of reductions using transitivity of “≤ ” ∏’  

Showing equivalence between difficult problems TSP P3 P4 Clique P5 Options: –Show reductions between all pairs of problems –Reduce the number of reductions using transitivity of “≤ ” –Show that all problems in NP are reducible to a fixed ∏. To show that some problem ∏ ’  NP is equivalent to all difficult problems, we only show ∏ ≤ p ∏ ’. ∏ ∏’   

The first problem ∏ Boolean satisfiability problem (SAT): –Given: a formula φ with m clauses over n variables, e.g., x 1 v x 2 v x 5, x 3 v ¬ x 5 –Check if there exists TRUE/FALSE assignments to the variables that makes the formula satisfiable Any SAT can be rewritten into conjunctive normal form (CNF) –a conjunction of clauses, where a clause is a disjunction of literalsconjunctionclauses disjunctionliterals A literal is a Boolean variable or its negation –E.g. (A  B)  (  B  C  D) is in CNF (A  B)  (  A  C  D)is not in CNF

SAT is NP-complete Fact: SAT  NP Theorem [Cook ’ 71]: For any ∏ ’  NP, we have ∏ ’ ≤ SAT. Definition: A problem ∏ such that for any ∏ ’  NP we have ∏ ’ ≤ p ∏, is called NP-hard –Not necessarily decision problem Definition: An NP-hard problem that belongs to NP is called NP-complete Corollary: SAT is NP-complete.

SAT: the boolean satisfiability problem for formulas in conjunctive normal formSATconjunctive normal form 0-1 INTEGER PROGRAMMING CLIQUE (see also independent set problem)CLIQUEindependent set problem SET PACKING VERTEX COVER SET COVERING FEEDBACK NODE SET FEEDBACK ARC SET DIRECTED HAMILTONIAN CIRCUIT UNDIRECTED HAMILTONIAN CIRCUIT 3-SAT CHROMATIC NUMBER (also called the Graph Coloring Problem)CHROMATIC NUMBERGraph Coloring Problem CLIQUE COVER EXACT COVER HITTING SET STEINER TREE 3-dimensional MATCHING KNAPSACK (Karp's definition of Knapsack is closer to Subset sum)KNAPSACKSubset sum JOB SEQUENCING PARTITION MAX-CUT Karp's 21 NP-complete problems, 1972 Since then thousands problems have been proved to be NP- complete

Clique again Clique (decision variant): –Input: undirected graph G=(V,E), K –Output: is there a subset C of V, |C|≥K, such that every pair of vertices in C has an edge between them

SAT ≤ p Clique Given a SAT formula φ=C 1, …,C m over x 1, …,x n, we need to produce G=(V,E) and K, such that φ satisfiable iff G has a clique of size ≥ K. x’ f(x’)=x

SAT ≤ p Clique reduction For each literal t occurring in φ, create a vertex v t Create an edge v t – v t ’ iff: –t and t ’ are not in the same clause, and –t is not the negation of t ’

SAT ≤ p Clique example Formula: (x 1 V x 2 V x 3 )  ( ¬ x 2 v ¬ x 3 )  ( ¬ x 1 v x 2 ) Graph: x1x1 x2x2 x3x3 ¬x 2 ¬ x 1 ¬ x 3 x2x2 Claim: φ satisfiable iff G has a clique of size ≥ m t and t’ are not in the same clause, and t is not the negation of t’ Edge v t – v t’ 

Proof “→” part: –Take any assignment that satisfies φ. E.g., x 1 =F, x 2 =T, x 3 =F –Let the set C contain one satisfied literal per clause –C is a clique x1x1 x2x2 x3x3 ¬x 2 ¬ x 1 ¬ x 3 x2x2 t and t’ are not in the same clause, and t is not the negation of t’ Edge v t – v t’ 

Proof “←” part: –Take any clique C of size ≥ m (i.e., = m) –Create a set of equations that satisfies selected literals. E.g., x 3 =T, x 2 =F, x 1 =F –The set of equations is consistent and the solution satisfies φ x1x1 x2x2 x3x3 ¬x 2 ¬ x 1 ¬ x 3 x2x2 t and t’ are not in the same clause, and t is not the negation of t’ Edge v t – v t’ 

Altogether We constructed a reduction that maps: –YES inputs to SAT to YES inputs to Clique –NO inputs to SAT to NO inputs to Clique The reduction works in polynomial time Therefore, SAT ≤ p Clique →Clique NP- hard Clique is in NP → Clique is NP-complete

Independent set (IS) Input: undirected graph G=(V,E) Output: is there a subset S of V, |S|≥K such that no pair of vertices in S has an edge between them Want to show: IS is NP- complete

Clique ≤ IS Given an input G=(V,E), K to Clique, need to construct an input G’=(V’,E’), K’ to IS, such that G has clique of size ≥K iff G ’ has IS of size ≥K ’. Construction: K ’ =K,V ’ =V,E ’ =E Reason: C is a clique in G iff it is an IS in G ’ s complement. x’ f(x’)=x

Classes of problems