NP and NP-completeness The Chinese University of Hong Kong Fall 2010 CSCI 3130: Formal languages and automata theory NP and NP-completeness Andrej Bogdanov http://www.cse.cuhk.edu.hk/~andrejb/csc3130

Some more problems A clique is a subset of vertices that are all interconnected 1 2 {1, 4}, {2, 3, 4}, {1} are cliques An independent set is a subset of vertices so that no pair is connected 3 4 {1, 2}, {1, 3}, {4} are independent sets Graph G there is no independent set of size 3 A vertex cover is a set of vertices that touches (covers) all edges {2, 4}, {3, 4}, {1, 2, 3} are vertex covers

Boolean formula satisfiability A boolean formula is an expression made up of variables, ands, ors, and negations, like The formula is satisfiable if one can assign values to the variables so the expression evaluates to true (x1∨x2 ) ∧ (x2 ∨x3 ∨x4) ∧ (x1) Above formula is satisfiable because this assignment makes it true: x1 = F x2 = F x3 = T x4 = T

3SAT SAT = {〈f〉: f is a satisfiable Boolean formula} 3SAT = {〈f〉: f is a satisfiable Boolean formula in conjunctive normal form with 3 literals per clause} literal: xi or xi (x1∨x2∨x2 ) ∧ (x2∨x3∨x4) CNF: AND of ORs of literals literals clause (conjunctive normal form) 3CNF: CNF with 3 literals per clause (repetitions are allowed)

Status of these problems CLIQUE = {〈G, k〉: G is a graph with a clique of k vertices} IS = {〈G, k〉: G is a graph with an independent set of k vertices} VC = {〈G, k〉: G is a graph with a vertex cover of k vertices} SAT = {〈f〉: f is a satisfiable Boolean formula} problem CLIQUE IS VC 3SAT running time of best-known algorithm 2O(n) 2O(n) 2O(n) 2O(n) What do these problems have in common?

Checking solutions efficiently We don’t know how to solve them efficiently But if someone told us the solution, we would be able to verify it very quickly Example: Is (G, 5) in CLIQUE? 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1,5,9,12,14

Example: Formula satisfiability (x1∨x2 ) ∧ (x2 ∨x3 ∨x4) ∧ (x1) Finding a solution: Verifying a solution: Try all possible assignments FFTT FFFF FTFF TFFF TTFF substitute FFFT FTFT TFFT TTFT x1 = F x2 = F x3 = T x4 = T FFTF FTTF TFTF TTTF FFTT FTTT TFTT TTTT evaluate formula For n variables, there are 2n possible assignments f = (F ∨T ) ∧ (F∨T∨F) ∧ (T) can be done in linear time Takes exponential time

The class NP A verifier for L is a TM V such that x ∈ L s is a potential solution for x We say V runs in polynomial time if on every input x, it runs in time polynomial in |x| (for every s) x ∈ L V accepts 〈x, s〉 for some s NP is the class of all languages that have polynomial-time verifiers

Examples 3SAT is in NP: ✔ CLIQUE is in NP: ✔ V := On input a 3CNF f and a candidate assignment a, If a satisfies f accept, otherwise reject. running time = O(m + n) ✔ m = number of clauses and n = number of variables CLIQUE is in NP: V := On input 〈G, k〉 and a set of vertices C, If C has size k and all edges between vertices of C are present in G, accept, otherwise reject. running time = O(n2) ✔

P versus NP P is contained in NP decidable NP (efficiently checkable) because the verifier can ignore the solution Conceptually, finding solutions can only be harder than checking them IS SAT VC CLIQUE P (efficient) PATH L01

Millenium prize problems Recall how in 1900, Hilbert gave 23 problems that guided mathematics in the 20th century In 2000, the Clay Mathematical Institute gave 7 problems for the 21st century 1 P versus NP 2 The Hodge conjecture 3 The Poincaré conjecture 4 The Riemann hypothesis 5 Yang–Mills existence and mass gap 6 Navier–Stokes existence and smoothness 7 The Birch and Swinnerton-Dyer conjecture computer science Perelman 2006 (refused money) $1,000,000 Hilbert’s 8th problem

P versus NP The answer to the question is not known. But one reason it is believed to be negative is because, intuitively, searching is harder than verifying For example, solving homework problems (searching for solutions) is harder than grading (verifying the solution is correct) $1,000,000 Is P equal to NP?

Searching versus verifying Mathematician: Given a mathematical claim, come up with a proof for it. Scientist: Given a collection of data on some phenomena, find a theory explaining it. Engineer: Given a set of constraints (on cost, physical laws, etc.) come up with a design (of an engine, bridge, etc.) which meets them. Detective: Given the crime scene, find “who’s done it”.

P and NP P = languages that can be decided on a TM with polynomial running time (problems that admit efficient algorithms) NP = languages whose solutions can be verified on a TM with polynomial running time (solutions can be checked efficiently) decidable We believe that NP is bigger than P, but we are not 100% sure NP P

Evidence that NP is bigger than P CLIQUE = {〈G, k〉: G is a graph with a clique of k vertices} IS = {〈G, k〉: G is a graph with an independent set of k vertices} VC = {〈G, k〉: G is a graph with a vertex cover of k vertices} SAT = {〈f〉: f is a satisfiable Boolean formula} These (and many others) are in NP Their solutions, once found, are easy to verify But no efficient algorithms are known for any of them The fastest known programs take time ≈2n

Equivalence of certain NP languages We strongly suspect that problems like CLIQUE, SAT, etc. require time ≈2n to solve We do not know how to prove this, but what we can prove is that If any one of them can be solved on a polynomial- time TM, then all of them can be solved

Equivalence of some NP languages All these problems are as hard as one another Moreover, they are at the “frontier” of NP They are at least as hard as any problem in NP NP clique independent set vertex-cover P satisfiability

NP-completeness

NP-completeness

NP-completeness

Polynomial Time Reduction How to show that a problem R is not easier than a problem Q? Informally, if R can be solved efficiently, we can solve Q efficiently. Formally, we say Q polynomially reduces to R if: Given an instance q of problem Q There is a polynomial time transformation to an instance f(q) of R q is a “yes” instance if and only if f(q) is a “yes” instance Then, if R is polynomial time solvable, then Q is polynomial time solvable. If Q is not polynomial time solvable, then R is not polynomial time solvable.

Polynomial-time reductions What do we mean when we say, for example, We mean that Or, we can convert any polynomial-time TM for IS into one for CLIQUE “IS is at least as hard as CLIQUE” If CLIQUE has no polynomial-time TM, then neither does IS

Polynomial-time reductions IS = {〈G, k〉: G is a graph with an independent set of k vertices} CLIQUE = {〈G, k〉: G is a graph with a clique of k vertices} Theorem 1 2 If IS has a polynomial-time TM, so does CLIQUE 3 4 {1, 4}, {2, 3, 4}, {1} are independent sets {1, 2}, {1, 3}, {4} are cliques

Polynomial-time reductions If IS has a polynomial-time TM, so does CLIQUE Proof: Suppose IS has an poly-time TM A We want to use it to solve CLIQUE 〈G, k〉 reject if not accept if G has clique of size k reject if not accept if G’ has IS of size k A for IS 〈G’, k’ 〉

Reducing CLIQUE to IS We look for a polynomial-time TM R that turns the question: into: R “Does G have a clique of size k?” G, k G’, k’ “Does G’ have an IS of size k’?” 1 2 3 4 1 2 3 4 flip all edges G G’ k’ = k cliques of size k ISs of size k’

✓ ✓ Reducing CLIQUE to IS R On input 〈G, k〉 Construct G’ by flipping all edges of G Set k’ = k Output 〈G’, k’〉 R G, k G’, k’ cliques in G independent sets in G’ If G’ has an IS of size k, then G has a clique of size k ✓ If G’ does not have an IS of size k, then G has no clique of size k ✓

Reduction recap We showed that by converting an imaginary TM for IS into one for CLIQUE To do this, we came up with a reduction that transforms instances of CLIQUE into ones of IS If IS has a polynomial-time TM, so does CLIQUE

Polynomial-time reductions Language L polynomial-time reduces to L’ if there exists a polynomial-time TM R that takes an instance x of L into instance y of L’ s.t. x ∈ L if and only if y ∈ L’ L (CLIQUE) L’ (IS) x = 〈G, k〉 R y = 〈G’, k’〉 x ∈ L y ∈ L’ (G has clique of size k) (G’ has IS of size k’)

The meaning of reductions Saying L reduces to L’ means L is no harder than L’ In other words, if we can solve L’, then we can also solve L Therefore If L reduces to L’ and L’ ∈ P, then L ∈ P acc R poly-time TM for L’ x y rej x ∈ L y ∈ L’ TM accepts

The direction of reductions The direction of the reduction is very important Saying “A is easier than B” and “B is easier than A” mean different things However, it is possible that L reduces to L’ and L’ reduces to L This means that L and L’ are as hard as one another For example, IS and CLIQUE reduce to one another

The Cook-Levin Theorem Every L ∈ NP reduces to SAT SAT = {f: f is a satisfiable Boolean formula} E.g. (x1∨x2 ) ∧ (x2 ∨x3 ∨x4) ∧ (x1) So every problem in NP is easier than SAT But SAT itself is in NP, so SAT must be the “hardest problem” in NP: P SAT NP If SAT ∈ P, then P = NP

NP-completeness A language C is NP-complete if: 1. C is in NP, and Cook-Levin Theorem: 1. C is in NP, and 2. For every L in NP, L reduces to C. P C NP 3SAT is NP-complete

Our picture of NP A B A reduces to B NP-complete SAT NP CLIQUE IS P 0n1n PATH any CFL

More NP-complete problems A B A reduces to B 3SAT IS CLIQUE P NP PATH 0n1n In practice, most of the NP-problems are either in P (easy) or NP-complete (probably hard)

Interpretation of Cook-Levin Theorem Optimistic view: Pessimistic view: If we manage to solve SAT, then we can also solve CLIQUE, scheduling, and almost anything Since we do not believe P = NP, it is unlikely that we will ever have a fast algorithm for SAT

Clique Theorem CLIQUE is NP-hard CLIQUE = {(G, k): G is a graph with a clique of k vertices} Theorem CLIQUE is NP-hard VC IS 2 A clique is a subset of vertices so that all pairs are connected 1 CLIQUE 3SAT {1, 2, 3}, {1, 4}, {4} are cliques ✓ SAT 3 4

Reducing 3SAT to CLIQUE Proof: We give a reduction from 3SAT to CLIQUE 3SAT = {f: f is a satisfiable Boolean formula in 3CNF} CLIQUE = {(G, k): G is a graph with a clique of k vertices} 3CNF formula f R (G, k) G has a clique of size k f is satisfiable

Reducing 3SAT to CLIQUE Example: f = (x1∨x1∨x2 ) ∧ (x1∨x2∨x2) ∧ (x1∨x2∨x3) x1 x2 x1 x2 x1 x2 x3 Put a vertex for every literal Put an edge for every consistent pair

Reducing 3SAT to CLIQUE 3CNF formula f R (G, k) R: On input f, where f is a 3CNF formula with m clauses Construct the following graph G: G has 3m vertices, divided into m groups, one for each literal in f If a and b are in different groups and a ≠ b, put an edge (a, b) Output (G, m)

Reducing 3SAT to CLIQUE 3CNF formula f R (G, m) G has a clique of size m f is satisfiable x1 x1 x1 x1 x2 x2 x2 x2 x3 f = (x1∨x1∨x2 ) ∧ (x1∨x2∨x2) ∧ (x1∨x2∨x3) T T F F F T F F T

Reducing 3SAT to CLIQUE 3CNF formula f R (G, m) G has a clique of size m f is satisfiable x1 x1 x1 x1 x2 x2 x2 x2 x3 f = (x1∨x1∨x2 ) ∧ (x1∨x2∨x2) ∧ (x1∨x2∨x3) F F T T F F T T T

Reducing 3SAT to CLIQUE 3CNF formula f R (G, m) f is satisfiable G has a clique of size m ✓ SAT 3SAT IS CLIQUE VC Every satisfying assignment of f gives a clique of size m in G Conversely, every clique of size m in G gives a consistent satisfying assignment of f. ✓ ✓

Vertex cover Theorem VC is NP-hard VC = {(G, k): G is a graph with a vertex cover of size k} Theorem VC is NP-hard VC IS A vertex cover is a set of vertices that touches (covers) all edges ✓ CLIQUE 1 2 3 4 ✓ 3SAT ✓ {2, 4}, {3, 4}, {1, 2, 3} are vertex covers SAT

Reducing CLIQUE to VC Proof: We describe a reduction from IS to VC Example R (G, k) (G’, k’) G has an IS of size k G’ has a VC of size k’ vertex covers independent sets 1 2 3 4 {2, 4}, {3, 4}, {1, 2, 3}, {1, 2, 4}, {1, 3, 4}, {2, 3, 4}, {1, 2, 3, 4} ∅, {1}, {2}, {3}, {4}, {1, 2}, {1, 3}

Reducing IS to VC Claim S is an independent set of G if and Proof 1 2 3 4 S is an independent set of G if and only if S is a vertex cover of G ∅ {1} {2} {3} {4} {1, 2} {1, 3} {2, 4} {3, 4} {1, 2, 3} {1, 2, 4} {1, 3, 4} {2, 3, 4} {1, 2, 3, 4} IS VC S is an independent set of G no edge has both endpoints in S every edge has an endpoint in S S is a vertex cover of G

Reducing IS to VC ✓ R (G, k) (G’, k’) ✓ On input (G, k), R: SAT 3SAT IS CLIQUE VC R (G, k) (G’, k’) ✓ On input (G, k), R: Output (G, n – k). G has a VC of size n – k G has an IS of size k

The ubiquity of NP-complete problems We saw a few examples of NP-complete problems, but there are many more A surprising fact of life is that most CS problems are either in P or NP-complete A 1979 book by Garey and Johnson lists 100+ NP-complete problems

Practicing Reductions Instance: A set X and a size s(x) for each x in X. Question: Is there a subset X’ X such that PARTITION SUBSET-SUM Instance: A set X and a size s(x) for each x in X, and an integer B. Question: Is there a subset X’ X such that

Practicing Reductions HAMILTONIAN CYCLE Instance: A graph G=(V,E). Question: Does G contains a Hamiltonian cycle, i.e. a cycle which visits every vertex exactly once? HAMILTONIAN PATH Instance: A graph G=(V,E). Question: Does G contains a Hamiltonian path, i.e. a path which visits every vertex exactly once?

Techniques for Proving NP-completeness Restriction Show that a special case is already NP-complete. Local replacement Replace each basic unit by a different structure. Component design Design “components” with specific functionality.

u —v in E1 iff f (u)—f (v) in E2 Subgraph Isomorphism Two graphs G1 = (V1,E1) and G2 = (V2,E2) are isomorphic if  bijection f: V1 → V2 u —v in E1 iff f (u)—f (v) in E2 Instance: Two graphs G = (V1,E1) and H = (V2,E2). Question: Does G contain a subgraph isomorphic to H? Clique <= Subgraph Isomorphism

Bounded Degree Spanning Tree A spanning tree is a connected subgraph with |V|-1 edges. Instance: A graph G=(V,E) and a positive integer k. Question: Is there a spanning tree for G in which no vertex has degree > k? Hamiltonian path <= Bounded degree spanning tree

Minimum Cover Instance: Collection C of subsets of a set S, and a positive integer k. Question: Does C contains a cover for S of size k or less, that is, a subset C’ C with |C’| <= k and ? Vertex cover <= Minimum Cover

Dominating Set Instance: A graph G and a positive integer k. Question: Does there exist a subset S of at most k vertices such that every vertex in V-S is adjacent to at least one vertex in S? Vertex cover <= dominating set

Sequencing within Intervals Instance: A set T of jobs, each has a release time r(t), a deadline d(t) and a length l(t). Question: Does there exist a feasible schedule for T? Partition <= Sequencing within Intervals

Subset Sum Instance: A set X and a size s(x) for each x in X, and an integer B. Question: Is there a subset X’ X such that Vertex cover <= subset sum See this proof and many other problems and reductions from Prof. Cai notes: http://www.cse.cuhk.edu.hk/~csci3160/LectureNotes/11notes.pdf