1. NP and NP-completeness
The Chinese University of Hong Kong
Fall 2010
CSCI 3130: Formal languages and automata theory
NP and NP-completeness
Andrej Bogdanov

2. Some more problems A clique is a subset of vertices
that are all interconnected 1 2
{1, 4}, {2, 3, 4}, {1} are cliques
An independent set is a subset of
vertices so that no pair is connected 3 4
{1, 2}, {1, 3}, {4} are independent sets
Graph G
there is no independent set of size 3
A vertex cover is a set of vertices
that touches (covers) all edges
{2, 4}, {3, 4}, {1, 2, 3} are vertex covers

3. Boolean formula satisfiability
A boolean formula is an expression made up of variables, ands, ors, and negations, like
The formula is satisfiable if one can assign values to the variables so the expression evaluates to true
(x1∨x2 ) ∧ (x2 ∨x3 ∨x4) ∧ (x1)
Above formula is satisfiable because this assignment
makes it true:
x1 = F x2 = F x3 = T x4 = T

4. 3SAT SAT = {〈f〉: f is a satisfiable Boolean formula}
3SAT = {〈f〉: f is a satisfiable Boolean formula in conjunctive normal form with literals per clause}
literal: xi or xi
(x1∨x2∨x2 ) ∧ (x2∨x3∨x4)
CNF: AND of ORs of literals
literals
clause
(conjunctive normal form)
3CNF: CNF with 3 literals per clause
(repetitions are allowed)

5. Status of these problems
CLIQUE = {〈G, k〉: G is a graph with a clique of k vertices}
IS = {〈G, k〉: G is a graph with an independent set of k vertices}
VC = {〈G, k〉: G is a graph with a vertex cover of k vertices}
SAT = {〈f〉: f is a satisfiable Boolean formula}
problem
CLIQUE IS VC
3SAT
running time
of best-known algorithm
2O(n)
2O(n)
2O(n)
2O(n)
What do these problems have in common?

6. Checking solutions efficiently
We don’t know how to solve them efficiently
But if someone told us the solution, we would be able to verify it very quickly
Example:
Is (G, 5) in CLIQUE? 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
1,5,9,12,14

7. Example: Formula satisfiability
(x1∨x2 ) ∧ (x2 ∨x3 ∨x4) ∧ (x1)
Finding a solution:
Verifying a solution:
Try all possible assignments
FFTT
FFFF
FTFF
TFFF
TTFF
substitute
FFFT
FTFT
TFFT
TTFT
x1 = F x2 = F x3 = T x4 = T
FFTF
FTTF
TFTF
TTTF
FFTT
FTTT
TFTT
TTTT
evaluate formula
For n variables, there are 2n
possible assignments
f =
(F ∨T ) ∧ (F∨T∨F) ∧ (T)
can be done in linear time
Takes exponential time

8. The class NP A verifier for L is a TM V such that x ∈ L
s is a potential solution for x
We say V runs in polynomial time if on every input x, it runs in time polynomial in |x| (for every s)
x ∈ L
V accepts 〈x, s〉 for some s
NP is the class of all languages that have polynomial-time verifiers

9. Examples 3SAT is in NP: ✔ CLIQUE is in NP: ✔ V := On input a 3CNF f
and a candidate assignment a,
If a satisfies f accept, otherwise reject.
running time = O(m + n) ✔
m = number of clauses and n = number of variables
CLIQUE is in NP:
V :=
On input 〈G, k〉
and a set of vertices C,
If C has size k and all edges between vertices of
C are present in G, accept, otherwise reject.
running time = O(n2) ✔

10. P versus NP P is contained in NP
decidable
NP (efficiently checkable)
because the verifier can ignore the solution Conceptually, finding solutions can only be harder than checking them IS
SAT VC
CLIQUE
P (efficient)
PATH
L01

11. Millenium prize problems
Recall how in 1900, Hilbert gave 23 problems that guided mathematics in the 20th century
In 2000, the Clay Mathematical Institute gave 7 problems for the 21st century
1 P versus NP
2 The Hodge conjecture
3 The Poincaré conjecture
4 The Riemann hypothesis
5 Yang–Mills existence and mass gap
6 Navier–Stokes existence and smoothness
7 The Birch and Swinnerton-Dyer conjecture
computer science
Perelman 2006 (refused money)
$1,000,000
Hilbert’s 8th problem

12. P versus NP
The answer to the question is not known. But one reason it is believed to be negative is because, intuitively, searching is harder than verifying
For example, solving homework problems (searching for solutions) is harder than grading (verifying the solution is correct)
$1,000,000
Is P equal to NP?

13. Searching versus verifying
Mathematician: Given a mathematical claim, come up with a proof for it.
Scientist: Given a collection of data on some phenomena, find a theory explaining it.
Engineer: Given a set of constraints (on cost, physical laws, etc.) come up with a design (of an engine, bridge, etc.) which meets them.
Detective:
Given the crime scene, find “who’s done it”.

14. P and NP P = languages that can be decided on a TM
with polynomial running time
(problems that admit efficient algorithms)
NP =
languages whose solutions can be verified
on a TM with polynomial running time
(solutions can be checked efficiently)
decidable
We believe that NP is bigger than P,
but we are not 100% sure NP P

15. Evidence that NP is bigger than P
CLIQUE = {〈G, k〉: G is a graph with a clique of k vertices}
IS = {〈G, k〉: G is a graph with an independent set of k vertices}
VC = {〈G, k〉: G is a graph with a vertex cover of k vertices}
SAT = {〈f〉: f is a satisfiable Boolean formula}
These (and many others) are in NP
Their solutions, once found, are easy to verify
But no efficient algorithms are known for any of them
The fastest known programs take time ≈2n

16. Equivalence of certain NP languages
We strongly suspect that problems like CLIQUE, SAT, etc. require time ≈2n to solve
We do not know how to prove this, but what we can prove is that
If any one of them can be solved on a polynomial- time TM, then all of them can be solved

17. Equivalence of some NP languages
All these problems are as hard as one another
Moreover, they are at the “frontier” of NP
They are at least as hard as any problem in NP NP
clique
independent set
vertex-cover P
satisfiability

18. NP-completeness

19. NP-completeness

20. NP-completeness

21. Polynomial Time Reduction
How to show that a problem R is not easier than a problem Q?
Informally, if R can be solved efficiently, we can solve Q efficiently.
Formally, we say Q polynomially reduces to R if:
Given an instance q of problem Q
There is a polynomial time transformation to an instance f(q) of R
q is a “yes” instance if and only if f(q) is a “yes” instance
Then, if R is polynomial time solvable, then Q is polynomial time solvable.
If Q is not polynomial time solvable, then R is not polynomial time solvable.

22. Polynomial-time reductions
What do we mean when we say, for example,
We mean that
Or, we can convert any polynomial-time TM for IS into one for CLIQUE
“IS is at least as hard as CLIQUE”
If CLIQUE has no polynomial-time TM, then
neither does IS

23. Polynomial-time reductions
IS = {〈G, k〉: G is a graph with an independent set
of k vertices}
CLIQUE = {〈G, k〉: G is a graph with a clique of k vertices}
Theorem 1 2
If IS has a polynomial-time TM,
so does CLIQUE 3 4
{1, 4}, {2, 3, 4}, {1} are
independent sets
{1, 2}, {1, 3}, {4} are cliques

24. Polynomial-time reductions
If IS has a polynomial-time TM, so does CLIQUE
Proof: Suppose IS has an poly-time TM A
We want to use it to solve CLIQUE
〈G, k〉
reject
if not
accept
if G has clique of size k
reject
if not
accept
if G’ has IS of size k A
for IS
〈G’, k’ 〉

25. Reducing CLIQUE to IS
We look for a polynomial-time TM R that turns the question:
into: R
“Does G have a clique of size k?”
G, k
G’, k’
“Does G’ have an IS of size k’?” 1 2 3 4 1 2 3 4
flip all edges G G’
k’ = k
cliques of size k
ISs of size k’

26. ✓ ✓ Reducing CLIQUE to IS R On input 〈G, k〉
Construct G’ by flipping all edges of G
Set k’ = k
Output 〈G’, k’〉 R
G, k
G’, k’
cliques in G
independent sets in G’
If G’ has an IS of size k,
then G has a clique of size k ✓
If G’ does not have an IS of size k,
then G has no clique of size k ✓

27. Reduction recap
We showed that by converting an imaginary TM for IS into one for CLIQUE
To do this, we came up with a reduction that transforms instances of CLIQUE into ones of IS
If IS has a polynomial-time TM, so does CLIQUE

28. Polynomial-time reductions
Language L polynomial-time reduces to L’ if there exists a polynomial-time TM R that takes an instance x of L into instance y of L’ s.t.
x ∈ L if and only if y ∈ L’ L
(CLIQUE) L’
(IS) x
= 〈G, k〉 R y
= 〈G’, k’〉
x ∈ L
y ∈ L’
(G has clique of size k)
(G’ has IS of size k’)

29. The meaning of reductions
Saying L reduces to L’ means L is no harder than L’
In other words, if we can solve L’, then we can also solve L
Therefore
If L reduces to L’ and L’ ∈ P, then L ∈ P
acc R
poly-time TM for L’ x y
rej
x ∈ L
y ∈ L’
TM accepts

30. The direction of reductions
The direction of the reduction is very important
Saying “A is easier than B” and “B is easier than A” mean different things
However, it is possible that L reduces to L’ and L’ reduces to L
This means that L and L’ are as hard as one another
For example, IS and CLIQUE reduce to one another

31. The Cook-Levin Theorem
Every L ∈ NP reduces to SAT
SAT = {f: f is a satisfiable Boolean formula}
E.g.
(x1∨x2 ) ∧ (x2 ∨x3 ∨x4) ∧ (x1)
So every problem in NP is easier than SAT
But SAT itself is in NP, so SAT must be the “hardest problem” in NP: P
SAT NP
If SAT ∈ P, then P = NP

32. NP-completeness A language C is NP-complete if: 1. C is in NP, and
Cook-Levin Theorem:
1. C is in NP, and
2. For every L in NP, L reduces to C. P C NP
3SAT is NP-complete

33. Our picture of NP A B A reduces to B NP-complete SAT NP CLIQUE IS P
0n1n
PATH
any CFL

34. More NP-complete problemsA B
A reduces to B
3SAT IS
CLIQUE P NP
PATH
0n1n
In practice, most of the NP-problems are
either in P (easy) or NP-complete (probably hard)

35. Interpretation of Cook-Levin Theorem
Optimistic view:
Pessimistic view:
If we manage to solve SAT, then we can also solve
CLIQUE, scheduling, and almost anything
Since we do not believe P = NP, it is unlikely that
we will ever have a fast algorithm for SAT

36. Clique Theorem CLIQUE is NP-hard
CLIQUE = {(G, k): G is a graph with a clique of k vertices}
Theorem
CLIQUE is NP-hard VC IS 2
A clique is a subset of vertices so that all pairs are connected 1
CLIQUE
3SAT
{1, 2, 3}, {1, 4}, {4} are cliques ✓
SAT 3 4

37. Reducing 3SAT to CLIQUE Proof: We give a reduction from 3SAT to CLIQUE
3SAT = {f: f is a satisfiable Boolean formula in 3CNF}
CLIQUE = {(G, k): G is a graph with a clique of k vertices}
3CNF formula f R
(G, k)
G has a clique of size k
f is satisfiable

38. Reducing 3SAT to CLIQUE Example: f =
(x1∨x1∨x2 ) ∧ (x1∨x2∨x2) ∧ (x1∨x2∨x3) x1 x2 x1 x2 x1 x2 x3
Put a vertex for every literal
Put an edge for every consistent pair

39. Reducing 3SAT to CLIQUE 3CNF formula f R (G, k) R:
On input f, where f is a 3CNF formula with m clauses
Construct the following graph G:
G has 3m vertices, divided into m groups,
one for each literal in f
If a and b are in different groups and a ≠ b,
put an edge (a, b)
Output (G, m)

40. Reducing 3SAT to CLIQUE 3CNF formula f R (G, m)
G has a clique of size m
f is satisfiable x1 x1 x1 x1 x2 x2 x2 x2 x3
f = (x1∨x1∨x2 ) ∧ (x1∨x2∨x2) ∧ (x1∨x2∨x3) T T F F F T F F T

41. Reducing 3SAT to CLIQUE 3CNF formula f R (G, m)
G has a clique of size m
f is satisfiable x1 x1 x1 x1 x2 x2 x2 x2 x3
f = (x1∨x1∨x2 ) ∧ (x1∨x2∨x2) ∧ (x1∨x2∨x3) F F T T F F T T T

42. Reducing 3SAT to CLIQUE
3CNF formula f R
(G, m)
f is satisfiable
G has a clique of size m ✓
SAT
3SAT IS
CLIQUE VC
Every satisfying assignment of f gives a clique of size m in G
Conversely, every clique of size m in G gives a consistent satisfying assignment of f. ✓ ✓

43. Vertex cover Theorem VC is NP-hard
VC = {(G, k): G is a graph with a vertex cover of size k}
Theorem
VC is NP-hard VC IS
A vertex cover is a set of vertices that touches (covers) all edges ✓
CLIQUE 1 2 3 4 ✓
3SAT ✓
{2, 4}, {3, 4}, {1, 2, 3}
are vertex covers
SAT

44. Reducing CLIQUE to VC Proof: We describe a reduction from IS to VC
Example R
(G, k)
(G’, k’)
G has an IS of size k
G’ has a VC of size k’
vertex covers
independent sets 1 2 3 4
{2, 4}, {3, 4},
{1, 2, 3}, {1, 2, 4},
{1, 3, 4}, {2, 3, 4},
{1, 2, 3, 4}
∅, {1}, {2}, {3}, {4},
{1, 2}, {1, 3}

45. Reducing IS to VC Claim S is an independent set of G if and Proof1 2 3 4
S is an independent set of G if and
only if S is a vertex cover of G ∅
{1}
{2}
{3}
{4}
{1, 2}
{1, 3}
{2, 4}
{3, 4}
{1, 2, 3}
{1, 2, 4}
{1, 3, 4}
{2, 3, 4}
{1, 2, 3, 4} IS VC
S is an independent set of G
no edge has both endpoints in S
every edge has an endpoint in S
S is a vertex cover of G

46. Reducing IS to VC ✓ R (G, k) (G’, k’) ✓ On input (G, k), R:
SAT
3SAT IS
CLIQUE VC R
(G, k)
(G’, k’) ✓
On input (G, k), R:
Output (G, n – k).
G has a VC of size n – k
G has an IS of size k

47. The ubiquity of NP-complete problems
We saw a few examples of NP-complete problems, but there are many more
A surprising fact of life is that most CS problems are either in P or NP-complete
A 1979 book by Garey and Johnson lists 100+ NP-complete problems

48. Practicing Reductions
Instance: A set X and a size s(x) for each x in X.
Question: Is there a subset X’ X such that
PARTITION
SUBSET-SUM
Instance: A set X and a size s(x) for each x in X, and an integer B.
Question: Is there a subset X’ X such that

49. Practicing Reductions
HAMILTONIAN CYCLE
Instance: A graph G=(V,E).
Question: Does G contains a Hamiltonian cycle,
i.e. a cycle which visits every vertex exactly once?
HAMILTONIAN PATH
Instance: A graph G=(V,E).
Question: Does G contains a Hamiltonian path,
i.e. a path which visits every vertex exactly once?

50. Techniques for Proving NP-completeness
Restriction
Show that a special case is already NP-complete.
Local replacement
Replace each basic unit by a different structure.
Component design
Design “components” with specific functionality.

51. u —v in E1 iff f (u)—f (v) in E2
Subgraph Isomorphism
Two graphs G1 = (V1,E1) and G2 = (V2,E2) are isomorphic if
 bijection f: V1 → V2
u —v in E1 iff f (u)—f (v) in E2
Instance: Two graphs G = (V1,E1) and H = (V2,E2).
Question: Does G contain a subgraph isomorphic to H?
Clique <= Subgraph Isomorphism

52. Bounded Degree Spanning Tree
A spanning tree is a connected subgraph with |V|-1 edges.
Instance: A graph G=(V,E) and a positive integer k.
Question: Is there a spanning tree for G in which no vertex has degree > k?
Hamiltonian path <= Bounded degree spanning tree

53. Minimum Cover
Instance: Collection C of subsets of a set S, and a positive integer k.
Question: Does C contains a cover for S of size k or less, that is,
a subset C’ C with |C’| <= k and ?
Vertex cover <= Minimum Cover

54. Dominating Set Instance: A graph G and a positive integer k.
Question: Does there exist a subset S of at most k vertices such that
every vertex in V-S is adjacent to at least one vertex in S?
Vertex cover <= dominating set

55. Sequencing within Intervals
Instance: A set T of jobs, each has a release time r(t),
a deadline d(t) and a length l(t).
Question: Does there exist a feasible schedule for T?
Partition <= Sequencing within Intervals

56. Subset Sum
Instance: A set X and a size s(x) for each x in X, and an integer B.
Question: Is there a subset X’ X such that
Vertex cover <= subset sum
See this proof and many other problems and reductions from Prof. Cai notes: