1. CS 3343: Analysis of Algorithms
Lecture 25: P and NP

2. Have seen so far Algorithms for various problems
Sorting, order statistics, LCS, scheduling, MST, shortest path, etc
Running times O(n2) ,O(n log n), O(n), O(nm), etc.
i.e., polynomial in the input size
Those are called tractable problems
Can we solve all (or most of) interesting problems in polynomial time?

3. Example difficult problem5 9 8 10 4 3 2 11 6 7
Traveling Salesperson Problem (TSP)
Input: undirected graph with lengths on edges
Output: shortest tour that visits each vertex exactly once
An optimization problem
Best known algorithm: O(n 2n) time.
Intractable

4. Comparison of functions
log2n n
nlog2n n2 n3 2n n! 10
3.3 33
102
103
106
6.6
660
104
1030
10158
109 13
105
108
1012 17
1010
1015 20
107
1018
For a super computer that does 1 trillion operations per second, it will be longer than 1 billion years
2/24/2019

5. Another difficult problem
Maximum Clique:
Input: undirected graph G=(V,E)
Output: largest subset C of V such that every pair of vertices in C has an edge between them
An optimization problem
Best known algorithm: O(n 2n) time

6. What can we do ?
Spend more time designing algorithms for those problems
People tried for a few decades, no luck
Prove there is no polynomial time algorithm for those problems
Would be great
Seems really difficult

7. What else can we do ?
Show that those hard problems are essentially equivalent. i.e., if we can solve one of them in polynomial time, then all others can be solved in polynomial time as well.
Works for at least hard problems

8. The benefits of equivalence
Combines research efforts
If one problem has polynomial time solution, then all of them do
More realistically: Once an exponential lower bound is shown for one problem, it holds for all of them P1 P2 P3

9. Summing up
If we show that a problem ∏ is equivalent to ten thousand other well studied problems without efficient algorithms, then we get a very strong evidence that ∏ is hard.
We need to:
Identify the class of problems of interest
Define the notion of equivalence
Prove the equivalence(s)

10. Classes of problems: P and NP
Implicitly, we mean decision problems: answer YES or NO
Other problems can be converted into decision problems
k-clique: is there a clique of size = k?
P is set of problems that can be solved in polynomial time
NP (nondeterministic polynomial time) is the set of problems that can be solved in polynomial time by a nondeterministic computer

11. Nondeterminism
Think of a non-deterministic computer as a computer that magically “guesses” a solution, then has to verify that it is correct
If a solution exists, computer always guesses it
One way to imagine it: a parallel computer that can freely spawn an infinite number of processes
Have one processor work on each possible solution
All processors attempt to verify that their solution works
If a processor finds it has a working solution
So: NP = problems verifiable in polynomial time

12. P and NP P = problems that can be solved in polynomial time
NP = problems for which a solution can be verified in polynomial time
K-clique problem is in NP:
Easy to verify solution in polynomial time
Is sorting in NP?
Not a decision problem
Is sortedness in NP?
Yes. Easy to verify

13. P and NP Is P = NP? The biggest open problem in CS Most suspect not
the Clay Mathematics Institute has offered a $1 million prize for the first proof NP
P = NP? or P

14. NP-Complete Problems
We will see that NP-Complete problems are the “hardest” problems in NP:
If any one NP-Complete problem can be solved in polynomial time…
…then every NP-Complete problem can be solved in polynomial time…
…and in fact every problem in NP can be solved in polynomial time (which would show P = NP)
Thus: solve TSP in O(n100) time, you’ve proved that P = NP. Retire rich & famous.

15. Reduction The crux of NP-Completeness is reducibility
Informally, a problem ’ can be reduced to another problem  if any instance of ’ can be “easily rephrased” as an instance of , the solution to which provides a solution to the instance of ’
What do you suppose “easily” means?
This rephrasing is called transformation
Intuitively: If ’ reduces to , ’ is “no harder to solve” than 

16. Reductions: ∏’ to ∏
YES x
A for ∏ NO
YES x’
A’ for ∏’ NO

17. Reductions: ∏’ to ∏
YES
f(x’)= x f
A for ∏ NO
YES x’
A’ for ∏’ NO

18. Reduction: an example
’: Given a set of Booleans, is at least one TRUE?
: Given a set of integers, is their sum positive?
Transformation: (x1, x2, …, xn) = (y1, y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE

19. Reductions
∏’ is polynomial time reducible to ∏ ( ∏’≤p ∏) iff there is a polynomial time function f that maps inputs x’ for ∏’ into inputs x for ∏, such that for any x’
∏’(x’)=∏(f(x’))
Fact 1: if ∏P and ∏’ ≤p ∏ then ∏’P
Fact 2: if ∏NP and ∏’ ≤p ∏ then ∏’NP
Fact 3 (transitivity): if ∏’’ ≤p ∏’ and ∏’ ≤p ∏ then ∏” ≤p ∏

20. Recap We defined a large class of interesting problems, namely NP
We have a way of saying that one problem is not harder than another (∏’ ≤p ∏)
Our goal: show equivalence between hard problems

21. Showing equivalence between difficult problems
Options:
Show reductions between all pairs of problems
Reduce the number of reductions using transitivity of “≤”
TSP 
Clique P3 P4 P5 ∏’

22. Showing equivalence between difficult problems
Options:
Show reductions between all pairs of problems
Reduce the number of reductions using transitivity of “≤”
Show that all problems in NP are reducible to a fixed ∏. To show that some problem ∏’NP is equivalent to all difficult problems, we only show ∏ ≤p ∏’.
TSP 
Clique ∏ P3 P4   P5 ∏’

23. The first problem ∏ Boolean satisfiability problem (SAT):
Given: a formula φ with m clauses over n variables, e.g., x1v x2 v x5 , x3 v ¬ x5
Check if there exists TRUE/FALSE assignments to the variables that makes the formula satisfiable
Any SAT can be rewritten into conjunctive normal form (CNF)
a conjunction of clauses, where a clause is a disjunction of literals
A literal is a Boolean variable or its negation
E.g. (AB)  (BCD) is in CNF
(AB)  (ACD) is not in CNF

24. SAT is NP-complete Fact: SAT NP
Theorem [Cook’71]: For any ∏’NP , we have ∏’ ≤ SAT.
Definition: A problem ∏ such that for any ∏’NP we have ∏’ ≤p ∏, is called NP-hard
Not necessarily decision problem
Definition: An NP-hard problem that belongs to NP is called NP-complete
Corollary: SAT is NP-complete.

25. Karp's 21 NP-complete problems, 1972
SAT: the boolean satisfiability problem for formulas in conjunctive normal form
0-1 INTEGER PROGRAMMING
CLIQUE (see also independent set problem)
SET PACKING
VERTEX COVER
SET COVERING
FEEDBACK NODE SET
FEEDBACK ARC SET
DIRECTED HAMILTONIAN CIRCUIT
UNDIRECTED HAMILTONIAN CIRCUIT
3-SAT
CHROMATIC NUMBER (also called the Graph Coloring Problem)
CLIQUE COVER
EXACT COVER
HITTING SET
STEINER TREE
3-dimensional MATCHING
KNAPSACK (Karp's definition of Knapsack is closer to Subset sum)
JOB SEQUENCING
PARTITION
MAX-CUT
Karp's 21 NP-complete problems, 1972
Since then thousands problems have been proved to be NP-complete

26. Clique again Clique (decision variant):
Input: undirected graph G=(V,E), K
Output: is there a subset C of V, |C|≥K, such that every pair of vertices in C has an edge between them

27. SAT ≤p Clique x’
Given a SAT formula φ=C1,…,Cm over x1,…,xn, we need to produce G=(V,E) and K, such that φ satisfiable iff G has a clique of size ≥ K.
f(x’)=x

28. SAT ≤p Clique reduction
For each literal t occurring in φ, create a vertex vt
Create an edge vt – vt’ iff:
t and t’ are not in the same clause, and
t is not the negation of t’

29. SAT ≤p Clique example
t and t’ are not in the same clause, and
t is not the negation of t’
Edge vt – vt’ 
Formula: (x1 V x2 V x3 )  ( ¬ x2 v ¬ x3)  (¬ x1 v x2)
Graph:
¬x2 x1
¬ x3 x2 x3 x2
¬ x1
Claim: φ satisfiable iff G has a clique of size ≥ m

30. Proof “→” part: Take any assignment that satisfies φ.
t and t’ are not in the same clause, and
t is not the negation of t’
Edge vt – vt’ 
“→” part:
Take any assignment that satisfies φ.
E.g., x1=F, x2=T, x3=F
Let the set C contain one satisfied literal per clause
C is a clique
¬x2 x1
¬ x3 x2 x3 x2
¬ x1

31. Proof “←” part: t and t’ are not in the same clause, and
t is not the negation of t’
Edge vt – vt’ 
“←” part:
Take any clique C of size ≥ m (i.e., = m)
Create a set of equations that satisfies selected literals.
E.g., x3=T, x2=F, x1=F
The set of equations is consistent and the solution satisfies φ
¬x2 x1
¬ x3 x2 x3 x2
¬ x1

32. Altogether We constructed a reduction that maps:
YES inputs to SAT to YES inputs to Clique
NO inputs to SAT to NO inputs to Clique
The reduction works in polynomial time
Therefore, SAT ≤p Clique →Clique NP-hard
Clique is in NP → Clique is NP-complete

33. Independent set (IS) Input: undirected graph G=(V,E)
Output: is there a subset S of V, |S|≥K such that no pair of vertices in S has an edge between them
Want to show: IS is NP-complete

34. Clique ≤ IS x’
Given an input G=(V,E), K to Clique, need to construct an input G’=(V’,E’), K’ to IS, such that G has clique of size ≥K iff G’ has IS of size ≥K’.
Construction: K’=K,V’=V,E’=E
Reason: C is a clique in G iff it is an IS in G’s complement.
f(x’)=x

35. Classes of problems