CSE 024: Design & Analysis of Algorithms Chapter 9: NP Completeness Sedgewick Chp:40 David Luebke’s Course Notes / University of Virginia, Computer Science

25 October 2015CS 615 Design & Analysis of Algorithms2 Course Content 1.Introduction, Algorithmic Notation and Flowcharts (Brassard & Bratley, Chap. 3) 2.Efficiency of Algorithms (Brassard & Bratley, Chap. 2) 3.Basic Data Structures (Brassard & Bratley, Chap. 5) 4.Sorting (Weiss, Chap. 7) 5.Searching (Brassard & Bratley, Chap. 9) 6.Graph Algorithms (Weiss, Chap. 9) 7.Randomized Algorithms (Weiss, Chap. 10) 8.String Searching (Sedgewick, Chap. 19) 9.NP Completeness (Sedgewick, Chap. 40)

25 October 2015CS 615 Design & Analysis of Algorithms3 Lecture Content Intractability P & NP Problems Hamiltonian Path Problem Reduction NP-Hard and NP-Complete Problems The Traveling Salesman Problem Conjunctive Normal Form Clique & Vertex Cover General Comments

Tractibility Some problems are undecidable: no computer can solve them E.g., Turing’s “Halting Problem” take an Abstract Machines & Languages class Other problems are decidable, but intractable: as they grow large, we are unable to solve them in reasonable time What constitutes reasonable time? Polynomial algorithms! 25 October 2015CS 615 Design & Analysis of Algorithms4

Efficiency of Algorithms: polynomial time / non-polynomial time 25 October 2015CS 615 Design & Analysis of Algorithms5 O(1) O(n logn) O(n 2 ) O(n 3 ) O(n 10 ) O(n log n ) O(2 n ) O(n!) O(2 2 2 n ) polynomial time O(n c ) for some constant c non-polynomial time

Polynomial Time Algorithms Are there problems solvable in polynomial time? Answer is yes: every algorithm we’ve studied provides polynomial-time solution to some problems Define P: the class of problems solvable in polynomial time Can we solve all problems that are solvable in polynomial time? No: Turing’s “Halting Problem” is not solvable by any computer no matter how much time is given Such problems are clearly intractable, not in P 25 October 2015CS 615 Design & Analysis of Algorithms6

Turing’s Halting Problem Definition of algorithm a procedure always terminates and gives an answer. Formal Definition of Algorithm a Turing machine that always halts whether succeeds or fails A problem is solvable if there is an algorithm solves every instance of the given problem A problem is said to be unsolvable if no algorithm exists to solve the given problem The Halting Problem for Turing machines: “Does there exist an effective procedure for a computable function e(M) to decide for every pair (e(M),x); does M halt for x?” The Halting problem for Turing machine is unsolvable! If it were solvable There is a Turing Machine that can solve the halting problem There is another Turing Machine to solve the problem above October 2015CS 615 Design & Analysis of Algorithms7

P and NP P (Polynomial Time) the set of problems can be solved in polynomial time NP (Nondeterministic Polynomial Time) provably decidable in polynomial time on a nondeterministic computer P  NP The big question: Does P = NP? 25 October 2015CS 615 Design & Analysis of Algorithms8

Determinist-Nondeterminist Machines 25 October 2015CS 615 Design & Analysis of Algorithms9 Nondeterministic Finite State Automata Example Deterministic Finite State Automata Example

Reduction A problem P can be reduced to another problem Q if any instance of P can be rephrased to an instance of Q, This rephrasing is called a transformation Intuitively: If P reduces in polynomial time to Q, P is “no harder to solve” than Q 25 October 2015CS 615 Design & Analysis of Algorithms10

Reducibility: Example P: Given a set of Booleans, is at least one TRUE? Q: Given a set of integers, is their sum positive? Transformation: (x 1, x 2, …, x n ) = (y 1, y 2, …, y n ) where y i = 1 if x i = TRUE, y i = 0 if x i = FALSE 25 October 2015CS 615 Design & Analysis of Algorithms11

NP-Hard and NP-Complete P  p Q P is polynomial-time reducible to Q, If all problems R  NP are reducible to P, then P is NP-Hard We say P is NP-Complete if P is NP-Hard and P  NP If P  p Q and P is NP-Complete, Q is also NP- Complete 25 October 2015CS 615 Design & Analysis of Algorithms12

Using Reductions If P is polynomial-time reducible to Q, we denote this P  p Q Definition of NP-Complete: If P is NP-Complete, P  NP and all problems R are reducible to P Formally: R  p P  R  NP If P  p Q and P is NP-Complete, Q is also NP-Complete 25 October 2015CS 615 Design & Analysis of Algorithms13

Popular NP-Complete Problems Given one NP-Complete problem, we can prove many interesting problems are NP-Complete Popular Problems: Boolean satisfiability problem (Sat.) N-puzzle Knapsack problem Hamiltonian path problem Travelling salesman problem Subgraph isomorphism problem Subset sum problem Clique problem Vertex cover problem Independent set problem Dominating set problem Graph coloring problem 25 October 2015CS 615 Design & Analysis of Algorithms14

NP-Complete Problems NP-Complete problems are the “hardest” problems in NP: If any NP-Complete problem can be solved in polynomial time then every NP-Complete problems can be solved in polynomial time To show every problem in NP solve hamiltonian-cycle in O(n 100 ) time Then it is proved that P = NP 25 October 2015CS 615 Design & Analysis of Algorithms15

Hamiltonian Cycle & Graph Given a graph G=(V,E) Find a path To visit all nodes without visiting any nodes twice! A Hamiltonian cycle, also: Hamiltonian circuit, vertex tour or graph cycle a cycle that visits each vertex exactly once except the vertex which is both the start and end, and so is visited twice A graph that contains a Hamiltonian cycle is called a Hamiltonian graph. 25 October 2015CS 615 Design & Analysis of Algorithms16

The Traveling Salesman Problem The well-known traveling salesman problem: a salesman must travel to n cities, visiting each city exactly once and finishing where he begins with minimum cost! Model as complete graph with cost c(i,j) to go from city i to city j How would we turn this into a decision problem? Is there a simple cycle over cities with cost < k ? ask if  a TSP with cost < k 25 October 2015CS 615 Design & Analysis of Algorithms17 An optimal TSP tour through Germany’s 15 largest cities. It is the shortest among possible tours visiting each city exactly once

Hamiltonian Cycle  TSP How can we prove the TSP is NP- Complete? A: Prove TSP  NP; reduce the undirected hamiltonian cycle problem to TSP TSP  NP: straightforward Reduction: need to show that if we can solve TSP we can solve hamiltonian cycle problem 25 October 2015CS 615 Design & Analysis of Algorithms18

The Satisfiability (SAT) Problem One of the first problems to be proved NP-Complete was satisfiability (SAT): Given a Boolean expression on n variables, can we assign values such that the expression is TRUE? Ex: ((x 1  x 2 )   ((  x 1  x 3 )  x 4 ))  x 2 Cook’s Theorem: The satisfiability problem is NP-Complete 25 October 2015CS 615 Design & Analysis of Algorithms19

Conjunctive Normal Form A Boolean formula is in conjunctive normal form, or CNF, if it is an AND of clauses, each of which is an OR of literals Ex: (x 1   x 2 )  (  x 1  x 3  x 4 )  (  x 5 ) 3-CNF: each clause has exactly 3 distinct literals Ex: (x 1   x 2   x 3 )  (  x 1  x 3  x 4 )  (  x 5  x 3  x 4 ) Result is true if at least one literal in each clause is true 25 October 2015CS 615 Design & Analysis of Algorithms20

The 3-CNF Problem Theorem 36.10: Satisfiability of Boolean formulas in 3-CNF form is NP-Complete Proof: Nope the 3-CNF problem is it is relatively easy to reduce to others Thus by proving 3-CNF NP-Complete we can prove many seemingly unrelated problems NP-Complete 25 October 2015CS 615 Design & Analysis of Algorithms21

3-CNF  Clique Clique of a graph G a subset of vertices fully connected to each other i.e. a complete subgraph of G The clique problem: how large is the maximum-size clique in a graph? k-clique problem can we turn clique problem into a decision problem? Is the k-clique problem within NP? 25 October 2015CS 615 Design & Analysis of Algorithms22 A graph with 23 1-vertex cliques (its vertices), 42 2-vertex cliques (its edges), 19 3-vertex cliques (the light blue triangles), 2 4-vertex cliques (dark blue).

Clique  Vertex Cover A vertex cover for a graph G is a set of vertices incident to every edge in G The vertex cover problem: what is the minimum size vertex cover in G? Restated as a decision problem: does a vertex cover of size k exist in G? Thm 36.12: vertex cover is NP-Complete 25 October 2015CS 615 Design & Analysis of Algorithms23

Clique  Vertex Cover First, show vertex cover in NP Next, reduce k-clique to vertex cover The complement G C of a graph G contains exactly those edges not in G Compute G C in polynomial time G has a clique of size k iff G C has a vertex cover of size |V| - k 25 October 2015CS 615 Design & Analysis of Algorithms24

Clique  Vertex Cover Claim: If G has a clique of size k, G C has a vertex cover of size |V| - k Let V’ be the k-clique Then V - V’ is a vertex cover in G C Let (u,v) be any edge in G C Then u and v cannot both be in V’ Why? Thus at least one of u or v is in V-V’ Why? So edge (u, v) is covered by V-V’ Since true for any edge in G C, V-V’ is a vertex cover 25 October 2015CS 615 Design & Analysis of Algorithms25

Clique  Vertex Cover Claim: If G C has a vertex cover V’  V, with |V’| = |V| - k, then G has a clique of size k For all u,v  V, if (u,v)  G C then u  V’ or v  V’ or both Why? Contrapositive: if u  V’ and v  V’, Then (u,v)  E In other words, all vertices in V-V’ are connected by an edge, thus V-V’ is a clique Since |V| - |V’| = k, the size of the clique is k 25 October 2015CS 615 Design & Analysis of Algorithms26

General Comments Literally hundreds of problems have been shown to be NP-Complete Some reductions are profound, some are comparatively easy, many are easy once the key insight is given 25 October 2015CS 615 Design & Analysis of Algorithms27

Some NP-Complete Problems Subset-sum: Given a set of integers, does there exist a subset adds up to some target T ? 0-1 knapsack: when weights not just integers Hamiltonian path: Obvious Graph coloring: can a given graph be colored with k colors such that no adjacent vertices are the same color? Etc… 25 October 2015CS 615 Design & Analysis of Algorithms28

25 October 2015CS 615 Design & Analysis of Algorithms29 Chapter 9 NP Completeness