1. NP-completeness Algorithms and Networks

2. Algorithms and Networks: NP-completeness2 Today Complexity of computational problems Formal notion of computations NP-completeness –Why is it relevant? –What is it exactly? –How is it proven? P vs. NP Some animals from the complexity zoo

3. 1 Introduction

4. Algorithms and Networks: NP-completeness4 Easy problems / Hard problems Finding the shortest simple path between vertices v and w in a given graph Determine if there is an Euler tour in a given graph Testing 2-colorability Satisfiability when each clause has two literals Finding the longest simple path between vertices v and w in a given graph Determine if there is a Hamiltonian circuit in a given graph Testing 3-colorability Satisfiability when each clause has three literals

5. Algorithms and Networks: NP-completeness5 Fast and slow Algorithms whose running time is polynomial in input size Algorithms whose running time is exponential in input size Or worse… Or in between???

6. Algorithms and Networks: NP-completeness6 EXPTIME Many problems appear not to have a polynomial time algorithm For a few, we can proof that each algorithm needs exponential time: –EXPTIME hardness, in particular generalized games (Generalized Go, Generalized Chess) For most, including many important and interesting problems, we cannot.

7. Algorithms and Networks: NP-completeness7 NP-completeness Theory shows relations and explains behavior of many combinatorial problems From many fields: –Logic –Graphs, networks, logistics, scheduling –Databases –Compiler optimization –Graphics –…

8. Algorithms and Networks: NP-completeness8 We need formalisation! Formal notion of –problem instance –decision problem –computation –running time

9. 2 Abstract and concrete problems

10. Algorithms and Networks: NP-completeness10 Different versions of problems Decision problems –Answer is yes or no Optimization problems –Answer is a number Construction problems –Answer is some object (set of vertices, function, …) Focus on decision problems Focus on decision problems

11. Abstract versus concrete problems Abstract: –Talks about graphs, logic formulas, applications,... Concrete: –Inputs are sets of strings in finite alphabet Algorithms and Networks: NP-completeness11

12. Algorithms and Networks: NP-completeness12 Abstract problem instances Computers work with bit strings Problems are described using objects: –G is a graph, … –Given a logic formula, … –Is there a clique of size at least k,... We must map objects to bit strings (or another encoding like Gödel numbers...)

13. Algorithms and Networks: NP-completeness13 Formal problem instances 1 2 3 4 5 6 78 01100010 10100000 11011100 00101000 00110000 00100010 10000101 00000010

14. Algorithms and Networks: NP-completeness14 Abstract decision problems Abstract decision problem: –Set of instances I Objects like graphs, integers, logic formulas, etc. –Subset of I: instances where the answer to the problem is YES.

15. Algorithms and Networks: NP-completeness15 Encoding / concrete problem Encoding: mapping of set of instances I to bitstrings in {0,1}* Concrete (decision) problem: –Subset of {0,1}* With encoding, an abstract decision problem maps to a concrete decision problem

16. 3 P and NP

17. Algorithms and Networks: NP-completeness17 Complexity Class P FORMAL Class of languages L, for which there exists a deterministic Turing Machine deciding whether i  L, using running time O(p(|i|)) for some polynomial p INFORMAL Class of decision problems that have polynomial time algorithms solving them

18. Algorithms and Networks: NP-completeness18 P An algorithm solves a problem –Decides if string in {0,1}* belongs to subset Time: deterministic, worst case Algorithm uses polynomial time, if there is a polynomial p such that on inputs of length n the algorithm uses at most p(n) time. –Size of input x is denoted |x|. P is the class of decision problems that have an algorithm that solves it in polynomial time

19. Algorithms and Networks: NP-completeness19 Using P for abstract problems Abstract problem (with encoding) is in P, if the resulting problem is in P In practice: encoding and corresponding concrete problem is assumed very implicitly For polynomiality, encoding does not matter! –If we can transform encodings in polynomial time to each other Details: see e.g., chapter 34 of Introduction to Algorithms

20. Algorithms and Networks: NP-completeness20 Language of a problem Decision problem as a language: –Set of all yes-instances P is the set of all languages that have a polynomial time decision algorithm

21. Algorithms and Networks: NP-completeness21 Verification algorithm Verification algorithm has two arguments: –Problem input –Certificate (“solution”) Answers “yes” or “no” Checks if 2nd argument is certificate for first argument for studied problem The language verified by the verification algorithm A is –{i | there is an c with A(i,c)= true}

22. Algorithms and Networks: NP-completeness22 Complexity Class NP Two equivalent definitions of NP Class of languages L, for which there exists a Non-Deterministic Turing Machine deciding whether i  L, using running time O(p(|i|)) Class of languages L, for which there exists a Deterministic Turing Machine verifying whether i  L, using a polynomial sized certificate c, and using running time O(p(|i|)) –This definition we use: details follow

23. Algorithms and Networks: NP-completeness23 NP Problems with polynomial time verification algorithm and polynomial size certificates Problem L belongs to the class NP, if there exists a 2-argument algorithm A, with –A runs in polynomial time –There is a constant d such that for each i, i is a yes- instance to L, if and only if there is a certificate c with A(i,c) = true |c| = O(|i| d )

24. Algorithms and Networks: NP-completeness24 Many problems are in NP Examples: Hamiltonian Path, Maximum Independent Set, Satisfiability, Vertex Cover, … Al of these have trivial certificates (set of vertices, truth assignment, …) In NP (not trivial): Integer Linear Program

25. Algorithms and Networks: NP-completeness25 P  NP If A decides L in polynomial time, then as verification algorithm, compute –B(i,c) = A(i) –“We do not need a certificate”. Famous open problem: P = NP ?? Or not??

26. 4 Reducibility

27. Algorithms and Networks: NP-completeness27 Reducibility Language L 1 is polynomial time reducible to language L 2 (or: L 1  P L 2 ), if there exists a polynomial time computable function f : {0,1}*  {0,1}* such that –For all x  {0,1}*: x  L 1 if and only if f(x)  L 2

28. Algorithms and Networks: NP-completeness28 Lemma Lemma: If L 1  P L 2 then if L 2  P, then L 1  P. Proof-idea: run an algorithm for L 2 on f(i) for input i to problem L 1. Also: If L 1  P L 2 then if L 2  NP, then L 1  NP.

29. 5 NP-completeness and the Cook-Levin theorem

30. Algorithms and Networks: NP-completeness30 NP-completeness A language L is NP-complete, if 1.L  NP 2.For every L’  NP: L’  P L A language L is NP-hard, if 1.For every L’  NP: L’  P L NP-hardness sometimes also used as term for problems that are not a decision problem, and for problems that are ‘harder than NP’

31. Algorithms and Networks: NP-completeness31 What does it mean to be NP-complete? Evidence that it is (very probably) hard to find an algorithm that solves the problem –Always –Exact –In polynomial time

32. Algorithms and Networks: NP-completeness32 CNF-Satisfiability Given: expression over Boolean variables in conjunctive normal form Question: Is the expression satisfiable? (Can we give each variable a value true or false such that the expression becomes true). –CNF: “and” of clauses; each clause “or” of variables or negations (x i or not(x j ))

33. Algorithms and Networks: NP-completeness33 Cook-Levin theorem Satisfiability is NP-complete –Most well known is Cook’s proof, using Turing machine characterization of NP.

34. Brief sketch of ideas in proof of Cook-Levin theorem Suppose problem Q is in NP Build a Turing Machine that –gets on an input tape a string (i, c), with c a certificate of size O(|i| d ) –Runs the verification algorithm A in O(|i| q ) time –d, q constants Make a logic formula that expresses “A halts in an accepting state on (i,c)” –Variables: a(t,r,s): true if at time t, the r’s symbol on the tape is symbol s b(t,a): true if the machine is at time t at state a c(t,r): true if at time t, the reading head at the tape is at position r –Lots of clauses that express that the variable act like a Turing Machine tells them to do Algorithms and Networks: NP-completeness34

35. 5 Proving that problems are NP-complete

36. Algorithms and Networks: NP-completeness36 Proving problems NP-complete Lemma 1.Let L’  P L and let L’ be NP-complete. Then L is NP-hard. 2.Let L’  P L and let L’ be NP-complete, and L  NP. Then L is NP-complete.

37. Algorithms and Networks: NP-completeness37 3-Sat 3-Sat is the special case of CNF-Satisfiability where each clause has exactly three literals Lemma: CNF-Satisfiability  P 3-Sat –Clauses with one or two literals: Use two extra variables p and q Replace 2-literal clause (x or y) by (x or y or p) and (x or y or not(p)) Similarly, replace 1-literal clause by 4 clauses –Clauses with more than three literals: Repeat until no such clauses –For (l 1 or l 2 or … l r ) add new variable t and take as replacement clauses (l 1 or l 2 or t) and (not(t) or l 3 or … or l r )

38. Algorithms and Networks: NP-completeness38 3-Sat is NP-complete Membership in NP Reduction –3-Sat is important starting problem for many NP-completeness proofs

39. Algorithms and Networks: NP-completeness39 Clique Given: graph G=(V,E), integer k Question: does G have a clique with at least k vertices? Clique is NP-complete. In NP … easy! NP-hardness: using 3-sat.

40. Algorithms and Networks: NP-completeness40 Reduction for Clique One vertex per literal per clause Edges between vertices in different clauses, except edges between x i and not(x i ) If m clauses, look for clique of size m x1x1 x1x1 x2x2 not(x 3 ) not(x 2 )x3x3 … Clause: {x 1,not(x 2 ),x 3 } Clause: {x 1,x 2,not(x 3 )}

41. Algorithms and Networks: NP-completeness41 Correctness There is a satisfying truth assignment, if and only if there is a clique with m vertices =>: Select from each clause the true literal. The corresponding vertices form a clique with m vertices. <=: Set variable x i to true, if a vertex representing x i is in the clique, otherwise set it to false. This is a satisfying truth assignment: –The clique must contain one vertex from each 3 vertices representing a clause. –It cannot contain a vertex representing x i and a vertex representing not(x i ).

42. Algorithms and Networks: NP-completeness42 Independent set Independent set: set of vertices W  V, such that for all v,w  W: {v,w}  E. Independent set problem: –Given: graph G, integer k –Question: Does G have an independent set of size at least k? Independent set is NP-complete

43. Algorithms and Networks: NP-completeness43 Independent set is NP-complete In NP. NP-hard: transform from Clique. W is a clique in G, if and only if W is an independent set in the complement of G (there is an edge in G c iff. there is no edge in G).

44. Algorithms and Networks: NP-completeness44 How do I write down this proof? Theorem. Independent Set is NP-complete. Proof: The problem belongs to NP: as certificates, we use sets of vertices; we can check in polynomial time for a set that it is a clique, and that its size is at least k. To show NP-hardness, we use a reduction from Clique. Let (G,k) be an input to the clique problem. Transform this to (G c,k) with G c the complement of G. As G has a clique with at least k vertices, if and only if G c has an independent set with k vertices, this is a correct transformation. The transformation can clearly be carried out in polynomial time. QED

45. Writing an NP-completeness proof State the theorem Proof starts (or ends) with showing that problem belongs to NP. –Sometimes (e.g., in journal paper: trivial). Only if it is. –Otherwise, explain the certificates and how / that they can be checked in polynomial time State which known NP-complete problem you transform from –Careful: do not go in the wrong direction Explain the transformation Give the proof: input to original problem is Yes-instance, if and only if transformed input is Yes-input for known NP-complete problem –Note: you need to proof this in two directions Phrase: transformation can be carried out in polynomial time, hence problem is NP-complete QED Algorithms and Networks: NP-completeness45

46. Algorithms and Networks: NP-completeness46 Vertex Cover Set of vertices W  V with for all {x,y}  E: x  W or y  W. Vertex Cover problem: –Given G, find vertex cover of minimum size = vertex cover

47. Algorithms and Networks: NP-completeness47 Vertex cover is NP-complete In NP. NP-hard: transform from independent set. W is a vertex cover in G, if and only if V-W is an independent set in G.

48. 3-coloring 3-Coloring –Given: Graph G=(V,E) –Question: Can we color the vertices with 3 colors, such that for all edges {v,w} in E, the color of v differs from the color of w 3-coloring is NP- complete Algorithms and Networks: NP-completeness48 12 3 2 1 3 2

49. Proof of NP-completeness of 3-coloring In NP: … Transformation from 3-sat: We build a graph in 3 steps: 1.Take a clique with 3 vertices true, false, C 2.Take two adjacent vertices for each variable Algorithms and Networks: NP-completeness49 truefalse C x1x1 not x 1 x2x2 not x 2 We noemen de kleuren: C, true, false

50. Continue 3.For each clause {l 1,l 2,l 3 }, take the following gadget Algorithms and Networks: NP-completeness50 true C l1l1 l2l2 l3l3

51. Finish One can show: If the formula is satisfiable, then there is a 3- coloring of G If there is a 3-coloring of G, then the formula is satisfiable In both cases, the intuition is a literal vertex is colored true, iff we take it to be true in the formula … argue transformation takes polynomial time; QED Algorithms and Networks: NP-completeness51

52. 6 Three techniques for NP- completeness proofs Algorithms and Networks: NP-completeness52

53. Algorithms and Networks: NP-completeness53 Techniques for proving NP-hardness 1)Local replacement 2)Restriction 3)Component Design

54. 6.1 Local replacement proofs

55. Algorithms and Networks: NP-completeness55 Technique 1: Local replacement Form an instance of our problem by –Taking an instance of a known NP-complete problem –Making some change “everywhere” –Such that we get an equivalent instance, but now of the problem we want to show NP-hard

56. Algorithms and Networks: NP-completeness56 Examples of Local Replacement We saw or will see: –3-Satisfiability –Independent Set –TSP –Vertex Cover

57. Example Local Replacement Planar 3-coloring –Given: Planar graph G=(V,E) –Question: Can we color the vertices of G with 3 colors, such that adjacent vertices have different colors? Plan: transform from 3-coloring –Take arbitrary graph G –Draw G on the plane –So, we possibly get some crossings –Replace the crossings by something clever Algorithms and Networks: NP-completeness57

58. Clever reduction Garey et al found the following: Algorithms and Networks: NP-completeness58 c c d d a a b b c c a a d d b b

59. Property of the gadget If a and b have the same color, we cannot color the gadget with 3 colors If c and d have the same color, we cannot color the gadget with 3 colors Otherwise, we can... Some additional details (basically, repeat the step) when an edge has more than one crossing … Algorithms and Networks: NP-completeness59

60. 7 Restriction proofs

61. Algorithms and Networks: NP-completeness61 Technique 2: Restriction Take the problem. Add a restriction to the set of instances. NOT to the problem definition! Show that this is a known NP-complete problem

62. Algorithms and Networks: NP-completeness62 Restriction: Weighted Vertex Cover Weighted vertex cover –Given: Graph G=(V,E), for each vertex v  V, a positive integer weight w(v), integer k. –Question: Does G have a vertex cover of total weight at most k? NP-complete –In NP. –NP-hardness: set all weights to 1 (VC).

63. Algorithms and Networks: NP-completeness63 Restriction: Knapsack Knapsack –Given: Set S of items, each with integer value v and integer weight w, integers W and V. –Question: is there a subset of S of weight no more than W, with total value at least V? NP-complete –In NP –NP-hardness: set all weights equal to their values (Subset sum)

64. 9 Component design proofs

65. Algorithms and Networks: NP-completeness65 Technique 3: Component design Build (often complicated) parts of an instance with certain properties Glue them together in such a way that the proof works Examples: Clique, Hamiltonian Circuit

66. Algorithms and Networks: NP-completeness66 Hamiltonian circuit Given: Graph G Question: does G have a simple cycle that contains all vertices?

67. Algorithms and Networks: NP-completeness67 NP-completeness of Hamiltonian Circuit HC is in NP. Vertex Cover  P Hamiltonain Circuit: complicated proof (component design) –Widgets –Selector vertices –Given a graph G and an integer k, we construct a graph H, such that H has a HC, if and only if G has a VC of size k.

68. Algorithms and Networks: NP-completeness68 Widget For each edge {u,v} we have a widget W uv [u,v,1] [u,v,6][v,u,6] [v,u,1]

69. Algorithms and Networks: NP-completeness69 Only possible ways to visit all vertices in widget

70. Algorithms and Networks: NP-completeness70 Selector vertices We have k selector vertices s 1, …, s k These will represent the vertices selected for the vertex cover

71. Algorithms and Networks: NP-completeness71 Connecting the widgets For each vertex v we connect the widgets of the edges {v,w}. Suppose v has neighbors x 1, …, x r : add edges {[v,x 1,6],[v,x 2,1]}, {[v,x 2,6],[v,x 3,1]}, …, {[v,x r-1,6],[v,x r,1]}.

72. Algorithms and Networks: NP-completeness72 Connecting the selector vertices to the widgets Each selector vertex is attached to the first neighbor widget of each vertex, i.e. to vertex [v,x 1,1] and to the last neighbor widget [v,x r,6] sisi Vertex in example has degree 2

73. Algorithms and Networks: NP-completeness73 Correctness of reduction Lemma: G has a vertex cover of size (at most) k, if and only if H has a Hamiltonian circuit.

74. Algorithms and Networks: NP-completeness74 Finally The reduction takes polynomial time. So, we can conclude that Hamiltonian Circuit is NP-complete.

75. Algorithms and Networks: NP-completeness75 TSP NP-completeness of TSP by local replacement: –In NP. –Reduction from Hamiltonian Circuit: Take city for each vertex Take cost(i,j) = 1 if {i,j}  E Take cost(i,j) = 0, if {i,j}  E G has HC, if and only if there is a TSP-tour of length 0. Remark: variant with triangle inequality: use weights 2, 1 and n

76. 10 Weak and strong NP-completeness Algorithms and Networks: NP-completeness76

77. Algorithms and Networks: NP-completeness77 Problems with numbers Strong NP-complete: –Problem is NP-complete if numbers are given in unary Weak NP-complete: –Problem is NP-complete if numbers are given in binary, but polynomial time solvable when numbers are given in unary

78. Algorithms and Networks: NP-completeness78 Examples Subset-sum –Given: set of positive integers S, integer t. –Question: Is there a subset of S with total sum t? Weak NP-complete. (Solvable in pseudo-polynomial time using dynamic programming: O(nt) time…) 3-Partition –Given: set of positive integers S, (integer t). –Question: can we partition S into sets of exactly 3 elements each, such that each has the same sum (t)? Strong NP-hard. t must be the sum of S divided by |S|/3 = number of groups Starting point for many reductions

79. Remark Easily made mistake: reductions from subset sum that create exponentially large instances Subgraph Isomorphism for degree 2 graphs –Given: Graphs G and H, such that each vertex in G and H has degree at most 2 –Question: Is G a subgraph of H? NP-hardness proof can be done with 3-PARTITION Algorithms and Networks: NP-completeness79

80. Subgraph isomorphism for forests Input: Forest F =(V,E), forest H = (W,F) Question: Is there an injection f: W -> V, such that for all {v,w} in F, {f(v),f(w)} is in E, i.e., is H isomorphic to a subgraph of F? Theorem: Subgraph isomorphism for forests is NP-complete Algorithms and Networks: NP-completeness80

81. Proof In NP NP-complete: transform from 3-partition Suppose we have numbers a(1), …, a(3m) with total sum mt. F is forest with m paths, each with t vertices H is forest with 3m paths, with lengths a(1), …, a(3m) Algorithms and Networks: NP-completeness81

82. 11 Some discussion

83. How to prove NP-completeness Is there an easy proof by reduction? Is there a problem that “looks like our problem” – try to do local replacement Try component design: –Good starting problems are: 3-Sat; special cases of 3-sat; 3-partition; subset sum; simple graph problems (like clique, IS, VC, Coloring) –… puzzle … Algorithms and Networks: NP-completeness83

84. Algorithms and Networks: NP-completeness84 Discussion Is P  NP? (who thinks so?) www.claymath.org/prizeproblems/pvsnp.htm : one of the millennium problems Why so hard to prove? What to do with problems that are NP-complete? Other complexity notions… NP P NP- complete

85. Algorithms and Networks: NP-completeness85 P vs NP is hard to prove P = NP? Hard to design poly algorithm... Current mathematical knowledge does not suffice to prove P != NP: –“Natural Proofs” can not separate P from NP (Razborov & Rudich, 1993) –P A = NP A, but P B != NP B for some oracles A and B, so diagonalisation can not separate P from NP (Baker, Gill, & Solovay, 1975)

86. 12 A Few Animals from The Complexity Zoo

87. Algorithms and Networks: NP-completeness87 Much more classes In Theoretical Computer Science, a large number of other complexity classes have been defined Here, we give an informal introduction to a few of the more important ones There is much, much, much more…

88. coNP Complement of a class: switch “yes” and “no” coNP: complement of problems in NP, e.g.: NOT-HAMILTONIAN –Given: Graph G –Question: Does G NOT have a Hamiltonian circuit UNSATISFIABLE –Given: Boolean formula in CNF –Question: Do all truth assignments to the variable make the formula false? Algorithms and Networks: NP-completeness88

89. Algorithms and Networks: NP-completeness89 PSPACE All decision problems solvable in polynomial space Unknown: is P=PSPACE? Savitch, 1970: PSPACE = NPSPACE –NPSPACE: solvable with non-deterministic program in polynomial time PSPACE-complete, e.g., –generalized Tic-Tac-Toe, generalized Reversi, –Some puzzles that need an exponential number of moves –Quantified Boolean formula’s (QBF):

90. Algorithms and Networks: NP-completeness90 EXPTIME Decision problems that can be solved in exponential time P is unequal EXPTIME (Stearns, Hartmanis, 1965) EXPTIME complete problems: –Generalized chess, generalized checkers, generalized go (Japanese drawing rule) –Given a Turing Machine M and integer k, does M halt after k steps?

91. Algorithms and Networks: NP-completeness91 And a few more NEXPTIME: non-deterministic exponential time EXPSPACE = NEXPSPACE: exponential space

92. Algorithms and Networks: NP-completeness92 Graph Isomorphism Discussed in another lecture Given two graphs, are they isomorphic? In NP, not known to be NP-complete; not known to be in P Several problems are equaly hard: Isomorphism-complete

93. Algorithms and Networks: NP-completeness93 NC NC: “Nicks class”, after Nick Pippinger Talks about the time to solve a problem with a parallel machine Model: we have a polynomial number of processors, that use the same memory –Variants depending on what happens when processors try to read or write the same memory location simultaneously

94. Algorithms and Networks: NP-completeness94 NC – the definition NC: decision problems that can be solved with a PRAM (Parallel Random Access Machine) with polynomial number of processors in polylogarithmic time –O( (log n) d ) for some constant d Unknown: P=NC? P-complete problems are expected not to be in NC. An example is –Linear Programming (formulated as decision problem)

95. Algorithms and Networks: NP-completeness95 Counting #P: (“Sharp-P”) Problems that outputs a number The precise definition will not be given here. Think as: “what is the number of certificates for this instance”, with polynomial checking of certificates #P-complete e.g.: –Number of satisfying truth assignments of 3SAT-formula –Number of Hamiltonian circuits in a graph –Number of perfect matchings in a given graph PP is a related class (vaguely: “decide if the number of solutions is at most given number k”)

96. On PP and #P Inference: –Given: probabilistic network, observations O, variable X, value x, value p in [0,1] –Question: Pr(X = x | O ) <= p? Decision variant of problem from course Probabilistic Reasoning Is PP-complete; variants are #P-complete PP-hard and #P-hard problems are probably not polynomial... Algorithms and Networks: NP-completeness96

97. LSPACE or L Problems can be solved with only logarithmic extra space: –You can read the input as often as you want –You may use only O(log n) extra memory E.g.:  (1) pointers to your input NL: non-deterministic logspace... Algorithms and Networks: NP-completeness97

98. Algorithms and Networks: NP-completeness98 More The Polynomial Time Hierarchy Oracles UP: unique solutions