1. Prof. Mario Pavone Computazione Naturale CdL Magistrale in Informatica SDAI – DMI – UniCt mpavone@dmi.unict.it http://www.dmi.unict.it/mpavone/ Teoria della Complessità: la classe NP

2. Problem A problem is a general question to be answered, usually possessing several parameters, or free variables, whose values are left unspecified. A problem is described by giving: a general description of all its parameters, and a statement of what properties the answer, or solution, is required to satisfy An instance of a problem is obtained by specifying particular values for all the problem parameters

3. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Search Space Psearch space P(S,f ) Given a combinatorial problem P, a search space associated to a mathematical formulation of P is defined by a couple (S,f ) S where S is a finite set of configurations (or nodes or points) and fcost function S f a cost function which associates a real number to each configurations of S. the minimum and the maximum costs combinatorial optimization problems For this structure two most common measures are the minimum and the maximum costs. In this case we have the combinatorial optimization problems.

4. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Example: K-SAT K-SAT problem An instance of the K-SAT problem consists of a set V of variables, a collection C of clauses over V such that each clause c  C has |c|= K The problem is to find a satisfying truth assignment for C search space for the 2-SAT(S,f ) The search space for the 2-SAT with |V|=2 is (S,f ) where S={ (T,T), (T,F), (F,T), (F,F) } S={ (T,T), (T,F), (F,T), (F,F) } and the cost function2-SAT the cost function for 2-SAT computes only the number of satisfied clauses f sat (s)= #SatisfiedClauses(F,s), s  S

5. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Example: SEARCH SPACE of K-SAT Let we consider F = (A   B)  (  A   B) A B f sat (F,s) T 1 T F 2 F T 1 F 2

6. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt SEARCH LANDSCAPE (S,f)search landscape (S,n,f)nneighborhood function Given a search space (S,f), a search landscape is defined by a triplet (S,n,f) where n is a neighborhood function which verifies n : S  2 S - {0} energy landscape neutral This landscape, also called energy landscape, can be considered as a neutral one since no search process is involved. weighted graphG=(S,n,F) the weights are defined on the nodes It can be conveniently viewed as weighted graph G=(S,n,F) where the weights are defined on the nodes, not on the edges.

7. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt PROCESS LANDSCAPE (S,n,f)process landscape (S,n,f,  )  search process Given a search landscape (S,n,f), a process landscape is defined by a quadruplet (S,n,f,  ) where  is a search process (S,n,f) The process landscape represents a particular view of the neutral landscape (S,n,f) seen by a search algorithm. Examples of search algorithms Examples of search algorithms: Local Search Algorithms Local Search Algorithms Complete Algorithms (e. g. Davis-Putnam algorithm) Complete Algorithms (e. g. Davis-Putnam algorithm) Evolutionary AlgorithmsGenetic Algorithms, Genetic Programming, Evolution Strategies, Evolution Programming, Immune Algorithms Evolutionary Algorithms: Genetic Algorithms, Genetic Programming, Evolution Strategies, Evolution Programming, Immune Algorithms

8. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt ALGORITHMS Algorithms are general, step-by-step procedures for solving problems For concreteness, we can think of them simply as being computer programs, written in some precise computer language. An algorithm is said to solve a problem P if that algorithm can be applied to any instance I of P and is guaranteed always to produce a solution for that instance I An algorithm is said to solve a problem P if that algorithm can be applied to any instance I of P and is guaranteed always to produce a solution for that instance I For example: An algorithm does not "solve" the TSP unless it always constructs an ordering that gives a minimum length tour

9. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt BIBLIOGRAPHY Computers and Intractability: A Guide to the Theory of NP-Completness M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completness, W. H. Freeman, 1st ed. (1979). A compendium of NP optimization problems at the link: OR: A compendium of NP optimization problems at the link: http://www.csc.kth.se/~viggo/problemlist/

10. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Problem: the start point PROBLEM PROBLEM: to refer to a question such as: "Is a given graph G=(V,E) K-colorable?" INSTANCE INSTANCE of a problem is a list of arguments, one argument for each parameter of the problem a problem is a binary relationIproblem instances Sproblem solutionsQ: I → S a problem is a binary relation on a set I of problem instances, and a set S of problem solutions: Q: I → S decision problemsQ: I → {0, 1} decision problems: Q: I → {0, 1} in general, if we can solve an optimization problem quickly, we can also solve its related decision problem quickly in general, if we can solve an optimization problem quickly, we can also solve its related decision problem quickly

11. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Problem: the start point UNDECIDABLE "yes""no" a problem is UNDECIDABLE if there is no algorithm that takes as input an instance of the problem and determines whether the answer to that instance is "yes" or "no" solvable by polinomial-time algorithms TRACTABLEsuperpolynomial time INTRACTABLE problems that are solvable by polinomial-time algorithms as being TRACTABLE; ones that require superpolynomial time as being INTRACTABLE

12. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Problem: the start point NP-complete problems are intractable NP-complete problems are intractable If any single NP-complete problem can be solved in polynomial time then every NP-complete problem has a polynomial-time algorithm If any single NP-complete problem can be solved in polynomial time then every NP-complete problem has a polynomial-time algorithm its intractability If you can establish a problem as NP-complete you provide good evidence for its intractability developing an approximation algorithm You would then do better spending your time developing an approximation algorithm rather than searching for a fast algorithm that solves the problem exactly Polynomial-time solvable problems are generally regarded as tractable Polynomial-time solvable problems are generally regarded as tractable

13. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Complexity Class P An algorithm solves a problem in time O(T(n)) problem instance i of length n=|i|the algorithm can produce the solution in at most O(T(n)) time An algorithm solves a problem in time O(T(n)) if, when it is provided a problem instance i of length n=|i|, the algorithm can produce the solution in at most O(T(n)) time problem is polynomial-time solvablean algorithm to solve it in time O(n k ) A problem is polynomial-time solvable if there exists an algorithm to solve it in time O(n k ) for some constant k COMPLEXITY CLASS P COMPLEXITY CLASS P: the set of decision problems that are solvable in polynomial time function f is polynomial-time computable Ax  I f(x) A function f is polynomial-time computable if there exists a polynomial-time algorithm A that, given any input x  I, produces as output f(x)

14. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Example: problem in P Minimum Weight Spanning Tree. Input: Graph G, integer k. Does G have a spanning tree of weight at most k? Decision problem: Does G have a spanning tree of weight at most k? If you are provided with a tree with weight at most k as part of the solution, the answer can be verified in O(n 2 ) time.

15. Polynomial-Time Algorithms Are some problems solvable in polynomial time? Of course: every algorithm we’ve studied (Algorithms 1) provides polynomial-time solution to some problem We define P to be the class of problems solvable in polynomial time Are all problems solvable in polynomial time? No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given Such problems are clearly intractable, not in P

16. Complexity Class NP Some problems are intractable: as they grow large, we are unable to solve them in reasonable time What constitutes reasonable time? Standard working definition: polynomial time On an input of size n the worst-case running time is O(n k ) for some constant k Polynomial time: O(n 2 ), O(n 3 ), O(1), O(n lg n) Not in polynomial time: O(2 n ), O(n n ), O(n!)

17. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Complexity Class NP decision problemclass NP A decision problem (yes/no question) is in the class NP if it has a nondeterministic polynomial time algorithm Informally, such an algorithm: 1. Guesses a solution (nondeterministically). 2. Checks deterministically in polynomial time that the answer is correct. Or equivalently, when the answer is "yes", there is a certificate (a solution meeting the criteria) that can be verified in polynomial time (deterministically) Complexity Class NPcan be verified by a polynomial-time algorithm The Complexity Class NP is the class of problems that can be verified by a polynomial-time algorithm Q  PQ  NPP  NP If Q  P then Q  NP => P  NP Open question is: P = NP Open question is: P = NP

18. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Complexity Classes: P and NP P P is set of problems that can be solved in polynomial time NP NP (Nondeterministic Polynomial time) is the set of problems that can be solved in polynomial time by a nondeterministic computer What the hell is that?

19. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Nondeterminism Think of a non-deterministic computer as a computer that magically “guesses” a solution, then has to verify that it is correct If a solution exists, computer always guesses it One way to imagine it: a parallel computer that can freely spawn an infinite number of processes Have one processor work on each possible solution All processors attempt to verify that their solution works If a processor finds it has a working solution So: NP = problems verifiable in polynomial time

20. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Review: P and NP Summary so far: P = problems that can be solved in polynomial time NP = problems for which a solution can be verified in polynomial time Unknown whether P = NP (most suspect not) Hamiltonian-cycle problem is in NP: Cannot solve in polynomial time Easy to verify solution in polynomial time (How?)

21. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Hamiltonian Cycle Problem A hamiltonian cycle of an undirected graph G=(V,E) is a simple cycle that contains each vertex in V. A graph that contains a hamiltonian cycle is said to be hamiltonian NOT ALL GRAPHS ARE HAMILTONIAN Hamiltonian-cycle problem: “Does a graph G have a hamiltonian cycle?”

22. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Hamiltonian Cycle Problem Hamiltonian Cycle Hamiltonian Cycle (input: a graph G) Does G have a Hamiltonian cycle? Does G have a Hamiltonian cycle? Solution: 0, 1, 2, 11, 10, 9, 8, 7, 6, 5, 14, 15, 6, 17, 18, 19, 12, 13, 3, 4

23. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt Hamiltonian Cycle Problem algorithm: lists all permutations of the vertices and check each permutation to see if it is a hamiltonian path there are m! Possible permutations of the vertices CHECK if the provided cycle is hamiltonian: if it is a permutation of the vertices of V and each consecutive edges along the cycle exists in the graph This verification can be implemented to run in O(n 2 ) time => a hamiltonian cycle exists in a graph can be verified in polynomial time N is the lenght of the econding of G

24. NP-Complete Problems The NP-Complete problems are an interesting class of problems whose status is unknown No polynomial-time algorithm has been discovered for an NP- Complete problem No suprapolynomial lower bound has been proved for any NP- Complete problem, either We call this the P = NP question The biggest open problem in CS

25. Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt NP Complete Problems problems in NP which are the “hardest” NP-complete problems The class of problems in NP which are the “hardest” are called the NP-complete problems QNPNP-complete Qexistence of a polynomial time algorithm for all problems in NP A problem Q in NP is NP-complete if the existence of a polynomial time algorithm for Q implies the existence of a polynomial time algorithm for all problems in NP Steve Cook in 1971 proved that SAT is NP-complete. Steve Cook in 1971 proved that SAT is NP-complete.

26. NP-Complete Problems We will see that NP-Complete problems are the “hardest” problems in NP: If any one NP-Complete problem can be solved in polynomial time… …then every NP-Complete problem can be solved in polynomial time… …and in fact every problem in NP can be solved in polynomial time (which would show P = NP) Thus: solve hamiltonian-cycle in O(n 100 ) time, you’ve proved that P = NP. Retire rich & famous.

27. Reduction The crux of NP-Completeness is reducibility Informally, a problem P can be reduced to another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of P What do you suppose “easily” means? This rephrasing is called transformation Intuitively: If P reduces to Q, P is “no harder to solve” than Q

28. Reducibility An example: P: Given a set of Booleans, is at least one TRUE? Q: Given a set of integers, is their sum positive? Transformation: (x 1, x 2, …, x n ) = (y 1, y 2, …, y n ) where y i = 1 if x i = TRUE, y i = 0 if x i = FALSE Another example: Solving linear equations is reducible to solving quadratic equations How can we easily use a quadratic-equation solver to solve linear equations?

29. Using Reductions If P is polynomial-time reducible to Q, we denote this P  p Q Definition of NP-Complete: If P is NP-Complete, P  NP and all problems R are reducible to P Formally: R  p P  R  NP If P  p Q and P is NP-Complete, Q is also NP-Complete

30. Ex. of some NP-complete problems Given one NP-Complete problem, we can prove many interesting problems NP-Complete Graph coloring (= register allocation) Hamiltonian cycle Hamiltonian path Knapsack problem Traveling salesman Job scheduling with penalities Many, many more

31. An Aside: Terminology What is the difference between a problem and an instance of that problem? To formalize things, we will express instances of problems as strings How can we express a instance of the hamiltonian cycle problem as a string? To simplify things, we will worry only about decision problems with a yes/no answer Many problems are optimization problems, but we can often re-cast those as decision problems

32. NP-Hard and NP-Complete If P is polynomial-time reducible to Q, we denote this P  p Q Definition of NP-Hard and NP-Complete: If all problems R  NP are reducible to P, then P is NP-Hard We say P is NP-Complete if P is NP-Hard and P  NP If P  p Q and P is NP-Complete, Q is also NP- Complete

33. Why Prove NP-Completeness? Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractable Therefore it can be important to prove that a problem is NP- Complete Don’t need to come up with an efficient algorithm Can instead work on approximation algorithms

34. Proving NP-Completeness What steps do we have to take to prove a problem P is NP-Complete? Pick a known NP-Complete problem Q Reduce Q to P Describe a transformation that maps instances of Q to instances of P, s.t. “yes” for P = “yes” for Q Prove the transformation works Prove it runs in polynomial time Oh yeah, prove P  NP (What if you can’t?)

35. Review: P And NP Summary P = set of problems that can be solved in polynomial time NP = set of problems for which a solution can be verified in polynomial time P  NP Open question: Does P = NP?

36. Review: Reduction A problem P can be reduced to another problem Q if any instance of P can be rephrased to an instance of Q, the solution to which provides a solution to the instance of P This rephrasing is called a transformation Intuitively: If P reduces in polynomial time to Q, P is “no harder to solve” than Q

37. Review: NP-Hard and NP-Complete If P is polynomial-time reducible to Q, we denote this P  p Q Definition of NP-Hard and NP-Complete: If all problems R  NP are reducible to P, then P is NP-Hard We say P is NP-Complete if P is NP-Hard and P  NP If P  p Q and P is NP-Complete, Q is also NP- Complete

38. Review: Proving NP-Completeness What steps do we have to take to prove a problem Q is NP-Complete? Pick a known NP-Complete problem P Reduce P to Q Describe a transformation that maps instances of P to instances of Q, s.t. “yes” for Q = “yes” for P Prove the transformation works Prove it runs in polynomial time Oh yeah, prove Q  NP (What if you can’t?)

39. Directed Hamiltonian Cycle  Undirected Hamiltonian Cycle What was the hamiltonian cycle problem again? For my next trick, I will reduce the directed hamiltonian cycle problem to the undirected hamiltonian cycle problem before your eyes Which variant am I proving NP-Complete? Draw a directed example on the board What transformation do I need to effect?

40. Transformation: Directed  Undirected Ham. Cycle Transform graph G = (V, E) into G’ = (V’, E’): Every vertex v in V transforms into 3 vertices v 1, v 2, v 3 in V’ with edges (v 1,v 2 ) and (v 2,v 3 ) in E’ Every directed edge (v, w) in E transforms into the undirected edge (v 3, w 1 ) in E’ (draw it) Can this be implemented in polynomial time? Argue that a directed hamiltonian cycle in G implies an undirected hamiltonian cycle in G’ Argue that an undirected hamiltonian cycle in G’ implies a directed hamiltonian cycle in G

41. Undirected Hamiltonian Cycle Thus we can reduce the directed problem to the undirected problem What’s left to prove the undirected hamiltonian cycle problem NP- Complete? Argue that the problem is in NP

42. Hamiltonian Cycle  TSP The well-known traveling salesman problem: Optimization variant: a salesman must travel to n cities, visiting each city exactly once and finishing where he begins. How to minimize travel time? Model as complete graph with cost c(i,j) to go from city i to city j How would we turn this into a decision problem? A: ask if  a TSP with cost < k

43. Hamiltonian Cycle  TSP The steps to prove TSP is NP-Complete: Prove that TSP  NP (Argue this) Reduce the undirected hamiltonian cycle problem to the TSP So if we had a TSP-solver, we could use it to solve the hamilitonian cycle problem in polynomial time How can we transform an instance of the hamiltonian cycle problem to an instance of the TSP? Can we do this in polynomial time?

44. The TSP Random asides: TSPs (and variants) have enormous practical importance E.g., for shipping and freighting companies Lots of research into good approximation algorithms Recently made famous as a DNA computing problem