1. CSE 3813 Introduction to Formal Languages and Automata Chapter 14 An Introduction to Computational Complexity These class notes are based on material from our textbook, An Introduction to Formal Languages and Automata, 4th ed., by Peter Linz, published by Jones and Bartlett Publishers, Inc., Sudbury, MA, 2006. They are intended for classroom use only and are not a substitute for reading the textbook.

2. Additional notes on complexity (Acknowledgement: These notes are derived from material developed by Dr. Susan Bridges and Dr. Lois Boggess, and from the course textbook for CS 8833.)

3. Algorithmics and Computational Complexity Algorithmics is the study of the performance characteristics of specific algorithms. Computational Complexity (also called Computability Theory) looks at the characteristics of problems, rather than algorithms.

4. Solvable and unsolvable We make a distinction between solvable and unsolvable problems. Within the solvable problems, we make a three-way distinction. So the subdivision looks like this:

5. Solvable and unsolvable Unsolvable problems Example: The halting problem Solvable problems Provably intractable problems (also called infeasible problems) Probably intractable problems Tractable problems (feasible problems)

6. Solvable and unsolvable Example of a provably intractable (infeasible) problem: Generalized chess - Chess is played on a board of size 8 x 8; generalized chess is played on a board of size N x N, with the set of pieces and the set of moves adjusted as needed. It has been proven to have an exponential lower bound.

7. Does P = NP? There is a question that has been unanswered in spite of a lot of work in computational theory: Does P = NP? Some related questions: What are the meanings of the terms, P, NP, NP- complete, and NP-hard? Why do we care whether P = NP? What is the best guess as to whether P = NP? What are the consequences, if that best guess is right?

8. Polynomial time algorithms What is a polynomial time algorithm? Traditional answer: A polynomial time algorithm is one that can be executed by a standard (deterministic) Turing machine in polynomial time, proportional to the size of the input. Easier to understand answer: A polynomial time algorithm is one in which the worst case running time is O(n k ) where n is the input size and k is a constant.

9. Polynomial time algorithms Can all problems be solved in polynomial time, if we just make k large enough? Answer: No. It can be proved that some problems that we can describe can’t be solved at all. Some problems can be proven to require at least exponential time. There is an important class of problems that “probably” require exponential time.

10. Polynomial time algorithms Are algorithms that are polynomial time considered efficient? Answer: Algorithms are considered efficient if there is a polynomial p(n) such that the algorithm can solve any instance of the problem of size n in O(p(n)) time. Of course, in reality, the efficiency of a polynomial-time algorithm also depends on:

11. Polynomial time algorithms the constant of proportionality that is being ignored in the big-O formalism. If very large, the algorithm still isn’t efficient, for any practical purposes the size of the exponent of the polynomial. An algorithm whose exponent is 50 wouldn’t be efficient in practice the size of n itself. Very, very large inputs make even n 2 algorithms unattractive.

12. What’s so great about polynomial time? What’s so great about polynomial time, then? A problem that is  (n 50 ) wouldn’t really be efficient, but polynomial time problems with that high an exponent are rare. Usually the exponents are small.

13. What’s so great about polynomial time? For most reasonable models of computation, a problem that can be solved in polynomial time in one model can be solved in polynomial time in another. So, for example, a problem that is polynomial in time on a serial random access machine is typically also polynomial on a Turing machine as well as on a parallel computing system, and vice versa.

14. What’s so great about polynomial time? The class of polynomial time problems has some nice closure properties: polynomial-time algorithms are closed under addition (f(n) + g(n)), multiplication (f(n)*g(n)), and composition (g(f(n)).

15. Decision problems What are decision problems? In the theory of NP-Completeness, we concentrate on one particular type of problem – decision problems. We think of each instance of a given problem as mapping onto a solution space that consists of {yes, no} or {0,1}. This may seem like a severe restriction on the type of problems that we will consider, but it is not. For example, optimization problems are fairly easy to restate as decision problems. And there exists a proof that if the decision problem is easy, then the optimization problem is easy.

16. Optimization problems Example optimization problem - A shortest path problem: Given a graph G =, two vertices, u, v  V, and a non-negative integer k, does a path exist in G between u and v whose length is k or less? Let i = (G, u, v, k) be an instance of the problem. Then the shortest path “question” with a yes/no answer becomes Path(i) = 1 or Path(i) = 0, where 1 = yes and 0 = no.

17. Optimization problems To change an optimization problem into a decision problem, we can usually just impose a bound on the value to be optimized. Examples from path problems include minimization problems (Is the length of the shortest path at most k?) and maximization problems (Is the length of the longest path at least k?).

18. Optimization problems Usually, if the optimization problem can be solved quickly, the decision problem can also be solved quickly. Conversely, evidence that the decision problem is hard normally is evidence that the optimization problem is also hard.

19. Abstract problems What are abstract problems? An abstract problem Q is a binary relation on a set I of problem instances and a set S of problem solutions.

20. Encoding What is an encoding? An encoding of a set S of abstract objects is a mapping e from S to the set of binary strings.

21. Concrete problems What is a concrete problem? A concrete problem is a problem whose instance is the set of binary strings.

22. What is P? The complexity class P is the set of concrete decision problems that are solvable in polynomial time. Alternately, P is the class of languages that are accepted in polynomial time by a standard Turing machine.

23. What is P? (The limitation to concrete, rather than abstract, decision problems is not a big limitation, because abstract decision problems are mapped into concrete problems by an encoding. Care must be taken for this step, however, because a bad encoding can make a problem much more difficult. We will ignore the difference between abstract and concrete, and assume that any encodings required were implemented reasonably well.)

24. The class NP What is a verification algorithm? A verification algorithm is a two-argument algorithm A, in which one argument is an input string x and the other is a binary string y called a certificate. The verification algorithm A verifies an input string x if there exists a certificate y such that A(x, y) = 1. The algorithm A verifies a language L if, for any string x  L, there is a certificate y that A can use to prove that x  L. For example, in the Hamiltonian cycle problem, the certificate is a list of vertices that comprise the Hamiltonian cycle.

25. The class NP The complexity class NP is the class of languages that can be verified by a polynomial time algorithm. Alternatively, a language L belongs to NP iff there exists a two-input polynomial-time algorithm A and a constant c such that L is a set of binary strings for which there exists a certificate y with |y| = O(|x| c ) such that A(x, y) = 1.

26. The class NP What does the term NP mean? NP stands for Nondeterministic Polynomial. That is, a nondeterministic Turing machine can solve it in polynomial time, but no known deterministic Turing machine can.

27. The class NP When does a problem belong to the class NP? A problem belongs to the class NP under the following conditions: Given that an oracle can guess an answer to the problem (provide a certificate),. then we can find an algorithm that can verify that the answer is correct in polynomial time.

28. The class NP Note that the problem is NP. It is the verifying algorithm that is P. Note also that the class of problems solvable in polynomial time (P) is a subset of NP by definition.

29. Example of an NP problem. Consider the question of whether a given graph has a Hamiltonian cycle (a simple cycle that visits every vertex in the graph). This question is NP as we show below. Given a certificate (a list of vertices that purports to be a Hamiltonian cycle in the graph): check to see if the first and last vertex on the list are the same check that every vertex in the graph is on the list check that every vertex in the list is connected by an edge of the graph to the next vertex on the list. check to see that no vertex other than the first/last vertex is repeated.

30. NP-Complete problems What is the class of NP-Complete problems? In some sense, the NP-complete problems are the “hardest” problems in the class NP. No polynomial time algorithm is known to solve any of them. Every NP-Complete problem is reducible to every other NP-complete problem in polynomial time.

31. NP-Complete problems What do we mean by reducible, in the previous sentence? A decision problem Q is polynomial-time reducible to decision problem Q' if a polynomial time algorithm can be developed which changes each instance of problem Q into an instance of problem Q' such that if the answer for an instance of Q' is yes, the answer to the corresponding instance of Q is yes. Then we write:

32. NP-Complete problems Lemma 1: If Q 1 and Q 2 are problems such that then Q 2  P implies Q 1  P. Proof: Suppose A 1 is a polynomial time algorithm that reduces instances of Q 1 to instances of Q 2 and that A 2 is a polynomial time algorithm for problem Q 2. Then the composition of A 1 and A 2 is a polynomial time algorithm for Q 1.

33. NP-Complete problems Definition of NP-Complete: A problem Q is NP-Complete if : Q is an element of the class NP for every Q' in NP

34. NP-Hard problems Definition of NP-Hard: A problem Q is NP-Hard if : for every Q' in NP

35. NP-Complete problems Theorem: If any NP-complete problem is polynomial- time solvable, then P = NP. If any problem in NP is not polynomial-time solvable, then all NP-complete problems are not polynomial-time solvable

36. P = NP??? Proof of the first statement: Let Q be NP-complete and polynomial. Then for all Q' in NP,, and by a previous Lemma it follows that Q' is in P.

37. P = NP??? Proof of the second statement: Let Q be NP but not polynomial. By definition of NP-complete, for all Q' in the set of NP-complete problems. It follows that all Q'  P. (Suppose not. Then there exists some Q' in P, but all elements of NPC, including Q', so Q is in P by a previous Lemma, which contradicts our initial assumption that Q was not in P.)

38. Conclusion It can be proved that some problems that we can describe can’t be solved at all. These are Unsolvable problems. Example: The halting problem.

39. Conclusion Of the problems that can be solved: some problems can be proven to require at least exponential time. These are infeasible or provably intractable problems. There is an important class of problems that “probably” require exponential time. These are probably intractable or NP problems. The problems that we prefer to work with require only polynomial time. These are feasible, tractable, or P problems.