Chapter 7 Inefficiency and Intractability CS 345 Spring Quarter, 2014

Towers of Hanoi Revisited We discovered that the time complexity of the Towers of Hanoi problem is 2 N -1 The original problem had 64 rings. If one ring could be moved every 10 seconds, it would take over 5 trillion years to solve the problem. Even moving 1 million rings a second takes over 1/2 million years

The Monkey Puzzle See pages for an explanation of the puzzle Given N cards, a brute-force solution takes N! steps. Arranging 1 million cards a second still takes over 490 billion years for 25 cards.

Reasonable vs. Unreasonable Time The factorial function grows very rapidly. Factorial grows even faster than 2 N. One of the fastest growing functions we know is Ackerman’s Function. For comparison, 2 slides that follow reproduce figures 7.3 (page 163) and 7.5 (page 167)

Ackerman’s Function A(1, j) = 2 j, for j  1 A(i, 1) = A(i-1, 2) for i  2 A(i, j) = A( i -1, A( i, j-1)) for i, j  2 How fast does this function grow? Note that A(i, j+1) > A( i,j) ; A(i+1,j) > A(i,j) and A(3,4) = 2^2^2 …^2 (a stack of 65,536 2’s) 65,536

N Func. 5N N LogN N ,600 10,000 90,000 1 mil N 3 8, ,000 1 mil 27 mil 1 bil 2 N digit 31 digit 91 digit 302 digit N! 19 digit 82 digit 161 digit 623 digit big N N 27 digit 107 digit 201 digit 744 digit real big

N f(N) N 2 1/2500ms 1/625ms 1/278ms 1/100ms 1/11ms N 3 1/300s 1/10s 78/100s 10s 40.5m 2 N 1/1000s 18.3m 36.5Y 400bilC 72digC N N 3.3bilY 46digC 89digC 182digC 725digC Figure 7.5, page 164 assume 1 instruction per nanosecond

Tractable Algorithms Examining the previous two slides indicate that some algorithms might be reasonable and others not reasonable. Algorithms whose run time is bounded from above by a polynomial function of N are called polynomial-time algorithms, are said to be reasonable, and are called tractable.

Intractable Algorithms An algorithm which has an exponential or super-exponential upper bound is said to be unreasonable and is called intractable. Note that an intractable algorithm has a solution. It just takes an unreasonably long time to find it.

Decision Problems Perhaps part of the long running time of an algorithm is that we are looking for a solution. What if we just asked, “Does a solution exist?” A decision problem is one which has a yes or no answer.

Most problems have a decision problem version. For example: What is the minimum grade I can get on the final and still get an A in the class? The decision problem version is: If I get a (some number) on the final, will I still get an A in the class?

More on the Monkey Puzzle Page Computers are becoming faster by the week. Over the last 10 years or so computer speed has increased roughly by a factor of 50. Perhaps obtaining a practical solution to the problem is just a question of awaiting an additional improvement in computer speed.

2. Doesn’t the fact that we have not found a better algorithm for this problem indicate our incompetence at devising efficient algorithms? Shouldn’t computer scientists be working at trying to improve the situation rather than spending their time writing books about it? 3. Haven’t people tried to look for an exponential-time lower bound on the problem, so that we might have a proof that no reasonable algorithm exists?

4. Maybe the whole issue is not worth the effort, as the monkey puzzle problem is just one specific problem. It might be a colorful one, but it certainly doesn’t look like a very important one. Reply to point 1: Examine figure 7.7 on page 166. Given an exponential algorithm, an improvement in computer speed by a constant factor--however large--will make only an insignificant improvement overall.

Reply to point 4: It so happens that the “Monkey Puzzle” problem is not alone. There are close to 1000 diverse algorithmic problems which have intractable solutions. But, none are known not to have tractable solutions. In many cases, the best-known lower bounds are  (N). That is, these problems have very large algorithmic gaps

The Class NP We denote these problems as NP. These are problems that have no known tractable solution, but we cannot prove that a tractable solution does not exist.

Some NP Problems Generalized jigsaw or arrangement Graph problems –Clique –TSP (the Decision Version) –Hamiltonian cycle and path –Eulerian Path: Königsberg bridges, 1736 –Graph Colorings

Nondeterminism and Cook’s Theorem The theory of NP began with a remarkable theorem of Cook in The notion of nondeterminism is rather non- intuitive. One should think of a nondeterministic algorithm as an abstract notion, not as a realistic goal.

Nondeterminism is more important to the development of the theory and the explanation of the existence of this class of problems than it is to the techniques for using the theory. A nondeterministic algorithm has, in addition to all the regular operations of a deterministic algorithm, a very powerful oracle.

Problems solved by nondeterministic algorithms have the property that they can take an unacceptably long time to produce a solution. If a solution is arrived at, however, there is an easy way to convince someone that it is a solution. Nondeterministic algorithms are very powerful, but their power is not unlimited. Not all problems can be solved efficiently by nondeterministic algorithms.

The class of problems that have a deterministic, polynomial time algorithm that solves them is called P. The class of problems that have a nondeterministic algorithm that can be checked in polynomial time is called NP. Are nondeterministic algorithms more powerful than deterministic ones???? Its clear that P  NP, but is P  NP ??? No one has yet demonstrated an algorithm that is in NP but not in P

In contrast, to prove P = NP requires us to show that every problem in NP can be solved by a deterministic polynomial time algorithms. Nobody has done this either (and very few believe it is true) The question does P = NP? (first asked in 1971) is the most difficult open question in theoretical Computer Science.

Polynomial Reduction Algorithm X is reducible to Algorithm Y if there is a polynomial time algorithm which transforms an input to X into an input to Y in such a manner that Y always gives the same answer ( “yes” or “no”) using the transformed input as X does with the original input.

If Alg X is reducible to Alg Y then, if Y can be solved with a polynomial time deterministic algorithm, so can X. This means that X is no harder than Y Definition: A problem X is called an NP- hard problem if every problem in NP is reducible to X. This means that no problem in NP is harder than X. Definition: A problem X is called an NP- complete problem if (1) it belongs to NP and (2) it is NP-hard.

Note that from the definition of NP-hard, if any NP-hard problem is shown to be in P, then P = NP. Cook’s remarkable theorem proved that there exists NP-complete problems. In particular, he demonstrated the SAT was NP-complete. Cook’s theorem makes showing a problem is NP-complete easier.

Definition: A problem X is an NP-complete problem if (1) X belongs in NP and (2) some problem Y reduces to X where Y is already known to be in NP-complete. Because all problems in NP reduce to each problem in NP-complete, if one NP- complete problem has a polynomial-time deterministic algorithm, then P = NP.

Imperfect Solutions to NP- Complete Problems NP-complete problems are usually taken as the decision problem (yes/no) version of optimization problems. E.g. TSP But, humans manage to solve these problems all the time in everyday life, and it doesn’t take centuries. Also, the human brain is orders of magnitudes slower than today’s computers.

The human brain is massively parallel Approximation and Probabilistic Algorithms Heuristics

Provably Intractable Problems We are not able to prove that problems in NP have no tractable solution There are problems, however, such as the Towers Of Hanoi, that can be shown to have an exponential lower bound and that have no certificate. These problems belong to classes that are even harder than NP.

Complexity Classes Theoretical Computer Scientists talk about a hierarchy of complexity classes. These classes are for both time and space requirements. Dual classes or co- classes (co-NP): the yes or no answers are not the same. Is it true that p is a prime number? / Is it true that p is a composite number?

Research on Complexity Classes Some theorists work with questions such as: Does P = NP? or Does NP = co-NP? or Can we refine the complexity classes? Others work to develop approximate solutions or heuristic solutions or alternate models of computation: genetic algorithms, neural networks, etc..