Eric Allender Rutgers University The Audacity of Computational Complexity Theory

Eric Allender: The Audacity of Computational Complexity Theory < 2 >< 2 > Today’s Message  Computational complexity theory is successful because of some audacious notions.

Eric Allender: The Audacity of Computational Complexity Theory < 3 >< 3 > The Case against Complexity Theory  Exhibit #1: The P vs NP question. – If you’ve heard anything at all about computational complexity theory, this is what you’ve heard about.

Eric Allender: The Audacity of Computational Complexity Theory < 4 >< 4 >  Polynomial time – Time n 100 is not feasible.  Turing Machines – Clearly an unrealistic model, even for deterministic machines.  Asymptotic Analysis – An example will illustrate an important point. Three Absurd Notions in Complexity Theory

Eric Allender: The Audacity of Computational Complexity Theory < 5 >< 5 > An example: the Game of Go  Computing strategies for Go requires exponential time. – More precisely, given an n -by- n Go board with tiles on it, no program can compute an optimal next move in fewer than c 2 n – d steps, for some constants c and d. – Thus any program solving this problem must run very slowly on large inputs. This is the essence of asymptotic analysis.

Eric Allender: The Audacity of Computational Complexity Theory < 6 >< 6 > An example: the Game of Go  Computing strategies for Go requires exponential time. – More precisely, given an n -by- n Go board with tiles on it, no program can compute an optimal next move in fewer than c 2 n – d steps, for some constants c and d. – This is a much stronger statement about complexity than we are able to prove for most problems.

Eric Allender: The Audacity of Computational Complexity Theory < 7 >< 7 > An example: the Game of Go  Computing strategies for Go requires exponential time. – More precisely, given an n -by- n Go board with tiles on it, no program can compute an optimal next move in fewer than c 2 n – d steps, for some constants c and d. – Conceivably, there is a linear-time program that computes optimal moves, even for Go boards of size 1000-by-1000!

Eric Allender: The Audacity of Computational Complexity Theory < 8 >< 8 > Conclusions  Complexity Theory is a failure.  For most problems of interest, no proofs of infeasibility are known.  Even for problems where we can prove that exponential time is required (such as Go), we can’t conclude anything about input sizes that are actually of practical interest.  Computational complexity theory is a huge waste of time, and we should all go home now.

Eric Allender: The Audacity of Computational Complexity Theory < 9 >< 9 > Wrong Conclusions  Complexity Theory is a failure.  For most problems of interest, no proofs of infeasibility are known.  Even for problems where we can prove that exponential time is required (such as Go), we can’t conclude anything about input sizes that are actually of practical interest.  Computational complexity theory is a huge waste of time, and we should all go home now.

Eric Allender: The Audacity of Computational Complexity Theory An Example of the Type of Theorem that We Need  Theorem: Any circuit that takes as input a logical formula (in WS1S) of length 616 and produces as output a correct answer, saying if the formula is valid or not, has at least gates. (Stockmeyer, 1974)  Corollary: No program can run on any computer existing today, and solve this problem with a running time of less than 1000 years.

Eric Allender: The Audacity of Computational Complexity Theory Possible Objections  Theorem: Any circuit that takes as input a logical formula (in WS1S) of length 616 and produces as output a correct answer, saying if the formula is valid or not, has at least gates. (Stockmeyer, 1974)  Is this a problem anyone would ever want to compute? Yes! It’s used in hardware verification.

Eric Allender: The Audacity of Computational Complexity Theory Possible Objections  Theorem: Any circuit that takes as input a logical formula (in WS1S) of length 616 and produces as output a correct answer, saying if the formula is valid or not, has at least gates. (Stockmeyer, 1974)  Isn’t this technology-dependent? Slightly. It’s based on AND and OR gates.

Eric Allender: The Audacity of Computational Complexity Theory Possible Objections  Theorem: Any circuit that takes as input a logical formula (in WS1S) of length 616 and produces as output a correct answer, saying if the formula is valid or not, has at least gates. (Stockmeyer, 1974)  Aha!! What about quantum computers? Essentially the same result holds. Some of the constants will be slightly different.

Eric Allender: The Audacity of Computational Complexity Theory No Possible Objections!  Theorem: Any circuit that takes as input a logical formula (in WS1S) of length 616 and produces as output a correct answer, saying if the formula is valid or not, has at least gates. (Stockmeyer, 1974)  Although this theorem is about finite functions, the proof is in terms of functions on infinite domains. (That is, it uses asymptotic analysis.)

Eric Allender: The Audacity of Computational Complexity Theory Two Parts to the Proof  Lemma: There is a problem A computable in space 2 n that requires circuits of size nearly 2 n. – Diagonalization is good for creating monsters.  Small circuits for the validity problem yield small circuits for A. – Reducibility An easy-to-compute function f such that x is in A if and only if f(x) is a valid formula

Eric Allender: The Audacity of Computational Complexity Theory Two Parts to the Proof  Lemma: There is a problem A computable in space 2 n that requires circuits of size nearly 2 n. – Diagonalization is good for creating monsters.  Small circuits for the validity problem yield small circuits for A. – Every problem computable in space 2 n is reducible to the validity problem.

Eric Allender: The Audacity of Computational Complexity Theory Reductions: A Reason to Celebrate  Reducibility is a shockingly effective tool for understanding the complexity of real-world computational problems.  One reason that we are here today, is to celebrate the discovery of this fact.  Let’s learn more about reductions.

Eric Allender: The Audacity of Computational Complexity Theory Desirable Properties for Reductions  Reductions should be “easy” functions.  If f and g are easy to compute, then it should also be easy to compute f(g(x)).  If f is computable in time n 2, then f is easy. But this implies that there are “easy” functions that time n 100,000 to compute!

Eric Allender: The Audacity of Computational Complexity Theory Desirable Properties for Reductions  Reductions should be “easy” functions.  If f and g are easy to compute, then it should also be easy to compute f(g(x)).  If f is computable in time n 2, then f is easy. We have two choices: 1. Forget about our properties above, or 2. Embrace them, and claim that they make sense.

Eric Allender: The Audacity of Computational Complexity Theory A Justification for Polynomial Time  Complexity theoreticians aren’t trying to show that things are easy; they’re trying to show that things are hard.  If a problem cannot be solved in polynomial time, then this is a strong indication that it is not feasible to compute.  A “Karp reduction” is a function f computable in polynomial time.

Eric Allender: The Audacity of Computational Complexity Theory Karp Reductions  Recall that if f reduces A to B, then A is “no harder than” B.  Say that A and B are “equivalent” if A is reducible to B and vice-versa.  Equivalent problems have roughly the same complexity. They are two different ways of looking at the same problem.  Hundreds of computational problems have been studied. Surprisingly, they fall into a very small number of equivalence classes.

Eric Allender: The Audacity of Computational Complexity Theory Checkers & Go  Checkers and Go are equivalent.  Fact: Optimal strategies can be computed in time approximately 2 n.  Fact: Every problem computable in time 2 poly(n) is reducible to Go.  Hence this is a very special equivalence class. It consists of all of the “hardest” problems in EXP (the class of problems computable in exponential time).

Eric Allender: The Audacity of Computational Complexity Theory Checkers & Go  Checkers and Go are equivalent.  Fact: Optimal strategies can be computed in time approximately 2 n.  Fact: Every problem computable in time 2 poly(n) is reducible to Go.  We say these are “complete for EXP”.  Since some problems in EXP require time 2 n, we know that the complete problems also require (nearly) this much time.

Eric Allender: The Audacity of Computational Complexity Theory Completeness  Fact: Another big equivalence class is complete for PSPACE (the class of problems that can be computed using a polynomial amount of memory).  If any problem in PSPACE requires exponential time to compute, then all of these PSPACE-complete problems do.

Eric Allender: The Audacity of Computational Complexity Theory Completeness  Fact: Another big equivalence class is complete for PSPACE (the class of problems that can be computed using a polynomial amount of memory).  A quick program solving any PSPACE- complete problem gives a general technique to speed up any memory-limited computation.  Thus, we guess that all PSPACE-complete problems require (nearly) exponential time.

Eric Allender: The Audacity of Computational Complexity Theory Completeness, continued  Some equivalence classes correspond to time and space complexity.  …but most don’t!  …and they are the classes with the problems we really care the most about.

Eric Allender: The Audacity of Computational Complexity Theory Nondeterminism  The solution: Consider weird machines.  Given an input x, a nondeterministic machine is allowed to search for free through an exponentially-large set of strings, looking for a proof that x should be accepted.  You may object: “This is absurd. No machine like this can be built!”  …That’s the point!

Eric Allender: The Audacity of Computational Complexity Theory NP  NP is the class of problems solvable in polynomial time on nondeterministic machines.  A huge number of important computational problems are NP-complete: – The traveling salesman problem – The 3-colorability problem – Partition – Knapsack …  Nondeterminism is forced on us.

Eric Allender: The Audacity of Computational Complexity Theory coNP  What is the relationship between the following two problems? – 3-colorability – Non-3-colorability  They have the same (deterministic) time complexity, but they seem not to be in the same Karp-equivalence class.  There are short proofs that a graph can be three-colored. –Simply present the coloring!  Is there always a short proof that a graph cannot be three-colored?

Eric Allender: The Audacity of Computational Complexity Theory coNP  What is the relationship between the following two problems? – 3-colorability – Non-3-colorability  They have the same (deterministic) time complexity, but they seem not to be in the same Karp-equivalence class.  coNP consists of the complements of problems in NP.  Nondeterminism highlights differences among problems of equivalent deterministic complexity.

Eric Allender: The Audacity of Computational Complexity Theory Other Equivalence Classes  Many other equivalence classes of important problems are captured in terms of time- and space-bounds on variants of nondeterministic machines.  Any two problems in P are in the same Karp- equivalence class. – But using more restricted reductions, a similar picture emerges inside P. – Amazingly, even with restricted reductions, the classes of complete sets for “big” complexity classes (EXP, NP, …) are essentially unchanged.

Eric Allender: The Audacity of Computational Complexity Theory Before Reducibility  There was almost nothing that we could say about the computational complexity of real- world computational problems.  For evidence of intractability, one could only point to the fact that other people had tried, and failed, to find an efficient algorithm.

Eric Allender: The Audacity of Computational Complexity Theory With Reducibility  Order is imposed on the chaos of problems.  Natural computational problems fall into a handful of equivalence classes.  We have a reason to believe that these classes are, in fact, distinct. – Time- and Space-bounded complexity classes (on different types of machines)  A theory that explains perceived differences in computational difficulty.

Eric Allender: The Audacity of Computational Complexity Theory Factoring  A good illustration of how far we’ve come.  Complexity theory has little to say about the factorization problem – Some cryptographic problems are “reducible” to factoring. – There is no machine model or complexity class for which factoring is complete.  The only real reason to think that factoring is hard, is because smart people have failed to find fast algorithms.

Eric Allender: The Audacity of Computational Complexity Theory Factoring  A good illustration of how far we’ve come.  Complexity theory has little to say about the factorization problem – RSA is “reducible” to factoring. – There is no machine model or complexity class for which factoring is complete.  This is the situation we had for almost all computational problems, before the 70’s.

Eric Allender: The Audacity of Computational Complexity Theory Factoring  A good illustration of how far we’ve come.  Complexity theory has little to say about the factorization problem – RSA is “reducible” to factoring. – There is no machine model or complexity class for which factoring is complete.  Now, the reverse holds. For almost all problems that we suspect to be difficult to compute, we can “explain” why they are hard.

Eric Allender: The Audacity of Computational Complexity Theory …but what about the problem of asymptotic analysis???  Recall that, although Go and other EXP- complete problems require exponential time, we couldn’t say anything about fixed input lengths.  In contrast, we could say something about the validity problem for inputs of length 616.

Eric Allender: The Audacity of Computational Complexity Theory Two Parts to the Proof that Validity is Hard for Inputs of size 616:  Lemma: There is a problem A computable in space 2 n that requires circuits of size nearly 2 n. – Diagonalization is good for creating monsters.  Everything computable in space 2 n is reducible to the validity problem. Can we mimic this, to show a similar result for Go?

Eric Allender: The Audacity of Computational Complexity Theory Two Parts to the Proof that Validity is Hard for Inputs of size 616:  Lemma: There is a problem A computable in space 2 n that requires circuits of size nearly 2 n. – Diagonalization is good for creating monsters.  Everything computable in space 2 n is reducible to the validity problem. Can we mimic this, to show a similar result for Go?

Eric Allender: The Audacity of Computational Complexity Theory An Attempt to Prove a Similar Infeasibility Theorem for Go.  Lemma: There is a problem A computable in sitime 2 n that requires circuits of size nearly 2 n. ….. – We don’t know how to prove this! is good for creating monsters.  Everything computable in sitime 2 n is reducible to the problem of finding optimal moves in Go. – This part goes through with no problem! Can we mimic this, to show a similar result for Go?

Eric Allender: The Audacity of Computational Complexity Theory A Central Question in Complexity Theory:  Does every problem in EXP have small circuits? – Is EXP contained in P/poly?  It would be very strange if this were true!  If NP requires large circuits, then it should be possible to prove infeasibility results for fixed input lengths for NP-complete problems.

Eric Allender: The Audacity of Computational Complexity Theory A Central Question in Complexity Theory:  Does every problem in EXP have small circuits? – Is EXP contained in P/poly?  It would be very strange if this were true!  In the next talk, Avi Wigderson will present more reasons why the circuit complexity of EXP is of such importance.

Eric Allender: The Audacity of Computational Complexity Theory Conclusions  Complexity theory does provide convincing proofs that certain transformations from input to output cannot be computed.  In order to prove that finite functions are hard to compute, we use a theoretical framework based on infinite functions.  This theoretical framework can be used to provide evidence of infeasibility, even when we cannot prove that functions are hard to compute.

Eric Allender: The Audacity of Computational Complexity Theory Conclusions  Reducibility is unreasonably effective in characterizing the complexity of problems.  With surprisingly few exceptions, most problems are complete for one of a handful of complexity classes.  A lot of exciting progress is being made. The next two talks will discuss some more recent developments.