1. Eric Allender Rutgers University 27 and Still Counting: Iterated Product, Inductive Counting, and the Structure of P ImmermanFest, Vienna, July 13, 2014

2. Eric Allender: 27 and Still Counting < 2 >< 2 > Why 27?? 3 3 years ago this month, the earth shifted:

3. Eric Allender: 27 and Still Counting < 3 >< 3 > Why 27?? Let us recall the landscape prior to July, 1987.

4. Eric Allender: 27 and Still Counting < 4 >< 4 > In the Beginning (1956) …  …there was the Chomsky Hierarchy. c.e. CSL CFL Regular co-c.e. co-CFL co-CSL??

5. Eric Allender: 27 and Still Counting < 5 >< 5 > In the Beginning (1956) …  …there was the Chomsky Hierarchy. Σ01Σ01 CSL CFL Regular Π01Π01 co-CFL co-CSL??

6. Eric Allender: 27 and Still Counting < 6 >< 6 > In the very Beginning (1943) …  …there was the Arithmetic Hierarchy. Σ03Σ03 Σ02Σ02 Σ01Σ01 Π03Π03 Π01Π01 Π02Π02

7. Eric Allender: 27 and Still Counting < 7 >< 7 > …and it was good!  Alternative characterizations in terms of – Logic (Alternating quantifiers and recursive predicates) – Alternating Turing machines. – Oracle Turing machines. – FO(Halting Problem) [not really] – AC 0 -Turing reductions to the Halting Problem [not really]

8. Eric Allender: 27 and Still Counting < 8 >< 8 > AC 0 Reductions B BB  A ≤ AC° B means that there is a constant-depth circuit computing A that has the usual AND, OR, and NOT gates, and also has ‘oracle gates’ for B.

9. Eric Allender: 27 and Still Counting < 9 >< 9 > The Arithmetic Hierarchy begat  …the Polynomial Hierarchy. Σp3Σp3 Σp2Σp2 Σ p 1 =NP Πp3Πp3 coNP=Π p 1 Πp2Πp2

10. Eric Allender: 27 and Still Counting …which was also pretty good!  Alternative characterizations in terms of – Logic (Alternating quantifiers and recursive predicates) – Alternating Turing machines. – Oracle Turing machines. – FO(SAT) [not really] – AC 0 -Turing reductions to SAT [not really] – Some fairly natural complete problems at levels 2 and 3.

11. Eric Allender: 27 and Still Counting The Polynomial Hierarchy begat  …the NL Alternation Hierarchy. Σ log 3 Σ log 2 Σ log 1 =NL Π log 3 coNL=Π log 1 Π log 2

12. Eric Allender: 27 and Still Counting …and it was not so great.  Alternative characterizations in terms of – Logic [if you played with the definitions] – Alternating Turing machines. – Oracle Turing machines. – FO(GAP) – AC 0 -Turing reductions to GAP – Some fairly natural complete problems at levels 2 and 3. [Rosier]

13. Eric Allender: 27 and Still Counting So what was the problem?  You’d like NL NL to be a subclass of P.  Unfortunately, it’s NP!  So Ruzzo, Simon, and Tompa introduced “RST” relativization. (The oracle machine must work deterministically while writing a query.)  May seem artificial – but it corresponds to AC 0 - and FO-Turing reducibility.  So of course, this gives us another hierarchy:

14. Eric Allender: 27 and Still Counting The NL Oracle Hierarchy  Where is the Alternation Hierarchy? NL NL NL NL NL coNL NL NL coNL coNL NL ALH

15. Eric Allender: 27 and Still Counting The NL Oracle Hierarchy  A lovely structure? NL NL NL NL NL coNL NL NL coNL coNL NL ALH

16. Eric Allender: 27 and Still Counting The NL Oracle Hierarchy  A lovely structure? Or a fine mess? NL NL NL NL NL coNL NL NL coNL coNL NL ALH

17. Eric Allender: 27 and Still Counting And the walls came a tumblin’ down  And here’s where you expect me to mention NL=coNL…  …but this collapse happened in 1986!  In two phases: – The NL Alternation Hierarchy = L NL [Lange, Jenner, Kirsig] – The NL Oracle Hierarchy = L NL [Schöning, Wagner][Buss, Cook, Dymond, Hay]

18. Eric Allender: 27 and Still Counting The NL “Hierarchy”  But true enlightenment had not yet arrived.  Within the year, the world would know that NL=coNL. NL coNL L NL

19. Eric Allender: 27 and Still Counting Impact of Inductive Counting  The discovery that NL=coNL provided the single most significant insight into the nature of space-bounded computation since the 60’s.  The list of complexity classes that have been impacted by these new insights into nondeterminism includes – LogCFL, VP, VP(2), DET, PL, #L, UL, Mod k L, SAC 1 (log), RUL, CNL, … – Some of these are not so important…but some assuredly are!

20. Eric Allender: 27 and Still Counting The NC Hierarchy AC 0 TC 0 NC 1 AC 1 TC 1 NC 2 L NL

21. Eric Allender: 27 and Still Counting The NC Hierarchy AC 0 TC 0 NC 1 AC 1 TC 1 NC 2 L NL These have natural complete sets. These …not so much. But there are other important problems in the vicinity. Determinant CFLs

22. Eric Allender: 27 and Still Counting Linear Algebra and Logspace  The connection between linear algebra and logspace-bounded computation was discovered rather late, and via excessively difficult arguments – primarily because inductive counting was discovered so late.  The relevant logspace classes were initially studied without any motivation from natural problems.  What are these classes? – PL, #L, GapL, C = L

23. Eric Allender: 27 and Still Counting Probabilistic Logspace  PL was introduced by [Gill, 1977], by analogy with PP (defined in the same paper).  The history of upper bounds on the complexity of PL: – PSPACE [Gill, 1977] – SPACE(log 6 n) [Simon, 1981] – NC 2 [Borodin Cook Pippenger, 1982] – L #L [Jung, 1985]  What was the problem??

24. Eric Allender: 27 and Still Counting The problem with PL  In a nutshell, the problem is that PL machines can continue to do useful work after exponential time.  For example: NL = RL!

25. Eric Allender: 27 and Still Counting The problem with PL  In a nutshell, the problem is that PL machines can continue to do useful work after exponential time.  For example: NL = RL! (If “RL” is defined without a polynomial time bound.)  But with Inductive Counting as a tool, it’s easy to see that PL is the same class, with or without a polynomial-time restriction. (Jung did this the hard way, in 1985.)  Thus PL is characterized by NL machines with more accepting than rejecting paths.

26. Eric Allender: 27 and Still Counting Linear Algebra and #L  The connection between #P and the Permanent was made in 1979.  #L was explicitly defined and studied in 1990.  The fact that Determinant is complete for GapL (= #L - #L) was not discovered until 1991-1992.  An immediate consequence was: {M : Det(M) > 0} is complete for PL.  …but we much more often ask: Is Det(M)=0?

27. Eric Allender: 27 and Still Counting Singular matrices  The set of singular matrices is complete for C = L (characterized by NL machines where the number of accepting and rejecting computations are equal.)  Hierarchies:  AC 0 (C = L) = C = L U C = L C = L U …  AC 0 (PL)  AC 0 (#L)

28. Eric Allender: 27 and Still Counting Singular matrices  The set of singular matrices is complete for C = L (characterized by NL machines where the number of accepting and rejecting computations are equal.)  Hierarchies:  AC 0 (C = L) = C = L U C = L C = L U …  AC 0 (PL) = PL Collapse! [Beigel, Fu, 1997]  AC 0 (#L)

29. Eric Allender: 27 and Still Counting Singular matrices  The set of singular matrices is complete for C = L (characterized by NL machines where the number of accepting and rejecting computations are equal.)  Hierarchies:  AC 0 (C = L) = L C = L Collapse! [A. Beals, Ogihara]  AC 0 (PL) = PL Collapse! [Beigel, Fu, 1997]  AC 0 (#L)

30. Eric Allender: 27 and Still Counting Singular matrices  The set of singular matrices is complete for C = L (characterized by NL machines where the number of accepting and rejecting computations are equal.)  Hierarchies:  AC 0 (C = L) = L C = L Collapse! [A. Beals, Ogihara]  AC 0 (PL) = PL Collapse! [Beigel, Fu, 1997]  AC 0 (#L) = ???? (No collapse known.)

31. Eric Allender: 27 and Still Counting The C = L Hierarchy C=LC=L coC = L LC=LLC=L

32. Eric Allender: 27 and Still Counting The C = L Hierarchy Singular Matrices Nonsingular Matrices Rank So this is strong evidence that these three classes are distinct.

33. Eric Allender: 27 and Still Counting Another view of #L  A complete problem for #L is: Counting the number of paths from s to t in a directed graph.  Equivalently: it’s the problem of computing the (1,1) entry of a product of several nxn matrices with entries in the Natural Numbers.  Similarly, iterated product of integer matrices is complete for GapL (i.e., the determinant class).  Immerman and Landau highlighted the connection between complexity classes and iterated product over several algebras.

34. Eric Allender: 27 and Still Counting Immerman & Landau Curiously, missing from this list is the most “algebraic” class: VP Complexity ClassAlgebra for which Iterated Product is complete TC 0 Integers NC 1 5x5 Boolean matrices NC 1 Permutations on 5 elements (S 5 ) #NC 1 6x6 Integer matrices LPermutations on n elements (S n ) NLnxn Boolean matrices GapLnxn integer matrices

35. Eric Allender: 27 and Still Counting VP and Polynomial Degree  Valiant introduced the study of the arithmetic complexity of n-variate polynomials with algebraic degree bounded by poly(n). Later, this came to be called VP (or VP R for algebra R).

36. Eric Allender: 27 and Still Counting VP and Polynomial Degree  VP has a nice circuit characterization: log depth poly-size circuits with unbounded fan-in + gates, fan-in 2 * gates. (Semiunbounded fan-in circuits). [VSBR ‘83, Vinay ‘91]  Over the Boolean ring, this was discovered earlier [Ruzzo ‘79, Venkateswaran ‘87].  VP ({0,1},V,Λ) is also known as LogCFL and SAC 1. – LogCFL = problems logspace-reducible to CFLs. – SAC 1 = Semi-unbounded fan-in circuits of depth log 1 n.

37. Eric Allender: 27 and Still Counting SemiUnbounded Fan-In  Immediately after NL=coNL was established, SAC 1 was shown to be closed under complement, too (using inductive counting). [Borodin, Cook, Dymond, Ruzzo, Tompa]  Thus the following two models are equivalent: – Unbounded fan-in V, bounded fan-in Λ – Bounded fan-in V, unbounded fan-in Λ  How about other algebras?  A similar result over, say GF2 would be remarkable: VP(2) would equal AC 1 !

38. Eric Allender: 27 and Still Counting VP and Iterated “Product”  There aren’t that many “natural” complete problems for VP. (Using the connection to LogCFL, one class of complete problems is counting # of parse trees showing that x is in L, for certain CFLs L.)  Recently, a complete problem was added to this list, that looks a lot like an “iterated product”: – Tensor Contraction [Capelli, Durand, Mengel]

39. Eric Allender: 27 and Still Counting VP and Iterated “Product”  Matrices with b rows and c columns are 2- dimensional tensors of order [b,c].  Given 2 tensors A and B (of orders, say [a,b,c] and [c,d,e]), their contraction has order [a,b,d,e]  Given a list of 1-, 2-, and 3-dimensional tensors (or even poly-dimensional tensors), computing an entry of their iterated contraction is complete for VP.  But this is non-associative. (The nesting must be given, say, as a tree.)

40. Eric Allender: 27 and Still Counting Undirected Reachability  The same paper that showed LogCFL = coLogCFL also applied inductive counting to SL (the problems reducible to reachability in undirected graphs).  SL had its own hierarchy, inside the NL hierarchy.  SL was known to be in RL (with a poly run- time), and the new insight was a coRL algorithm. Many developments followed.  Reingold ended this saga, showing SL=L.

41. Eric Allender: 27 and Still Counting Where do these classes fit? NL Mod 2 L SAC 1 Mod 3 LMod 5 L VP(2) VP(3)VP(5) AC 1 VP #L

42. Eric Allender: 27 and Still Counting Gal and Wigderson NL Mod 2 L SAC 1 Mod 3 LMod 5 L VP(2) VP(3)VP(5) AC 1 VP #L Lots of nonuniform inclusions:

43. Eric Allender: 27 and Still Counting NL in #L is Open NL Mod 2 L SAC 1 Mod 3 LMod 5 L VP(2) VP(3)VP(5) AC 1 VP #L But UL is in #L (by definition). UL

44. Eric Allender: 27 and Still Counting [GW] + Inductive Counting UL/poly = NL/poly Mod 2 L SAC 1 Mod 3 LMod 5 L VP(2) VP(3)VP(5) AC 1 VP #L More nonuniform inculsions [Reihnardt, A]

45. Eric Allender: 27 and Still Counting [GW] + Inductive Counting UL/poly = NL/poly Mod 2 L SAC 1 Mod 3 LMod 5 L VP(2) VP(3)VP(5) AC 1 VP #L These hold uniformly if SAT needs size 2 n/100.

46. Eric Allender: 27 and Still Counting A sampling of open questions  Inductive counting (and related techniques) have thus far failed to show: – UL = coUL – C = L = coC = L – NL = UL – A collapse of the #L hierarchy – Any relationship between AC 1 and #L. (Immerman and Landau conjecture that TC 1 reduces to #L.)

47. Eric Allender: 27 and Still Counting AC 1 and VP  TC 1 = #AC 1 (mod p n ) [Reif, Tate] – Arithmetic degree n log n.  AC 1 is contained in #AC 1 (mod p n ) where all multiplication gates have fan-in log n. [A, Jiao, Mahajan, Vinay] – Arithmetic degree n loglog n.  Thus if the degree could be lowered to poly(n), working over Q instead of [Z mod p n ], one would have AC 1 contained in VP.  If also GapL = VP, we’d have AC 1 in GapL.

48. Eric Allender: 27 and Still Counting A Continuing Legacy  Inductive Counting continues to shape the research agenda.  Case in point: Catalytic Computing (STOC ‘14 [Buhrman, Cleve, Koucky, Loff, Speelman])  CL = problems solvable using logarithmic space augmented with a “full memory” that must be restored to its original state.  TC 1 is contained in CL, which is contained in ZPP.  What about CNL?

49. Eric Allender: 27 and Still Counting CNL  A new “nondeterministic” class. An unfamiliar model. How can one program in this model?  There is currently exactly one nondeterministic algorithm known in this model. It is used in order to show… – CNL = coCNL.

50. Eric Allender: 27 and Still Counting Legacy  A lessening of confidence in the framework of complexity classes, and an increase in humility regarding popular conjectures.  An invaluable insight into the nature of nondeterminism, and space-bounded computation in general.  A fundamental shift in the way that we approach questions in complexity theory.

51. Eric Allender: 27 and Still Counting Thank you!  …and Thank You, Neil!  Congratulations on your 60 th !