1. Eric Allender Rutgers University The Strange Link between Incompressibility and Complexity China Theory Week, Aarhus August 13, 2012

2. Eric Allender: The Strange Link between Incompressibility and Complexity < 2 >< 2 > Today’s Goal:  To present new developments in a line of research dating back to 2002, presenting some unexpected connections between – Kolmogorov Complexity (the theory of randomness), and – Computational Complexity Theory  Which ought to have nothing to do with each other!

3. Eric Allender: The Strange Link between Incompressibility and Complexity < 3 >< 3 > Complexity Classes P NP BPP PSPACE NEXP EXPSPACE P/poly

4. Eric Allender: The Strange Link between Incompressibility and Complexity < 4 >< 4 > A Jewel of Derandomization  [Impagliazzo, Wigderson, 1997]: If there is a problem computable in time 2 n that requires circuits of size 2 εn, then P = BPP.

5. Eric Allender: The Strange Link between Incompressibility and Complexity < 5 >< 5 > Randomness  Which of these sequences did I obtain by flipping a coin?  0101010101010101010101010101010  0110100111111100111000100101100  1101010001011100111111011001010  1/π =0.3183098861837906715377675267450  Each of these sequences is equally likely, in the sense of probability theory – which does not provide us with a way to talk about “randomness”.

6. Eric Allender: The Strange Link between Incompressibility and Complexity < 6 >< 6 > Randomness  Which of these sequences did I obtain by flipping a coin?  0101010101010101010101010101010  0110100111111100111000100101100  1101010001011100111111011001010  1/π =0.3183098861837906715377675267450  How many bits of information does each of these sequences contain?

7. Eric Allender: The Strange Link between Incompressibility and Complexity < 7 >< 7 > Kolmogorov Complexity  C(x) = min{|d| : U(d) = x} – U is a “universal” Turing machine  K(x) = min{|d| : U(d) = x} – U is a “universal” prefix-free Turing machine  Important property – Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant).  x is random if C(x) ≥ |x|.

8. Eric Allender: The Strange Link between Incompressibility and Complexity < 8 >< 8 > Kolmogorov Complexity  C(x) = min{|d| : U(d) = x} – U is a “universal” Turing machine  K(x) = min{|d| : U(d) = x} – U is a “universal” prefix-free Turing machine  Important property – Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant).  x is random if C(x) ≥ |x|, or K(x) ≥ |x|.

9. Eric Allender: The Strange Link between Incompressibility and Complexity < 9 >< 9 > K, C, and Randomness  K(x) and C(x) are “close”: – C(x) ≤ K(x) ≤ C(x) + 2 log |x|  Two notions of randomness: – R C = {x : C(x) ≥ |x|} – R K = {x : K(x) ≥ |x|}  …actually, infinitely many notions of randomness: – R C U = {x : C U (x) ≥ |x|}

10. Eric Allender: The Strange Link between Incompressibility and Complexity K, C, and Randomness  K(x) and C(x) are “close”: – C(x) ≤ K(x) ≤ C(x) + 2 log |x|  Two notions of randomness: – R C = {x : C(x) ≥ |x|} – R K = {x : K(x) ≥ |x|}  …actually, infinitely many notions of randomness: – R C U = {x : C U (x) ≥ |x|}, R K U = {x : K U (x) ≥ |x|}

11. Eric Allender: The Strange Link between Incompressibility and Complexity K, C, and Randomness  When it makes no difference, we’ll write “R” instead of R C or R K.  Basic facts: – R is undecidable – …but it is not “easy” to use it as an oracle. – R is not NP-hard under poly-time ≤ m reductions, unless P=NP. – Things get more interesting when we consider more powerful types of reducibility.

12. Eric Allender: The Strange Link between Incompressibility and Complexity Three Bizarre Inclusions  NEXP is contained in NP R. [ABK06]  PSPACE is contained in P R. [ABKMR06]  BPP is contained in {A : A is poly-time ≤ tt R}. [BFKL10] – A ≤ tt reduction is a “non-adaptive” reduction. – On input x, a list of queries is formulated before receiving any answer from the oracle.

13. Eric Allender: The Strange Link between Incompressibility and Complexity Three Bizarre Inclusions  NEXP is contained in NP R. [ABK06]  PSPACE is contained in P R. [ABKMR06]  BPP is contained in P tt R. [BFKL10] “Bizarre”, because a non-computable “upper bound” is presented on complexity classes! We have been unable to squeeze larger complexity classes inside. Are these containments optimal?

14. Eric Allender: The Strange Link between Incompressibility and Complexity Three Bizarre Inclusions  NEXP is contained in NP R. [ABK06]  PSPACE is contained in P R. [ABKMR06]  BPP is contained in P tt R. [BFKL10] “Bizarre”, because a non-computable “upper bound” is presented on complexity classes! If we restrict attention to R K, then we can do better…

15. Eric Allender: The Strange Link between Incompressibility and Complexity Three Bizarre Inclusions  NEXP is contained in NP R K. – The decidable sets that are in NP R K for every U are in EXPSPACE. [AFG11]  PSPACE is contained in P R K.  BPP is contained in P tt R K. – The decidable sets that are in P tt R K for every U are in PSPACE. [AFG11]

16. Eric Allender: The Strange Link between Incompressibility and Complexity Three Bizarre Inclusions  NEXP is contained in NP R K (for every U). – The decidable sets that are in NP R K for every U are in EXPSPACE. [AFG11]  PSPACE is contained in P R K (for every U).  BPP is contained in P tt R K (for every U). – The decidable sets that are in P tt R K for every U are in PSPACE. [AFG11] – [CELM] The sets that are in P tt R K for every U are decidable.

17. Eric Allender: The Strange Link between Incompressibility and Complexity Three Bizarre Inclusions  NEXP is contained in NP R K (for every U). – The decidable sets that are in NP R K for every U are in EXPSPACE. [AFG11]  PSPACE is contained in P R K (for every U).  BPP is contained in P tt R K (for every U). – The sets that are in P tt R K for every U are in PSPACE. [AFG11]

18. Eric Allender: The Strange Link between Incompressibility and Complexity Three Bizarre Inclusions  NEXP is contained in NP R K (for every U). – The sets that are in NP R K for every U are in EXPSPACE. [AFG11]  PSPACE is contained in P R K (for every U).  BPP is contained in P tt R K (for every U). – The sets that are in P tt R K for every U are in PSPACE. [AFG11]

19. Eric Allender: The Strange Link between Incompressibility and Complexity Three Bizarre Inclusions  NEXP is contained in NP R K (for every U). – The sets that are in NP R K for every U are in EXPSPACE. [AFG11]  Conjecture: This should hold for R C, too.  BPP is contained in P tt R K (for every U). – The sets that are in P tt R K for every U are in PSPACE. [AFG11]

20. Eric Allender: The Strange Link between Incompressibility and Complexity Three Bizarre Inclusions  NEXP is contained in NP R K (for every U). – The sets that are in NP R K for every U are in EXPSPACE. [AFG11]  This holds even for sets in EXP tt R K for all U!  BPP is contained in P tt R K (for every U). – The sets that are in P tt R K for every U are in PSPACE. [AFG11]

21. Eric Allender: The Strange Link between Incompressibility and Complexity Three Bizarre Inclusions  NEXP is contained in NP R K (for every U). – The sets that are in NP R K for every U are in EXPSPACE. [AFG11]  Conjecture: This class is exactly NEXP.  BPP is contained in P tt R K (for every U). – The sets that are in P tt R K for every U are in PSPACE. [AFG11] Conjecture: This class is exactly BPP.

22. Eric Allender: The Strange Link between Incompressibility and Complexity Three Bizarre Inclusions  NEXP is contained in NP R K (for every U). – The sets that are in NP R K for every U are in EXPSPACE. [AFG11]  Conjecture: This class is exactly NEXP.  BPP is contained in P tt R K (for every U). – The sets that are in P tt R K for every U are in PSPACE. [AFG11] Conjecture: This class is exactly BPP P.

23. Eric Allender: The Strange Link between Incompressibility and Complexity K-Complexity and BPP vs P  BPP is contained in P tt R K (for every U). – The sets that are in P tt R K for every U are in PSPACE. Conjecture: This class is exactly P.  Some support for this conjecture [ABK06]: – The decidable sets that are in P dtt R K for every U are in P. – The decidable sets that are in P parity-tt R K for every U are in P.

24. Eric Allender: The Strange Link between Incompressibility and Complexity K-Complexity and BPP vs P  BPP is contained in P tt R K (for every U). – The sets that are in P tt R K for every U are in PSPACE. Conjecture: This class is exactly P.  New results support a weaker conjecture:  Conjecture: This class is contained in PSPACE ∩ P/poly.  More strongly: Every decidable set in P tt R is in P/poly.

25. Eric Allender: The Strange Link between Incompressibility and Complexity K-Complexity and BPP vs P  BPP is contained in P tt R K (for every U). – The sets that are in P tt R K for every U are in PSPACE. Conjecture: This class is exactly P.  New results support a weaker conjecture :  Conjecture: This class is contained in PSPACE ∩ P/poly.  More strongly: Every decidable set in P tt R is in P/poly (i.e., for every U, and for both C and K).

26. Eric Allender: The Strange Link between Incompressibility and Complexity The Central Conjecture  Conjecture: Every decidable set in P tt R is in P/poly.  What can we show?

27. Eric Allender: The Strange Link between Incompressibility and Complexity The Central Conjecture  Conjecture: Every decidable set in P tt R is in P/poly.  What can we show?  We show that a similar statement holds in the context of time-bounded K-complexity.

28. Eric Allender: The Strange Link between Incompressibility and Complexity Time-Bounded K-complexity  Let t be a time bound. (Think of t as being large, such as Ackermann’s function.)  Define K t (x) to be min{|d| : U(d) = x in at most t(|x|) steps}.  Define R K t to be {x : K t (x) ≥ |x|}.  Define TTRT = {A : A is in P tt R K t for all large enough time bounds t}.  Vague intuition: Poly-time reductions should not be able to distinguish between R K t and R K, for large t.

29. Eric Allender: The Strange Link between Incompressibility and Complexity The Central Conjecture  Conjecture: Every decidable set in P tt R is in P/poly.  We show that a similar statement holds in the context of time-bounded K-complexity: – TTRT is contained in P/poly [ABFL12].  If t(n) = 2 2 n, then R K t is NOT in P/poly.  …which supports our “vague intuition”, because this set is not reducible to the time-t’- random strings for t’ >> t.

30. Eric Allender: The Strange Link between Incompressibility and Complexity The Central Conjecture  Conjecture: Every decidable set in P tt R is in P/poly.  We show that a similar statement holds in the context of time-bounded K-complexity: – TTRT is contained in P/poly [ABFL12].  BUT – The same P/poly bound holds, even if we consider P R K t instead of P tt R K t.  …and recall PSPACE is contained in P R.  So the “vague intuition” is wrong!

31. Eric Allender: The Strange Link between Incompressibility and Complexity The Central Conjecture: An Earlier Approach  Conjecture: Every decidable set in P tt R is in P/poly.  We give a proof of a statement of the form: A n A j Ψ( n,j ) such that: if for each n and j there is a proof in PA of Ψ(n,j) then the conjecture holds.

32. Eric Allender: The Strange Link between Incompressibility and Complexity Basic Proof Theory  Recall that Peano Arithmetic cannot prove the statement “PA is consistent”.  Let PA 1 be PA + “PA is consistent”.  Similarly, one can define PA 2, PA 3, …  “PA is consistent” can be formulated as “for all j, there is no length j proof of 0=1”.  For each j, PA can prove “there is no length j proof of 0=1”.

33. Eric Allender: The Strange Link between Incompressibility and Complexity The Central Conjecture: An Earlier Approach  Conjecture: Every decidable set in P tt R is in P/poly.  We give a proof (in PA 1 ) of a statement of the form: A n A j Ψ( n,j ) such that: if for each n and j there is a proof in PA of Ψ(n,j) then the conjecture holds.

34. Eric Allender: The Strange Link between Incompressibility and Complexity The Central Conjecture: The Earlier Approach Fails  The connections to proof theory were unexpected and intriguing, and seemed promising…  But unfortunately, it turns out that many of the statements Ψ(n,j) are independent of PA (and a related approach yields statements Ψ(n,j,k) that are independent of each system PA r ).

35. Eric Allender: The Strange Link between Incompressibility and Complexity A High-Level View of the “Earlier Approach”  Let A be decidable, and let M be a poly-time machine computing a ≤ tt -reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then there is a d such that for all x, there is a V containing only strings of length at most d+log f(|x|), such that M V (x) = A(x). Note: V says “long queries are non-random”.

36. Eric Allender: The Strange Link between Incompressibility and Complexity A Warm-Up  Let A be decidable, and let M be a poly-time machine computing a ≤ tt -reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then there is a d such that for all x, there is a V containing only strings of length at most d+log f(|x|), such that M V (x) = A(x). Note: If some V works for all x of length n, then A is in P/poly.

37. Eric Allender: The Strange Link between Incompressibility and Complexity Proof  Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), M V (x)≠A(x).  Consider the machine that takes input (d,r) and finds x (as above) and outputs the r th element of Q(x).  This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1)

38. Eric Allender: The Strange Link between Incompressibility and Complexity Proof  Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), M V (x)≠A(x).  Consider the machine that takes input (d,r) and finds x (as above) and outputs the r th element of Q(x).  This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).  Thus if we pick V* to be R∩{0,1} d+log f(|x|), we see that M V* (x) = M R (x)

39. Eric Allender: The Strange Link between Incompressibility and Complexity Proof  Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), M V (x)≠A(x).  Consider the machine that takes input (d,r) and finds x (as above) and outputs the r th element of Q(x).  This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).  Thus if we pick V* to be R∩{0,1} d+log f(|x|), we see that M V* (x) = M R (x) = A(x). Contradiction!

40. Eric Allender: The Strange Link between Incompressibility and Complexity Cleaning Things Up  Let A be decidable, and let M be a poly-time machine computing a ≤ tt -reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then there is a d such that for all x, there is a V containing only strings of length at most d+log f(|x|) g A (|x|), such that M V (x) = A(x).

41. Eric Allender: The Strange Link between Incompressibility and Complexity Cleaning Things Up  Let A be decidable, and let M be a poly-time machine computing a ≤ tt -reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then there is a d such that for all x, there is a V containing only strings of length at most d+log f(|x|) g A (|x|), such that M V (x) = A(x).

42. Eric Allender: The Strange Link between Incompressibility and Complexity Cleaning Things Up  Let A be decidable, and let M be a poly-time machine computing a ≤ tt -reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then for all x, there is a V containing only strings in R of length at most g A (|x|) such that M V (x) = A(x).

43. Eric Allender: The Strange Link between Incompressibility and Complexity A Refinement  Let A be decidable, and let M be a poly-time machine computing a ≤ tt -reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then for all x, there is a V containing only strings in R of length at most g A (|x|) such that M V (x) = A(x).

44. Eric Allender: The Strange Link between Incompressibility and Complexity Approximating R  We can obtain a series of approximations to R (up to length g A (n)) as follows:  R n,0 = all strings of length at most g A (n).  R n,i+1 = R n,i minus the i+1 st string of length at most g A (n) that is found, in an enumeration of non-random strings.  R n,0, R n,1, R n,2, … R n,i* = R∩{0,1} g A (n)

45. Eric Allender: The Strange Link between Incompressibility and Complexity A Refinement  Let A be decidable, and let M be a poly-time machine computing a ≤ tt -reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then for all xє{0,1} n, for all i, there is a V containing only strings in R n,i such that M V (x) = A(x).

46. Eric Allender: The Strange Link between Incompressibility and Complexity Proof  Assume that for each d there is some x,i such that, for all V containing strings in R n,i of length at most d+log f(|x|), M V (x)≠A(x).  Consider the machine that takes input (d,r) and finds x,i (as above) and outputs the r th element of Q(x).  This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).  Thus if we pick V* to be R∩{0,1} d+log f(|x|), we see that M V* (x) = M R (x) = A(x). Contradiction!

47. Eric Allender: The Strange Link between Incompressibility and Complexity Where does PA come in??  Let A be decidable, and let M be a poly-time machine computing a ≤ tt -reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then for all xє{0,1} n, for all i, there is a V containing only strings in R n,i such that M V (x) = A(x) And for all k, there is not a length-k proof that “for all i, V is not equal to R n,i ”.

48. Eric Allender: The Strange Link between Incompressibility and Complexity What went wrong with the earlier approach.  We have shown: For all xє{0,1} n, for all i, there is a V containing only short strings in R n,i such that M V (x) = A(x).  We were aiming at showing that one can swap the quantifiers, so that for all n, there is a V containing only short strings in R n,i such that, for all x of length n, M V (x) = A(x).  But there is a (useless) reduction M for which this is false. (M already knows the outcome of its queries, assuming that the oracle is R.)

49. Eric Allender: The Strange Link between Incompressibility and Complexity Open Questions:  Decrease the gap (NEXP vs EXPSPACE) between the lower and upper bounds on the complexity of the problems that are in NP R K for every U.  Some of our proofs rely on using R K. Do similar results hold also for R C ? – Disprove: The halting problem is in P tt R C.  Can the PSPACE ∩ P/poly bound (in the time- bounded setting) be improved to BPP?  Is this approach relevant at all to the P=BPP question?

50. Eric Allender: The Strange Link between Incompressibility and Complexity P vs BPP  Our main intuition for P=BPP comes from [Impagliazzo, Wigderson]. Circuit lower bounds imply derandomization.  Note that this provides much more than “merely” P=BPP; it gives a recipe for simulating any probabilistic algorithm.  Goldreich has argued that any proof of P=BPP actually yields pseudorandom generators (and hence a “recipe” as above)… – …but this has only been proved for the “promise problem” formulation of P=BPP.

51. Eric Allender: The Strange Link between Incompressibility and Complexity P vs BPP  Recall that TTRT sits between BPP and PSPACE ∩ P/poly.  A proof that TTRT = P would show that BPP = P – but it is not clear that this would yield any sort of recipe for constructing useful pseudorandom generators.  Although it would be a less “useful” approach, perhaps it might be an easier approach?

52. Eric Allender: The Strange Link between Incompressibility and Complexity How to use R as an Oracle

53. Eric Allender: The Strange Link between Incompressibility and Complexity Circuit Complexity  Let D be a circuit of AND and OR gates (with negations at the inputs). Size(D) = # of wires in D.  Size(f) = min{Size(D) : D computes f}  We may allow oracle gates for a set A, along with AND and OR gates.  Size A (f) = min{Size(D) : D A computes f}

54. Eric Allender: The Strange Link between Incompressibility and Complexity Oracle Gates  An “oracle gate” for B in a circuit: B

55. Eric Allender: The Strange Link between Incompressibility and Complexity K-complexity ≈ Circuit Complexity  There are some obvious similarities in the definitions.  C(x) = min{|d| : U(d) = x}  Size(f) = min{Size(D) : D computes f}

56. Eric Allender: The Strange Link between Incompressibility and Complexity K-complexity ≈ Circuit Complexity  There are some obvious similarities in the definitions. What are some differences?  A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings. – Given any string x, let f x be the function whose truth table is the string of length 2 log|x|, padded out with 0’s, and define Size(x) to be Size(f x ).

57. Eric Allender: The Strange Link between Incompressibility and Complexity K-complexity ≈ Circuit Complexity  There are some obvious similarities in the definitions. What are some differences?  A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.  A more fundamental difference: – C(x) is not computable; Size(x) is.

58. Eric Allender: The Strange Link between Incompressibility and Complexity Lightning Review: Computability  Here’s all we’ll need to know about computability: – The halting problem, everyone’s favorite uncomputable problem: – H = {(i,x) : M i halts on input x} – Every r.e. problem is poly-time many-one reducible to H. – One such r.e. problem: {x : C(x) < |x|}. – There is no time-bounded many-one reduction in other direction, but there is a (large) time t Turing reduction (and hence C is not computable).

59. Eric Allender: The Strange Link between Incompressibility and Complexity K-complexity ≈ Circuit Complexity  There are some obvious similarities in the definitions. What are some differences?  A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.  A more fundamental difference: – C(x) is not computable; Size(x) is.  In fact, there is a fascinating history, regarding the complexity of the Size function.

60. Eric Allender: The Strange Link between Incompressibility and Complexity MCSP  MCSP = {(x,i) : Size(x) < i}.  MCSP is in NP, but is not known to be NP- complete.  Some history: NP-completeness was discovered independently in the 70s by Cook (in North America) and Levin (in Russia).  Levin delayed publishing his results, because he wanted to show that MCSP was NP- complete, thinking that this was the more important problem.

61. Eric Allender: The Strange Link between Incompressibility and Complexity MCSP  MCSP = {(x,i) : Size(x) < i}.  Why was Levin so interested in MCSP?  In the USSR in the 70’s, there was great interest in problems requiring “perebor”, or “brute-force search”. For various reasons, MCSP was a focal point of this interest.  It was recognized at the time that this was “similar in flavor” to questions about resource- bounded Kolmogorov complexity, but there were no theorems along this line.

62. Eric Allender: The Strange Link between Incompressibility and Complexity MCSP  MCSP = {(x,i) : Size(x) < i}.  3 decades later, what do we know about MCSP?  If it’s complete under the “usual” poly-time many-one reductions, then BPP = P.  MCSP is not believed to be in P. We know: – Factoring is in BPP MCSP. – Every cryptographically-secure one-way function can be inverted in P MCSP /poly.

63. Eric Allender: The Strange Link between Incompressibility and Complexity So how can K-complexity and Circuit complexity be the same?  C(x) ≈ Size H (x), where H is the halting problem.  For one direction, let U(d) = x. We need a small circuit (with oracle gates for H) for f x, where f x (i) is the i-th bit of x. This is easy, since {(d,i,b) : U(d) outputs a string whose i-th bit is b} is computably-enumerable.  For the other direction, let Size H (f x ) = m. No oracle gate has more than m wires coming into it. Given a description of D (size not much bigger than m) and the m-bit number giving the size of {y in H : |y| ≤ m}, U can simulate D H and produce f x.

64. Eric Allender: The Strange Link between Incompressibility and Complexity So how can K-complexity and Circuit complexity be the same?  C(x) ≈ Size H (x), where H is the halting problem.  So there is a connection between C(x) and Size(x) …  …but is it useful?  First, let’s look at decidable versions of Kolmogorov complexity.

65. Eric Allender: The Strange Link between Incompressibility and Complexity Time-Bounded Kolmogorov Complexity  The usual definition:  C t (x) = min{|d| : U(d) = x in time t(|x|)}.  Problems with this definition – No invariance! If U and U’ are different universal Turing machines, C t U and C t U’ have no clear relationship. – (One can bound C t U by C t’ U’ for t’ slightly larger than t – but nothing can be done for t’=t.)  No nice connection to circuit complexity!

66. Eric Allender: The Strange Link between Incompressibility and Complexity Time-Bounded Kolmogorov Complexity  Levin’s definition:  Kt(x) = min{|d|+log t : U(d) = x in time t}.  Invariance holds! If U and U’ are different universal Turing machines, Kt U (x) and Kt U’ (x) are within log |x| of each other.  And, there’s a connection to Circuit Complexity: – Let A be complete for E = Dtime(2 O(n) ). Then Kt(x) ≈ Size A (x).

67. Eric Allender: The Strange Link between Incompressibility and Complexity Time-Bounded Kolmogorov Complexity  Levin’s definition:  Kt(x|φ) = min{|d|+log t : U(d,φ)=x in time t}.  Why log t? – This gives an optimal search order for NP search problems. – This algorithm finds satisfying assignments in poly time, if any algorithm does: – On input φ, search through assignments y in order of increasing Kt(y|φ).

68. Eric Allender: The Strange Link between Incompressibility and Complexity Time-Bounded Kolmogorov Complexity  Levin’s definition:  Kt(x) = min{|d|+log t : U(d) = x in time t}.  Why log t? – This gives an optimal search order for NP search problems. – Adding t instead of log t would give every string complexity ≥ |x|.  …So let’s look for a sensible way to allow the run-time to be much smaller.

69. Eric Allender: The Strange Link between Incompressibility and Complexity Revised Kolmogorov Complexity  C(x) = min{|d| : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x} (where bit # i+1 of x is *). – This is identical to the original definition.  Kt(x) = min{|d|+log t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t}. – The new and old definitions are within O(log |x|) of each other.  Define KT(x) = min{|d|+t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t}.

70. Eric Allender: The Strange Link between Incompressibility and Complexity Kolmogorov Complexity is Circuit Complexity  C(x) ≈ Size H (x).  Kt(x) ≈ Size E (x).  KT(x) ≈ Size(x).  Other measures of complexity can be captured in this way, too: – Branching Program Size ≈ KB(x) = min{|d|+2 s : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in space s}.

71. Eric Allender: The Strange Link between Incompressibility and Complexity Kolmogorov Complexity is Circuit Complexity  C(x) ≈ Size H (x).  Kt(x) ≈ Size E (x).  KT(x) ≈ Size(x).  Other measures of complexity can be captured in this way, too: – Formula Size ≈ KF(x) = min{|d|+2 t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t}, for an alternating Turing machine U.

72. Eric Allender: The Strange Link between Incompressibility and Complexity …but is this interesting?  The result that Factoring is in BPP MCSP was first proved by observing that, in P MCSP, one can accept a large set of strings having large KT complexity: R KT = {x : KT(x) ≥ |x|}.  (Basic Idea): There is a pseudorandom generator based on factoring, such that factoring is in BPP T for any test T that distinguishes truly random strings from pseudorandom strings. R KT is just such a test T.

73. Eric Allender: The Strange Link between Incompressibility and Complexity This idea has many variants.  Consider R KT, R Kt, and R C.  R KT is in coNP, and not known to be coNP hard.  R C is not hard for NP under poly-time many-one reductions, unless P=NP. – How about more powerful reductions? – Is there anything interesting that we could compute quickly if C were computable, that we can’t already compute quickly? – Proof uses PRGs, Interactive Proofs, and the fact that an element of R C of length n can be found in  But R C is undecidable! Perhaps H is in P relative to R C ?

74. Eric Allender: The Strange Link between Incompressibility and Complexity This idea has many variants.  Consider R KT, R Kt, and R C.  R KT is in coNP, and not known to be coNP hard.  R C is not hard for NP under poly-time many-one reductions, unless P=NP. – How about more powerful reductions? We show: – PSPACE is in P relative to R C. – NEXP is in NP relative to R C. – Proof uses PRGs, Interactive Proofs, and the fact that an element of R C of length n can be found in poly time, relative to R C [BFNV].  But R C is undecidable! Perhaps H is in P relative to R C ?

75. Eric Allender: The Strange Link between Incompressibility and Complexity Relationship between H and R C  Perhaps H is in P relative to R C ?  This is still open. It is known that there is a computable time bound t such that H is in DTime(t) relative to R C [Kummer].  …but the bound t depends on the choice of U in the definition of C(x).  We also showed: H is in P/poly relative to R C.  Thus, in some sense, there is an “efficient” reduction of H to R C.

76. Eric Allender: The Strange Link between Incompressibility and Complexity Playing the Game in PSPACE  We can also define a space-bounded measure KS, such that Kt(x) < KS(x) < KT(x).  RKS is complete for PSPACE under BPP reductions (and, in fact, even under “ZPP” reductions, which implies completeness under NP reductions and P/poly reductions).  [This makes use of an even stronger form of the Impagliazzo-Wigderson generator, which relies on the “hard” function being “downward self-reducible and random self-reducible”.]

77. Eric Allender: The Strange Link between Incompressibility and Complexity Thank you!