Download presentation
Presentation is loading. Please wait.
1
Eric Allender Rutgers University Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds DCFS 2008
2
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 2 >< 2 > Today’s Goal: To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity. And to show that this is useful. To plant seeds of optimism, regarding the prospects of proving lower bounds in circuit complexity.
3
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 3 >< 3 > Kolmogorov Complexity C(x) = min{|d| : U(d) = x} Important property – Invariance: The choice of the universal Turing machine U is unimportant. x is random if C(x) ≥ |x|. C A (x) = min{|d| : U A (d) = x}
4
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 4 >< 4 > Circuit Complexity Let D be a circuit of AND and OR gates (with negations at the inputs). Size(D) = # of wires in D. Size(f) = min{Size(D) : D computes f} We may allow oracle gates for a set A, along with AND and OR gates. Size A (f) = min{Size(D) : D A computes f}
5
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 5 >< 5 > K-complexity ≈ Circuit Complexity There are some obvious similarities in the definitions. What are some differences? A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings. – Given any string x, let f x be the function whose truth table is the string of length 2 log|x|, padded out with 0’s, and define Size(x) to be Size(f x ).
6
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 6 >< 6 > K-complexity ≈ Circuit Complexity There are some obvious similarities in the definitions. What are some differences? A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings. A more fundamental difference: – C(x) is not computable; Size(x) is. The Minimum Circuit Size Problem (MCSP): {(x,i) : Size(x) ≤ i}.
7
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 7 >< 7 > MCSP MCSP is in NP, but is not known to be NP- complete. MCSP is not believed to be in P. – Factoring is in BPP MCSP. – Every cryptographically-secure one-way function can be inverted in P MCSP /poly.
8
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 8 >< 8 > So how can K-complexity and Circuit complexity be the same? C(x) ≈ Size H (x), where H is the halting problem. For one direction, let U(d) = x. We need a small circuit (with oracle gates for H) for f x, where f x (i) is the i-th bit of x. This is easy, since {(d,i,b) : U(d) outputs a string whose i-th bit is b} is computably-enumerable. For the other direction, let Size H (f x ) = m. No oracle gate has more than m wires coming into it. Given a description of D (size not much bigger than m) and the m-bit number giving the size of {y in H : |y| ≤ m}, U can simulate D H and produce f x
9
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 9 >< 9 > So how can K-complexity and Circuit complexity be the same? C(x) ≈ Size H (x), where H is the halting problem. So there is a connection between C(x) and Size(x) … …but is it useful? First, let’s look at decidable versions of Kolmogorov complexity.
10
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Time-Bounded Kolmogorov Complexity The usual definition: C t (x) = min{|d| : U(d) = x in time t(|d|)}. Problems with this definition – No invariance! If U and U’ are different universal Turing machines, C t U and C t U’ have no clear relationship. – (One can bound C t U by C t’ U’ for t’ slightly larger than t – but nothing can be done for t’=t.) No nice connection to circuit complexity!
11
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Time-Bounded Kolmogorov Complexity Levin’s definition: Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}. Invariance holds! If U and U’ are different universal Turing machines, Kt U (x) and Kt U’ (x) are within log |x| of each other. Let A be complete for E = Dtime(2 O(n) ). Then Kt(x) ≈ Size A (x).
12
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Time-Bounded Kolmogorov Complexity Levin’s definition: Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}. Why log t? – This gives an optimal search order for NP search problems. – Adding t instead of log t would give every string complexity ≥ |x|. …So let’s look at how to make the run-time be much smaller.
13
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Revised Kolmogorov Complexity C(x) = min{|d| : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x} (where bit # i+1 of x is *). – This is identical to the original definition. Kt(x) = min{|d|+log t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}. – The new and old definitions are within O(log |x|) of each other. Define KT(x) = min{|d|+t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.
14
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Kolmogorov Complexity is Circuit Complexity C(x) ≈ Size H (x). Kt(x) ≈ Size E (x). KT(x) ≈ Size(x). Other measures of complexity can be captured in this way, too: – Branching Program Size ≈ KB(x) = min{|d|+2 s : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in space s(|d|)}.
15
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Kolmogorov Complexity is Circuit Complexity C(x) ≈ Size H (x). Kt(x) ≈ Size E (x). KT(x) ≈ Size(x). Other measures of complexity can be captured in this way, too: – Formula Size ≈ KF(x) = min{|d|+2 t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.
16
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? …but is this interesting? The result that Factoring is in BPP MCSP was first proved by observing that, in P MCSP, one can accept a large set of strings having large KT complexity (and by making use of many important results in the theory of pseudorandom generators and derandomization). (Basic Idea): There is a pseudorandom generator based on factoring, such that factoring is in BPP T for any test T that distinguishes truly random strings from pseudorandom strings. MCSP is such a test.
17
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? This idea has many variants. Consider R KT, R Kt, and R C. R KT is in coNP, and not known to be coNP hard. R C is not hard for NP under poly-time many-one reductions, unless P=NP. – How about more powerful reductions? – Is there anything interesting that we could compute quickly if C were computable, that we can’t already compute quickly? – Proof uses PRGs, Interactive Proofs, and the fact that an element of R C of length n can be found in But R C is undecidable! Perhaps H is in P relative to R C ?
18
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? This idea has many variants. Consider R KT, R Kt, and R C. R KT is in coNP, and not known to be coNP hard. R C, is not hard for NP under poly-time many-one reductions, unless P=NP. – How about more powerful reductions? – PSPACE is in P relative to R C. – NEXP is in NP relative to R C. – Proof uses PRGs, Interactive Proofs, and the fact that an element of R C of length n can be found in poly time, relative to R C [BFNV]. But R C is undecidable! Perhaps H is in P relative to R C ?
19
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Relationship between H and R C Perhaps H is in P relative to R C ? This is still open. It is known that there is a computable time bound t such that H is in DTime(t) relative to R C [Kummer]. …but the bound t depends on the choice of U in the definition of C(x). We also know that H is in P/poly relative to R C.
20
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? This idea has many variants. Consider R KT, R Kt, and R C. What about R Kt ? R Kt, is not hard for NP under poly-time many- one reductions, unless E=NE. – How about more powerful reductions? – EXP = NP(R Kt ). – R Kt is complete for EXP under P/poly reductions. – Open if R Kt is in P!
21
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Kolmogorov Complexity is Circuit Complexity C(x) ≈ Size H (x). Kt(x) ≈ Size E (x). KT(x) ≈ Size(x). Other measures of complexity can be captured in this way, too: – Formula Size ≈ KF(x) = min{|d|+2 t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.
22
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Kolmogorov Complexity is Circuit Complexity C(x) ≈ Size H (x). Kt(x) ≈ Size E (x). KT(x) ≈ Size(x). Other measures of complexity can be captured in this way, too: – A similar definition captures depth k threshold circuit size. [This is the clever transition to start the discussion of circuit lower bounds…]
23
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Big Complexity Classes NP PP .. .. NC L (Deterministic Logspace)
24
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? TC 0 O(1)-Depth Circuits of MAJ gates AC 0 [6] NC 1 Log-Depth Circuits AC 0 can’t compute Mod 2 [FSS,A] AC 0 O(1)-Depth Circuits of AND/OR gates The Main Objects of Interest: Small Complexity Classes
25
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? TC 0 O(1)-Depth Circuits of MAJ gates AC 0 [6] NC 1 Log-Depth Circuits AC 0 can’t compute Mod 2 [FSS,A] AC 0 O(1)-Depth Circuits of AND/OR gates The Main Objects of Interest: Small Complexity Classes
26
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? TC 0 O(1)-Depth Circuits of MAJ gates NC 1 Log-Depth Circuits AC 0 [2] can’t compute Mod 3 [R,S] AC 0 [2] AC 0 O(1)-Depth Circuits of AND/OR gates The Main Objects of Interest: Small Complexity Classes
27
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? NC 1 Log-Depth Circuits TC 0 O(1)-Depth Circuits of MAJ gates AC 0 [6] AC 0 [2] AC 0 O(1)-Depth Circuits of AND/OR gates The Main Objects of Interest: Small Complexity Classes
28
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? NC 1 poly-size formulae TC 0 O(1)-Depth Circuits of MAJ gates AC 0 [6] AC 0 [2] AC 0 O(1)-Depth Circuits of AND/OR gates The Main Objects of Interest: Small Complexity Classes
29
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? NP has complete sets (under polynomial time reducibility ≤ P ) These small classes have complete sets, too (under ≤ AC° ) Complete Problems
30
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Reductions A ≤ AC° B means that there is a constant-depth circuit computing A that has the usual AND and OR gates, and also has ‘oracle gates’ for B. B
31
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? NC 1 TC 0 AC 0 [6] AC 0 [2] AC 0 Complete Problems sorting, multiplication, division [Naor,Reingold] Pseudorandom Generator
32
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? NC 1 TC 0 AC 0 [6] AC 0 [2] AC 0 Complete Problems BFE: Balanced Boolean Formula Evaluation (AND,OR,XOR) Word problem over S 5
33
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? The Word Problem Over S 5 A regular set complete for NC 1 =
34
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Complexity Classes are not Invented – They’re Discovered NP (SAT, Clique, TSP,…) P (Linear Programming, CVP, …) NL (Connectivity, Shortest Paths, 2SAT, …) L (Undirected Connectivity, Acyclicity, …) NC 1 (BFE, Regular Sets) TC 0 (Sorting, Multiplication, Division) We’re interested in NC 1 (for instance) not because we want to build formulae for these functions…
35
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Complexity Classes are not Invented – They’re Discovered NP (SAT, Clique, TSP,…) P (Linear Programming, CVP, …) NL (Connectivity, Shortest Paths, 2SAT, …) L (Undirected Connectivity, Acyclicity, …) NC 1 (BFE, Regular Sets) TC 0 (Sorting, Multiplication, Division) … but because we want to know if the blocks of this partition are distinct.
36
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Complexity Classes are not Invented – They’re Discovered NP (SAT, Clique, TSP,…) P (Linear Programming, CVP, …) NL (Connectivity, Shortest Paths, 2SAT, …) L (Undirected Connectivity, Acyclicity, …) NC 1 (BFE, Regular Sets) TC 0 (Sorting, Multiplication, Division) These classes are real. They’re important.
37
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Longstanding Open Problems Is P = NP? Is AC 0 [6] = NP? Is depth 3 AC 0 [6] = NP? We’ll focus on questions such as : Is BFE in TC 0 ? Is BFE in AC 0 [6]?
38
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? How Close Are We to Proving Circuit Lower Bounds? Conventional Wisdom: Not Close At All! No new superpolynomial size lower bounds in over two decades. Razborov and Rudich: Any “natural” argument proving a lower bound against a circuit class C yields a proof that C can’t compute a pseudorandom function generator. Since the [Naor, Reingold] generator is computable in TC 0, this is bad news.
39
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? More Modest Goals Problems requiring formulae of size n 3 [Håstad] Problems requiring branching programs of size nearly n loglog n [Beame, Saks, Sun, Vee] Problems requiring depth d TC 0 circuits of size n 1+ c [Impagliazzo, Paturi, Saks] Time-Space Tradeoffs [Fortnow, Lipton, Van Melkebeek, Viglas] There is little feeling that these results bring us any closer to separating complexity classes.
40
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? How close are the following two statements? TC 0 Circuits for BFE must be of size n 1+Ω(1) For some c >0, TC 0 Circuits for BFE must be of size n 1+ c. How Close Are We to Proving Circuit Lower Bounds?
41
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? How close are the following two statements? TC 0 Circuits for BFE must be of size n 1+Ω(1) For some c >0, FTC 0 Circuits for BFE must be of size n 1+ c How Close Are We to Proving Circuit Lower Bounds? This is known [IPS’97] This implies TC 0 ≠ NC 1 [A, Koucky]
42
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self-Reducibility A set B is said to be “self-reducible” if B ≤ P B
43
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self-Reducibility A set B is said to be “self-reducible” if B ≤ P B via a reduction that, on input x, does not ask about whether x is in B. Very well-studied notion. For example, φ is in SAT if and only if (φ 0 is in SAT) or (φ 1 is in SAT)
44
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self-Reducibility Many of the important problems in (or near) NC 1 have a special self-reducibility property:
45
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self-Reducibility Many of the important problems in (or near) NC 1 have a special self-reducibility property: Instances of length n are AC 0 -Turing (or TC 0 - Turing) reducible to instances of length n ½ via reductions of linear size. Examples: – BFE – the word problem over S 5 – MAJORITY – Iterated Product of 3-by-3 Integer Matrices
46
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self Reducibility BFE A subformula near the root Subformulae near inputs
47
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self Reducibility S5S5
48
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self Reducibility The self-reduction of S 5, on inputs of size n, uses ( n ½ + 1) oracle gates of size n ½. Thus if S 5 has TC 0 circuits of size n k, it also has circuits of size ( n ½ + 1) n k/ 2 = O(n (k+ 1)/2 ). Similar arguments hold for other classes (such as AC 0 [6] and NC 1 ). More complicated self-reductions can be presented for MAJORITY and Iterated Product of 3-by-3 matrices.
49
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? A Corollary If BFE has TC 0 or AC 0 [6] circuits, then it has such circuits of nearly linear size. If S 5 has TC 0 or AC 0 [6] circuits, then it has such circuits of nearly linear size. If MAJ has AC 0 [6] circuits, then it has such circuits of nearly linear size. (Etc.) Thus, e.g., to separate NC 1 from TC 0, it suffices to show that BFE requires TC 0 circuits of size n 1.0000001.
50
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? A Corollary If BFE has TC 0 or AC 0 [6] circuits, then it has such circuits of nearly linear size. If S 5 has TC 0 or AC 0 [6] circuits, then it has such circuits of nearly linear size. If MAJ has AC 0 [6] circuits, then it has such circuits of nearly linear size. (Etc.) How widespread is this phenomenon? Is it true for SAT? (I.e., can we show NP ≠ TC 0 by proving that SAT requires TC 0 circuits of size n 1.0000001 ?)
51
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Limitations of Self-Reducibility Any problem for which instances of length n are TC 0 -Turing reducible to instances of length n ½ via poly-size reductions lies in NC. Thus there is no obvious way to apply these techniques to SAT or to problems complete for P. …but perhaps, rather than showing directly that SAT has this strong form of self- reducibility, one can argue that if SAT is in TC 0 then it has TC 0 circuits of nearly-linear size.
52
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Limitations of Self-Reducibility Any problem for which instances of length n are TC 0 -Turing reducible to instances of length n ½ via poly-size reductions lies in NC.
53
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Limitations of Self-Reducibility Any problem for which instances of length n are TC 0 -Turing reducible to instances of length n ½ via poly-size reductions lies in NC. d levels of oracle gates
54
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Limitations of Self-Reducibility Any problem for which instances of length n are TC 0 -Turing reducible to instances of length n ½ via poly-size reductions lies in NC. d 2 levels of oracle gates
55
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Limitations of Self-Reducibility Any problem for which instances of length n are TC 0 -Turing reducible to instances of length n ½ via poly-size reductions lies in NC. d 3 levels of oracle gates After log log rounds, the depth is log O(1) n
56
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Prospects for Progress We have seen that existing techniques prove bounds that are “nearly” good enough to separate NC 1 and TC 0. Some of these proofs are “natural”. Don’t the results of [Razborov & Rudich] indicate that further progress will require very different approaches? Not necessarily!
57
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Prospects for Progress The [Razborov & Rudich] framework of natural proofs assumes that a “natural” proof of a lower bound will make use of a combinatorial property that (among other things) is shared by a large fraction of the functions on n bits. In contrast, we are making use of a self- reducibility property that allows us to boost a n 1+ ε lower bound to a superpolynomial lower bound. This self-reducibility property holds for only a vanishingly small fraction of all functions.
58
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Prospects for Progress These observations are simple, but … they have forever changed the way that we look at quadratic (and smaller) lower bounds. We are not claiming to have found a way around the obstacles identified by [Razborov & Rudich]. (Such a claim will have to wait until someone proves that NC 1 ≠ TC 0.) But we do believe that this avenue deserves further exploration.
59
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? There are good reasons to develop and explore the connections between Kolmogorov complexity and circuit complexity. There are many open problems in this area that I will be delighted to discuss with you in more detail. There are two bad typos in the proceedings version of the paper. (“P” should be “NP”.) A corrected version is available at my home page. Conclusion
60
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Connections between Kolmogorov Complexity and Circuit Complexity might be relevant to the question of whether NEXP is contained in (non-uniform) TC 0 (depth 3). Speculation
61
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? [IKW] showed that NEXP is in P/poly iff NEXP = MA iff MA cannot be derandomized The proof shows that NEXP is in P/poly iff every set in P contains strings of KT- complexity O(log n) iff NEXP = IP[P/poly]. Speculation
62
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Similar techniques show: NEXP is in nonuniform NC 1 iff every set in P contains strings of KF- complexity O(log n) iff NEXP = MIPNC 1 iff MIPNC 1 cannot be derandomized. NEXP is in nonuniform TC 0 iff every set in P contains strings of small complexity iff NEXP = MIPTC 0 iff MIPTC 0 cannot be derandomized. Speculation
63
Eric Allender: How Close Are We to Proving Circuit Lower Bounds? What else happens in such a collapse? If NP = uniform TC 0, then #P is not contained in non-uniform TC 0 (so NEXP is not in non-uniform TC 0 ). So let’s consider NEXP = MIPTC 0 and NP ≠ uniform TC 0. If this “hardness assumption” were sufficient to “derandomize” MIPTC 0 then this would give the desired lower bound on NEXP… [Fortnow, Klivans], [van Melkebeek, Santhanam] Speculation
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.