1. Boolean Circuits of Depth-Three and Arithmetic Circuits with General Gates Oded Goldreich Weizmann Institute of Science Based on Joint work with Avi Wigderson Original title: “On the Size of Depth-Three Boolean Circuits for Computing Multilinear Functions”, ECCC TR13-043.

2. Constant Depth Boolean Circuits Parity n requires depth d circuits of size exp(  (n 1/(d-1) )). Famous frontier: Stronger circuit models. Another frontier: Stronger lower bounds (i.e., exp(  (n))). Multi-linear functions : x=(x (1),…,x (t) ), x (i)  0,1  n F(x (1),…,x (t) ) =  (i_1,…,i_t)  T x i_1 (1)  x i_t (t) associated with tensor T  [n] t Conj (sanity check): For every t>1, there exists a t-linear function that requires depth-three circuits of size exp(  (tn t/(t+1) )). [holds for t=1…] Think of t=2,… log n

3. The Program * t-linear functions x=(x (1),…,x (t) ), |x (i) |=n F(x (1),…,x (t) ) =  (i_1,…,i_t)  T x i_1 (1)  x i_t (t) Conj (1 st sanity check): For every t>1, there exists a t-linear function that requires depth-three circuits of size exp(  (tn t/(t+1) )). [holds for t=1] Goal: For every t>1, present an explicit t-linear function that requires depth-three circuits of size exp(  (tn t/(t+1) )). [holds for t=1] A 2 nd sanity check: Consider a restricted model of (depth-three) circuits, and prove the L.B. in it. *) Taking advantage of Avi’s absence.

4. Arithmetic Circuits with General Gates Motivation: Depth-three Boolean Circuits for Parity n are obtained by implementing a sqrt(n)-way sum of sqrt(n)- way sums. In general, depth-three BC are obtained via depth-two AC with general ML-gates. Model: Depth-two (set-)multi-linear circuits with arbitrary (set-)multi-linear gates. Complexity measure (C 2 ) = the (max.) arity of a gate. Recall: We use a fix partition of the variables, and multi-linear means being linear in each variable-block. We get depth-three BC for F of size exponential in C 2 (F) Depth-three BC obtained this way are restricted in (1) their structure arising from direct composition, and (2) ML gates.

5. Arithmetic Circuits with General Gates (cont.) Model: Unbounded-depth (set-)multi-linear circuits with arbitrary (set-)multi-linear gates. Complexity measure (C) = max(arity, #gates). PROP: Every ML function F has a depth-three BC of size exp(O(C(F)). PF: guess & verify. THM: There exist bilinear functions F such that C(F)=sqrt(n) but C 2 (F)=  (n 2/3 ). OBS: For every t-linear F, C t+1 (F) ≤ 2C(F).

6. Arith. Circuits with General Gates: Results Model: Unbounded-depth (set-)multi-linear circuits with arbitrary (set-)multi-linear gates. Complexity measure (C) = max(arity, #gates); C 2 for depth-two. THM 1: There exist bilinear functions F such that C(F)=sqrt(n) but C 2 (F)=  (n 2/3 ). THM 2: For every t-linear function F it holds that C(F) ≤ C 2 (F) = O(tn t/(t+1) ). THM 3: Almost all t-linear functions F satisfy C 2 (F) ≥ C(F) =  (tn t/(t+1) ). Open: An explicit function as in Thm 3; for starters  (tn 0.51 ).

7. Arith. Circuits with General Gates: Results (cont.) Model: Unbounded-depth (set-)multi-linear circuits with arbitrary (set-)multi-linear gates. Complexity measure (C) = max(arity, #gates); C 2 for depth-two. An approach (a candidate): The 3-linear function assoc. with tensor T=  (i,j,k): |i-(n/2)|+|j-(n/2)|+|k-(n/2)|≤n/2 . Open: An explicit function as in Thm 3; for starters  (tn 0.51 ). Note: A restricted notion of (“structured”) rigidity suffices. Open: Show that  Toeplitz matrix w. rigidity n 1.51 for rank n 0.51. PROP: The complexity of the above 3-linear function is lower bounded by the maximum complexity of all bilinear functions associated w. Toeplitz matrices. THM: If matrix M has rigidity m 3 for rank m, then the corresponding bilinear function has complexity  (m).

8. Comments on the proofs Model: Multi-linear circuits with arbitrary multi-linear gates. Complexity measure (C) = max(arity, #gates); C 2 for depth-two. THM 1: There exist bilinear functions F such that C(F)=sqrt(n) but C 2 (F)=  (n 2/3 ). THM 2: For every t-linear function F it holds that C(F) ≤ C 2 (F) = O(tn t/(t+1) ). THM 3: Almost all t-linear functions F satisfy C 2 (F) ≥ C(F) =  (tn t/(t+1) ). THM 4: If matrix M has rigidity m 3 for rank m, then the corresponding bilinear function has complexity  (m). PF: Covering by m cubes of side m. PF: A counting argument. PF idea: s=sqrt(n), f(x,y)=g(x,L 1 (y),…,L s (y )). PF idea: The m linear function yield a rank m matrix, whereas the m quadratic forms (in variables) cover m 3 entries.

9. Add’l comments on the proof of THM 1 Model: Multi-linear circuits with arbitrary multi-linear gates. Complexity measure (C) = max(arity, #gates); C 2 for depth-two. THM 1: There exist bilinear functions F such that C(F)=sqrt(n) but C 2 (F)=  (n 2/3 ). PF: For s=sqrt(n), let f(x,y)=g(x,L 1 (y),…,L s (y)), where g is generic (over n+s bits), each L i computes the sum of s variables in y. A generic depth-two ML circuit of complexity m computes f as B(F 1 (x),…,F m (x),G 1 (y),…,G m (y)) +  i  [m] B i (x,y) where the B i ’s are quadratic and each function has arity m. Hitting y with a random restriction that leaves one variable alive in each block, we get B(F 1 (x),…,F m (x),G’ 1 (y’),…,G’ m (y’)) +  i  [m] B’ i (x,y’) where each B’ I (and G’ I ) depends on O(m/s) variables. Hence, the description length is O(m 3 /s) ; cf. to ns=n 2 /s.

10. Structured Rigidity THM 4’: If matrix M has (m,m,m)-structured rigidity for rank m, then the corresponding bilinear function has complexity  (m). PF idea: The proof of Thm 4 goes through w.o. any change. DEF: Matrix M has (m 1,m 2,m 3 )-structured rigidity for rank r if  matrix R of rank  r the non-zeros of M-R cannot be covered by m 1 (gen.) m 2 -by-m 3 rectangles. Rigidity m 1 m 2 m 3 implies (m 1,m 2,m 3 ) structured rigidity for the same rank, but not vice versa. THM 5: There exist matrices of (m,m,m)-structured rigidity for rank m that do not have rigidity 3mn for rank 0 (let alone for rank m). For every m  [n 0.51,n 0.66 ]. PF: Consider a random matrix with 3mn one-entries.

11. END Slides available at http://www.wisdom.weizmann.ac.il/~oded/T/kk.pptx Paper available at http://www.wisdom.weizmann.ac.il/~oded/p_kk.html