Tensor-Rank and Lower Bounds for Arithmetic Formulas Ran Raz Weizmann Institute

Arithmetic Formulas: Field: F Variables: X 1,...,X n Gates: Every gate in the formula computes a polynomial in F[X 1,...,X n ] Example: (X 1 ¢ X 1 ) ¢ (X 2 + 1)

The Holy Grail: Super-polynomial lower bounds for the size of arithmetic circuits and formulas (for explicit polynomials) Our Result: Connections between Tensor-Rank and super-polynomial lower bounds for arithmetic formulas

Tensor-Rank: A: [n] r ! F is of rank 1 if 9 a 1,…,a r : [n] ! F s.t. A = a 1 ­ a 2 ­ … ­ a r, that is A(i 1,…,i r ) = a 1 (i 1 ) ¢¢¢ a r (i r ) Rank(A) = Min k s.t. A=A 1 +…+A k where A 1,…,A k are of rank 1 8 A: [n] r ! F Rank(A) · n r-1 (generalization of matrix rank)

Tensors and Polynomials: Given A: [n] r ! F and n ¢ r variables x 1,1,…,x r,n define

Tensor-Rank and Arithmetic Circuits: [Str73]: explicit A:[n] 3 ! F of rank m ) explicit lower bound of  (m) for arithmetic circuits (for f A ) (may give lower bounds of up to  (n 2 )) (best known bound:  (n)) Our Result: 8 r · logn/loglogn explicit A:[n] r ! F of rank n r(1-o(1)) ) explicit super-poly lower bound for arithmetic formulas (for f A )

Depth-3 vs. General Formulas: Tensor-rank corresponds to depth-3 set-multilinear formulas (for f A ) Corollary : strong enough lower bounds for depth-3 formulas ) super-poly lower bounds for general formulas Folklore: strong enough bounds for depth-4 circuits ) exp bounds for general circuits [AV08]: any exp bound for depth-4 circuits ) exp bound for general circuits

The Tensor-Product Approach [Str]: Given A 1 :[n 1 ] r ! F, A 2 :[n 2 ] r ! F Define A = A 1 ­ A 2 : [n 1 ¢ n 2 ] r ! F by A((i 1,j 1 ),…,(i r,j r )) = A 1 (i 1,…,i r ) ¢ A 2 (j 1,…j r ) For r=2, Rank(A) = Rank(A 1 ) ¢ Rank(A 2 ) Is Rank(A) > Rank(A 1 ) ¢ Rank(A 2 )/n o(1) ( 8 r) ? YES ) super-poly lower bounds for arithmetic formulas

The Tensor-Product Approach [Str]: Given A 1 :[n 1 ] r ! F, A 2 :[n 2 ] r ! F Define A = A 1 ­ A 2 : [n 1 ¢ n 2 ] r ! F by A((i 1,j 1 ),…,(i r,j r )) = A 1 (i 1,…,i r ) ¢ A 2 (j 1,…j r ) For r=2, Rank(A) = Rank(A 1 ) ¢ Rank(A 2 ) Is Rank(A) > Rank(A 1 ) ¢ Rank(A 2 )/n o(1) ( 8 r) ? YES ) super-poly lower bounds for arithmetic formulas Proof: Let m=n 1/r Take A 1,…,A r :[m] r ! F of high rank Let A = A 1 ­ A 2 ­ … ­ A r : [n] r ! F How do we find A 1,…,A r of high rank ? We fix their r ¢ n entries as inputs !

Main Steps of the Proof: 1) New homogenization and multilinearization techniques 2) Defining syntactic-rank of a formula (bounds the tensor-rank) 3) 8 s we find the formula of size s with the largest syntactic-rank 4) Compute the largest syntactic- rank of a poly-size formula

Homogenous Polynomials: P 2 F[X 1,...,X n ] is homogenous if all its monomials are of the same degree Homogenous Formulas: A formula is homogenous if each of its nodes computes a homogenous polynomial

Homogenization: Given a formula C of size s for a homogenous polynomial f of deg r give a homogenous formula D for f [Str73]: D of size s O(log r) (optimality conjectured in [NW95]) Our Result: D of size (where d = product depth · O(log s)) If s=poly(n), and r · logn Size(D)=poly(n)

Conclusions (of Step 1): For r · logn/loglogn 1) super-poly lower bounds for homogenous formulas ) super-poly lower bounds for general formulas 2) super-poly lower bounds for set-mult formulas ) super-poly lower bounds for general formulas

Thank You!