Linear Algebra in Weak Formal Theories of Arithmetic Iddo Tzameret Royal Holloway, University of London Joint work with Stephen A. Cook University of Toronto

Extreme Reverse Mathematics of Linear Algebra Iddo Tzameret Royal Holloway, University of London Joint work with Stephen A. Cook University of Toronto

In Short

Basic statements of linear algebra are provable in a weak formal theory of arithmetic. The weak theory corresponds to the computational complexity in which we can compute the determinant. This is almost the weakest theory believed to prove such statements. Was open (non-uniform was proved by Hrubes-T. 2015)

The Protagonists

The Determinant The Determinant Identities: (1) Det 𝐴𝐵 =Det 𝐴 Det(𝐵) Det 𝑋 ≔ 𝜎∈ 𝑆 𝑛 sgn(𝜎) 𝑖=1 𝑛 𝑥 𝑖𝜎(𝑗) The Determinant Identities: (1) Det 𝐴𝐵 =Det 𝐴 Det(𝐵) (2) Det 𝐶 = 𝑐 11 ⋯ 𝑐 𝑛𝑛 , for 𝐶 a triangular matrix

NC2 Complexity Class Thm (Berkowitz 1984): Det over ℤ is in NC2. Boolean circuits of poly(n) size and O(log2n) depth “Parallel Computation” “Balanced Circuits” Uniform (FO uniform) Thm (Berkowitz 1984): Det over ℤ is in NC2. O(log2n) intergers are bit-strings

NC2 Complete Problem Let C be a shallow O(log2n) depth circuit EVAL(C,a) is the problem of evaluating C under the assignment a Can be reduced to O(log2n) depth monotone layered circuit evaluation problem mEVAL What is the complexity class of mEVAL(C,a)? Answer: it’s complete for NC2 O(log2n) NC2 complete under AC0-reductions. e) NC2 Boolean circuit families: Let {Cn}∞n=1 be a family of Boolean circuits (with fan-in at most two ∨, ∧, ¬ gates). We say that this family is an NC2 circuit family if every circuit Cn in the family has depth O(log2 n) and size nO(n). A circuit taken from a given Boolean NC2 circuit family is said to be an NC2-circuit. It is known that the NC2 circuit value problem is complete under AC0-reductions for the class NC2 (Definition 2). We say that {Cn}∞n=1 is a uniform NC2-circuit family if its extended connection language is in FO (we refer the reader to [4, page 455] for the definitions). This definition coincides with Definition 2.

Bounded Arithmetic Formally: range over finite sets of numbers, encoding binary strings (with msb 1): e.g. {0,1,4} encodes string 10011 Essentially fragments of Peano Arithmetic with restricted comprehension axiom Following Cook-Nguyen (2010), two-sorted theory: Number sort: 𝑥 String sort: 𝑋 Language: Quantifi Example: Carry-Look-Ahead: Length of string (=max(X)+1)

Bounded Arithmetic Σ 0 𝐵 formulas: Σ 1 𝐵 formulas: uses no string quantifiers and all number quantifiers are bounded E.g., Carry(I,X,Y): “sets definable by constant depth circuits” Σ 1 𝐵 formulas: uses (block of) bounded existential string quantifier in front: ∃𝑋(|𝑋|≤𝑡∧𝜑) , for t a number term that doesn’t contain X, and 𝜑∈ Σ 0 𝐵 . ∃𝑖≤𝑡∀𝑗≤𝑖(𝑌 𝑗 =𝑍 𝑗 ) for t a number term that doesn’t contain x.

V0: Base theory for constant depth circuits Number sort: 𝑥 , String sort: 𝑋 Language: Basic axioms for the symbols of the language Examples: x+1≠0, x ≤ x+y Σ 0 𝐵 -Comprehension Axiom: Informally: “a set exists if it’s definable by a constant depth Boolean circuits.” for a bounded formula with no string quantifiers.

VNC2: theory for NC2 Number sort: 𝑥 , String sort: 𝑋 Language: Basic axioms for the symbols of the language Comprehension Axiom: for a bounded formula with no string quantifiers. NC2 axiom: “For every NC2 (monotone) circuit and assignment to the input gates, there exists a string that stores the evaluations of gates in the circuit, from the input gates up to the output gate”. output ⋀ O(log2n) ⋁ ⋁ 1 1

VNC2: theory for NC2 Thm (Cook-Ngyuen’10): A function is Σ 1 𝐵 -definable in VNC2 iff it is an NC2-function. Assume Boolean circuit compute function 𝐹( 𝑥 , 𝑋 ) whose output is bit-string 𝑥 number inputs in unary 𝑋 bit-strings inputs A function 𝐹( 𝑥 , 𝑋 ) is Σ 1 𝐵 -definable in VNC2 if there is a ϕ ∈Σ 1 𝐵 : with and perhaps the definablility definition should be for sigmaB1 definablility for simplicity

Our Result VNC2 ⊢ ∀A∀B, Det(A)Det(B) = Det(AB) VNC2 ⊢ ∀C triangular Det(C) = c11 ∙ ∙ ∙ cnn A,B,C matrices over ℤ Believed to be almost the weakest theory for that (we may also consider the theory V#L, which is sandwiched between VNC1 and VNC2 [Cook-Fontes’10]) Previously: only propositional proofs of these identities and only over GF(2) (Hrubes-T. (‘15)) Motivation is proof theoretical, complexity mate-mathematical; and proof complexity

Previous Work Soltys, Cook 2004: proof of matrix identities in V1, theory corresponding to P-time predicates. Hrubes-T. 2015: Polynomial-size NC2-Frege proofs of identities; Quasipolynomial-size Frege proofs of identities.

Algebraic circuit complexity The Proof in the Theory Algebraic circuit complexity VNC2 Illustration

˟ Algebraic circuits Fix a field 𝔽 (x1+x2)∙(x2+3)= x1x2+x22+3x1+3x2 An algebraic circuit over 𝔽 computes a formal polynomial over 𝔽 (x1+x2)∙(x2+3)= x1x2+x22+3x1+3x2 output ˟ + + x1 x2 3 14’ These statements can be encoded in the obvious way as n^2 equations Hard= super-polynomial size lower bounds Conj 2 also yields a quasipolynomial Frege proof

Defining Determinant Function in VNC2 Input: n × n integer matrix A Output: determinant of A Construct (syntactically) O(log2n)-depth algebraic circuit Detcirc that computes the determinant of n × n integer matrices: Circuit with divisions of high depth Division-elimination Eliminate high degrees (Homogenization) Balancing the circuit to O(log2n)-depth  “Detcirc” Evaluate the circuit “Detcirc” under assignment A. integers are represented as bit-strings

Defining Determinant Function in VNC2 Do all this in NC2 circuit Construct (syntactically) O(log2n)-depth algebraic circuit Detcirc that computes the determinant of n × n integer matrices: Circuit with divisions of high depth and syntactic-degree  Division-elimination Eliminate high degrees (Homogenization) Balancing the circuit to O(log2n)-depth  “Detcirc” Evaluate the circuit “Detcirc” under assignment A. integers are represented as bit-strings

Defining Determinant Function in VNC2 Do all this in NC2 circuit Construct (syntactically) O(log2n)-depth algebraic circuit Detcirc that computes the determinant of n × n integer matrices: Circuit with divisions of high depth and syntactic-degree  Division-elimination Eliminate high degrees (Homogenization) Balancing the circuit to O(log2n)-depth  “Detcirc” Evaluate the circuit “Detcirc” under assignment A. integers are represented as bit-strings

Step 1: Construct : circuit with division Inversion of matrix X using block-wise Inversion: , otherwise n>1: By induction this defines a (multi-output) polynomial size circuit for Schur’s Complement:

Step 2: Division Elimination To eliminate division gate 1/F define: where (constant term of polynomial computed by F) Then, = 1 + [monomials of degree > k] Proof: Abbreviate z= Then, = Now, eliminate high degrees Need to be nonzero.

There’s a Problem In we use an inverse. But we’re over Z ! In general we need to find a point that doesn’t nullify the inverse gate. Can’t do it in uniform VNC2. Solution: We show that having a=1 suffices.

We have defined the Det function in the theory. Now, how do we prove the determinant identities in the theory?

The Gist: Reflection Principle for balanced PI-proofs Show that VNC2 proves the existence of algebraic-equational proofs for polynomial identities (PI-proofs) of the determinant identities constructed in Hrubes-T. (‘15). Use soundness of balanced PI-proofs to conclude the identities are true.

PI-proofs Proofs for polynomial identities [Hrubeš and T. 2009] Proof-lines: equations between algebraic circuits Axioms: polynomial-ring axioms; f+g=g+f, etc. Start from axioms and derive new identities by derivation rules: ring identities axiom 2∙3=6 x=x reflexivity axiom product rule commutativity axiom 3x∙2=2∙3x 2∙3x=6x transitivity 3x∙2=6x

PI-proofs with Division Proofs for polynomial identities [Hrubeš and T. 2009] Proof-lines: equations between algebraic circuits w/ division gates Axioms: polynomial-ring axioms; f+g=g+f, etc. Add the axiom: F∙F-1 = 1

Proof Idea (in VNC2) Construct (syntactically) a PI-proof of the determinant identities: “F1=G1, F2=G2, …, Detcirc(X) Detcirc(Y)= Detcirc(XY)” Construction of PI-proof is done in stages: PI-proof with divisions of high depth Division-elimination from PI-proofs Homogenization: get rid of high degrees Balancing the circuit in PI- proofs to O(log2n)- depth integers are represented as bit-strings; here put picture of eqnlal proof. Maybe don’t need this---put it in the prvious ones! Or just make this the one, onstaed of “Gist” slide!

Proof Idea (in VNC2) Construct (syntactically) a PI-proof of the determinant identities: “F1=G1, F2=G2, …, Detcirc(X) Detcirc(Y)= Detcirc(XY)” Use soundness of proof to conclude: Detcirc(A) Detcirc(B)= Detcirc(AB) By definition of Det, we conclude ∀A ∀ B, Det(A)Det(B)=Det(AB) integers are represented as bit-strings; here put picture of eqnlal proof. Maybe don’t need this---put it in the prvious ones! Or just make this the one, onstaed of “Gist” slide!

Why do We Need this Reflection Principle? Well, we don’t know how to prove it differently… We need division gates to simulate Gaussian-elimination and related proofs This circumvent the need to prove the correctness of the (division free) Berkowitz (’84) algorithm.

Challenges Algorithmic challenge: Proof-theoretical challenge: We have evaluation function only for balanced Boolean circuits Challenges Recall: Every Σ 1 𝐵 -definable function in VNC2 is an NC2 function. Algorithmic challenge: Construct balanced PI-proofs for determinant identities Balance circuits within the class in NC2 Proof-theoretical challenge: VNC2 proves meaningful statements about these (NC2) constructions

Challenges (cont.) Uniformity: Solution: All constructions should be uniform PI-proofs in HT’15 used Strassen’73 division gate elimination: highly non-uniform: find field assignments that don’t nullify a given polynomial. Solution: Reason only about specific division gates elimination. The theory don’t evaluate circuits with division, only treats them as syntactic objects.

Corollaries We provide NC2 algorithms for balancing algebraic circuits homogenizing them evaluating them (given maximal syntactic- degrees) [Allender et al. ’98, Miller et al. ’88] All algorithms are Σ 1 𝐵 -definable in VNC2 Cayley-Hamilton Theorem in VNC2

Thank you for Listening!