1. Linear Algebra in Weak Formal Theories of Arithmetic
Iddo Tzameret
Royal Holloway, University of London
Joint work with
Stephen A. Cook
University of Toronto

2. Extreme Reverse Mathematics of Linear Algebra
Iddo Tzameret
Royal Holloway, University of London
Joint work with
Stephen A. Cook
University of Toronto

3. In Short

4. Basic statements of linear algebra are provable in a weak formal theory of arithmetic.
The weak theory corresponds to the computational complexity in which we can compute the determinant.
This is almost the weakest theory believed to prove such statements.
Was open (non-uniform was proved by Hrubes-T. 2015)

5. The Protagonists

6. The Determinant The Determinant Identities: (1) Det 𝐴𝐵 =Det 𝐴 Det(𝐵)
Det 𝑋 ≔ 𝜎∈ 𝑆 𝑛 sgn(𝜎) 𝑖=1 𝑛 𝑥 𝑖𝜎(𝑗)
The Determinant Identities:
(1) Det 𝐴𝐵 =Det 𝐴 Det(𝐵)
(2) Det 𝐶 = 𝑐 11 ⋯ 𝑐 𝑛𝑛 ,
for 𝐶 a triangular matrix

7. NC2 Complexity Class Thm (Berkowitz 1984): Det over ℤ is in NC2.
Boolean circuits of poly(n) size and O(log2n) depth
“Parallel Computation”
“Balanced Circuits”
Uniform (FO uniform)
Thm (Berkowitz 1984): Det over ℤ is in NC2.
O(log2n)
intergers are bit-strings

8. NC2 Complete Problem Let C be a shallow O(log2n) depth circuit
EVAL(C,a) is the problem of evaluating C
under the assignment a
Can be reduced to O(log2n) depth monotone layered circuit evaluation problem mEVAL
What is the complexity class of mEVAL(C,a)?
Answer: it’s complete for NC2
O(log2n)
NC2 complete under AC0-reductions.
e) NC2 Boolean circuit families: Let {Cn}∞n=1 be a
family of Boolean circuits (with fan-in at most two ∨, ∧, ¬ gates). We say that this family is an NC2 circuit family if
every circuit Cn in the family has depth O(log2 n) and size
nO(n). A circuit taken from a given Boolean NC2 circuit
family is said to be an NC2-circuit. It is known that the
NC2 circuit value problem is complete under AC0-reductions
for the class NC2 (Definition 2). We say that {Cn}∞n=1 is
a uniform NC2-circuit family if its extended connection
language is in FO (we refer the reader to [4, page 455] for
the definitions). This definition coincides with Definition 2.

9. Bounded Arithmetic
Formally: range over finite sets of numbers, encoding binary strings (with msb 1): e.g. {0,1,4} encodes string 10011
Essentially fragments of Peano Arithmetic with restricted comprehension axiom
Following Cook-Nguyen (2010), two-sorted theory:
Number sort: 𝑥 String sort: 𝑋
Language: Quantifi
Example: Carry-Look-Ahead:
Length of string (=max(X)+1)

10. Bounded Arithmetic Σ 0 𝐵 formulas: Σ 1 𝐵 formulas:
uses no string quantifiers and
all number quantifiers are bounded
E.g., Carry(I,X,Y):
“sets definable by constant depth circuits”
Σ 1 𝐵 formulas:
uses (block of) bounded existential string quantifier in front: ∃𝑋(|𝑋|≤𝑡∧𝜑) , for t a number term that doesn’t contain X, and
𝜑∈ Σ 0 𝐵 .
∃𝑖≤𝑡∀𝑗≤𝑖(𝑌 𝑗 =𝑍 𝑗 ) for t a number term that doesn’t contain x.

11. V0: Base theory for constant depth circuits
Number sort: 𝑥 , String sort: 𝑋
Language:
Basic axioms for the symbols of the language
Examples: x+1≠0, x ≤ x+y
Σ 0 𝐵 -Comprehension Axiom:
Informally: “a set exists if it’s definable by a constant depth Boolean circuits.”
for a bounded formula with no string quantifiers.

12. VNC2: theory for NC2 Number sort: 𝑥 , String sort: 𝑋 Language:
Basic axioms for the symbols of the language
Comprehension Axiom:
for a bounded formula with no string quantifiers.
NC2 axiom: “For every NC2 (monotone) circuit and assignment to the input gates, there exists a string that stores the evaluations of gates in the circuit, from the input gates up to the output gate”.
output ⋀
O(log2n) ⋁ ⋁ 1 1

13. VNC2: theory for NC2
Thm (Cook-Ngyuen’10): A function is Σ 1 𝐵 -definable in VNC2 iff it is an NC2-function.
Assume Boolean circuit compute function 𝐹( 𝑥 , 𝑋 ) whose output is bit-string
𝑥 number inputs in unary
𝑋 bit-strings inputs
A function 𝐹( 𝑥 , 𝑋 ) is Σ 1 𝐵 -definable in VNC2 if there is a ϕ ∈Σ 1 𝐵 :
with
and
perhaps the definablility definition should be for sigmaB1 definablility for simplicity

14. Our Result VNC2 ⊢ ∀A∀B, Det(A)Det(B) = Det(AB)
VNC2 ⊢ ∀C triangular Det(C) = c11 ∙ ∙ ∙ cnn
A,B,C matrices over ℤ
Believed to be almost the weakest theory for that (we may also consider the theory V#L, which is sandwiched between VNC1 and VNC2 [Cook-Fontes’10])
Previously: only propositional proofs of these identities and only over GF(2) (Hrubes-T. (‘15))
Motivation is proof theoretical, complexity mate-mathematical; and proof complexity

15. Previous Work
Soltys, Cook 2004: proof of matrix identities in V1, theory corresponding to P-time predicates.
Hrubes-T. 2015:
Polynomial-size NC2-Frege proofs of identities;
Quasipolynomial-size Frege proofs of identities.

16. Algebraic circuit complexity
The Proof in the Theory
Algebraic circuit complexity
VNC2
Illustration

17. ˟ Algebraic circuits Fix a field 𝔽 (x1+x2)∙(x2+3)= x1x2+x22+3x1+3x2
An algebraic circuit over 𝔽 computes a formal
polynomial over 𝔽
(x1+x2)∙(x2+3)= x1x2+x22+3x1+3x2
output ˟ + + x1 x2 3
14’
These statements can be encoded in the obvious way as n^2 equations
Hard= super-polynomial size lower bounds
Conj 2 also yields a quasipolynomial Frege proof

18. Defining Determinant Function in VNC2
Input: n × n integer matrix A
Output: determinant of A
Construct (syntactically) O(log2n)-depth algebraic circuit Detcirc that computes the determinant of n × n integer matrices:
Circuit with divisions of high depth
Division-elimination
Eliminate high degrees (Homogenization)
Balancing the circuit to O(log2n)-depth  “Detcirc”
Evaluate the circuit “Detcirc” under assignment A.
integers are represented as bit-strings

19. Defining Determinant Function in VNC2
Do all this in NC2 circuit
Construct (syntactically) O(log2n)-depth algebraic circuit Detcirc that computes the determinant of n × n integer matrices:
Circuit with divisions of high depth and syntactic-degree 
Division-elimination
Eliminate high degrees (Homogenization)
Balancing the circuit to O(log2n)-depth  “Detcirc”
Evaluate the circuit “Detcirc” under assignment A.
integers are represented as bit-strings

20. Defining Determinant Function in VNC2
Do all this in NC2 circuit
Construct (syntactically) O(log2n)-depth algebraic circuit Detcirc that computes the determinant of n × n integer matrices:
Circuit with divisions of high depth and syntactic-degree 
Division-elimination
Eliminate high degrees (Homogenization)
Balancing the circuit to O(log2n)-depth  “Detcirc”
Evaluate the circuit “Detcirc” under assignment A.
integers are represented as bit-strings

21. Step 1: Construct : circuit with division
Inversion of matrix X using block-wise Inversion:
, otherwise n>1:
By induction this defines a (multi-output) polynomial size circuit for
Schur’s Complement:

22. Step 2: Division Elimination
To eliminate division gate 1/F define:
where (constant term of polynomial computed by F)
Then, = 1 + [monomials of degree > k]
Proof:
Abbreviate z=
Then, =
Now, eliminate high degrees
Need to be nonzero.

23. There’s a Problem In we use an inverse. But we’re over Z !
In general we need to find a point that doesn’t nullify the inverse gate. Can’t do it in uniform VNC2.
Solution: We show that having a=1 suffices.

24. We have defined the Det function in the theory.
Now, how do we prove the determinant identities in the theory?

25. The Gist: Reflection Principle for balanced PI-proofs
Show that VNC2 proves the existence of algebraic-equational proofs for polynomial identities (PI-proofs) of the determinant identities constructed in Hrubes-T. (‘15).
Use soundness of balanced PI-proofs to conclude the identities are true.

26. PI-proofs Proofs for polynomial identities [Hrubeš and T. 2009]
Proof-lines: equations between algebraic circuits
Axioms: polynomial-ring axioms; f+g=g+f, etc.
Start from axioms and derive new identities by derivation rules:
ring identities axiom
2∙3=6
x=x
reflexivity axiom
product rule
commutativity axiom
3x∙2=2∙3x
2∙3x=6x
transitivity
3x∙2=6x

27. PI-proofs with Division
Proofs for polynomial identities [Hrubeš and T. 2009]
Proof-lines: equations between algebraic circuits w/ division gates
Axioms: polynomial-ring axioms; f+g=g+f, etc.
Add the axiom: F∙F-1 = 1

28. Proof Idea (in VNC2)
Construct (syntactically) a PI-proof of the determinant identities:
“F1=G1, F2=G2, …, Detcirc(X) Detcirc(Y)= Detcirc(XY)”
Construction of PI-proof is done in stages:
PI-proof with divisions of high depth
Division-elimination from PI-proofs
Homogenization: get rid of high degrees
Balancing the circuit in PI- proofs to O(log2n)- depth
integers are represented as bit-strings;
here put picture of eqnlal proof.
Maybe don’t need this---put it in the prvious ones! Or just make this the one, onstaed of “Gist” slide!

29. Proof Idea (in VNC2)
Construct (syntactically) a PI-proof of the determinant identities:
“F1=G1, F2=G2, …, Detcirc(X) Detcirc(Y)= Detcirc(XY)”
Use soundness of proof to conclude: Detcirc(A) Detcirc(B)= Detcirc(AB)
By definition of Det, we conclude
∀A ∀ B, Det(A)Det(B)=Det(AB)
integers are represented as bit-strings;
here put picture of eqnlal proof.
Maybe don’t need this---put it in the prvious ones! Or just make this the one, onstaed of “Gist” slide!

30. Why do We Need this Reflection Principle?
Well, we don’t know how to prove it differently…
We need division gates to simulate Gaussian-elimination and related proofs
This circumvent the need to prove the correctness of the (division free) Berkowitz (’84) algorithm.

31. Challenges Algorithmic challenge: Proof-theoretical challenge:
We have evaluation function only for balanced Boolean circuits
Challenges
Recall: Every Σ 1 𝐵 -definable function in VNC2 is an NC2 function.
Algorithmic challenge:
Construct balanced PI-proofs for determinant identities
Balance circuits within the class in NC2
Proof-theoretical challenge:
VNC2 proves meaningful statements about these (NC2) constructions

32. Challenges (cont.) Uniformity: Solution:
All constructions should be uniform
PI-proofs in HT’15 used Strassen’73 division gate elimination: highly non-uniform: find field assignments that don’t nullify a given polynomial.
Solution:
Reason only about specific division gates elimination.
The theory don’t evaluate circuits with division, only treats them as syntactic objects.

33. Corollaries We provide NC2 algorithms for
balancing algebraic circuits
homogenizing them
evaluating them (given maximal syntactic- degrees) [Allender et al. ’98, Miller et al. ’88]
All algorithms are Σ 1 𝐵 -definable in VNC2
Cayley-Hamilton Theorem in VNC2

34. Thank you for Listening!