Generating a d-dimensional linear subspace efficiently Raphael Yuster SODA’10

2 Problem statement Computing the rank of a matrix, and, more generally, generating a row-reduced equivalent matrix is a fundamental computational algebraic problem. Standard approach: Gaussian elimination. This takes O(n ω ) < O(n 2.38 ) time [H-B 74’, I-M-H 82’]. No faster algorithm is known for computing the rank of general n  n matrices. If matrix is sparse (only m << n 2 non-zero entries), can we compute a row reduced matrix faster? Not known (and probably impossible via G.E.)

3 Problem statement and main result If we allow randomness, the rank can be computed faster. O(m n) Monte Carlo [Wiedemann 86’]. O(n 3-1/(ω -1) ) if m=O(n) Las Vegas [Eberly et al. 07’]. Recall: If B is a row-reduced matrix of A then: B spans Rows(A). B has rank(A) rows. Any d rows of B span a d-dim. subspace of Rows(A). Our main result: generating a d-dimensional subspace of Rows(A) requires (in some cases significantly) less time than computing a row-reduced matrix.

4 Main result Let A be an n  n over an arbitrary field, with m nonzero entries, and let d  n:  There is an algorithm that computes a d  n matrix with full row rank which is a subspace of Rows(A).  If rank(A) < d the algorithm generates a rank(A)  n row-reduced matrix of A. The running time is: O(min{ n 2-2/ω m 1/ω d ω-2+1/ω, n 2 d ω-2 } ). Dense case Sparse case

5 Implications and comparison Can be used to answer queries of the form “rank(A) > d ?” A row-reduced matrix of A, and thus rank(A), computable in time which is a function of the rank (binary search). O(min{ n 2-2/ω m 1/ω rank(A) ω-2+1/ω, n 2 rank(A) ω-2 } ) If the matrix is dense, the runtime is O(n 2 d ω-2 ) < O(n 2 d 0.38 ). For d = o(n) this is faster than doing G.E. from scratch. Even more efficiency if matrix is sparse:

6 Implications and comparison If m = n 2-ε then for d suff. small we get better than O(n 2 d ω-2 ) and sometimes even better than O(n 2 ) : d < (n 2 /m) < (n 2 /m) 1/(ω-1) 2 sub-quadratic runtime. d < (n 2 /m) running time O(n 2 d ω-2 (md/n 2 ) 1/ω ). o(1)

7 An application Given a graph G with n vertices what’s the cardinality M(G) of the maximum matching? [Lovász 79’] proved that computing 2M(G) (randomly) reduces to computing the rank of an n  n matrix over a small finite field. Thus, his algorithm computes M(G) in O(n ω ) time. Our algorithm computes M(G) in O(n 2 M(G) ω-2 ) time, which is faster whenever M(G) = o(n).

8 Proof of main result - outline Proof consists of two parts: 1.Reducing the computation of a d-dim. subspace of a sparse matrix to the computation of a d-dim. subspace of a dense, but smaller, rectangular matrix. 2.Computing a d-dim. subspace of a (possibly dense, possibly rectangular) matrix by repeatedly filtering out non-essential rows and replacing other rows with a smaller equivalent set of rows spanning the same subspace.

9 Proof of main result – part 1 We need a result of Ibarra, Moran and Hui 82’: Lemma 1: Let B be an l  n matrix. Then a row-reduced matrix can be computed in O(n l ω-1 ) time. We prove: Sparse Lemma: Let A be an n  n matrix with m non- zeroes. Let d  n 2 /m. For any s with d  s  n there is a matrix L with at most dn/s rows so that: If rank(A) < d then L is row-equivalent to A. If rank(A)  d then Rows(L) is a subspace of dimension at least d of Rows(A). L is constructed in O(ms ω-1 ) time.

10 Proof of main result – part 1 Proof of Lemma 2: Partition the rows of A into t = n/s rectangular matrices, A 1,…,A t, each with s rows. Let c i denote the number of non-zero columns of A i. Compact A i by discarding zero columns (but remember their locations) thereby obtaining an s  c i matrix B i. Apply Lemma 1 to each B i to obtain reduced equivalent matrices C i with c i columns and rank(B i )=rank(A i ) rows.

11 Proof of main result – part 1 The time required to construct C i is O(c i s ω-1 ). So, to construct all C 1,…,C t takes O(ms ω-1 ). Let D i be the matrix obtained from C i by taking the first d rows of C i. Expand D i by reinserting the zero columns. If rank(A i ) < d then D i is row-equivalent to A i Otherwise, D i spanning a d-dim. subspace of Rows(A i ).

12 Proof of main result – part /4 -11/ / / A1A1 A2A2 A3A3 D1D1 C1C1 B1B1

13 Proof of main result – part 1 Let L be the union of the D i. L has at most dt = dn/s rows. If rank(A) < d then L is equivalent to A, as each D i is equivalent to A i. If rank(A)  d then Rows(L) is a subspace of Rows(A) with dimension at least d.

14 Proof of main result – part 2 Dense Lemma: Let L be an l  n matrix and let d  l /2. We construct L* with at most l /2 rows so that: If rank(L) < d then L* is row-equivalent to L. If rank(L)  d then Rows(L*) is a subspace of Rows(L) with dimension at least d. The running time is O( l nd ω-2 ).

15 Proof of main result – part 2 Proof of the main result – the algorithm: We are given an n  n matrix A containing m nonzero entries, and a positive integer d. If d < n 2 /m apply the sparse lemma with a value s that will be chosen later after optimization. The result is a matrix L 0 with n column and l 0  dn/s rows satisfying the conditions of the lemma. If d  n 2 /m we just set L 0 =A and l 0 = n. In any case now Rows(L 0 ) is a subspace of Rows(A) with dimension at least d (or else L 0 is equivalent to A).

16 Proof of main result – part 2 We check if d  l 0 /2. If so, we apply the dense lemma to L 0 and obtain L 1 with l 1  l 0 /2 rows. We repeatedly apply the dense lemma to L i as long as d  l i /2 and obtain the next matrix L i +1. We halt when d > l i /2. After halting, we perform a final Gaussian elimination on L i and obtain a reduced matrix B, which is the output of our algorithm.

17 Proof of main result – part 2 Proof of the main result- correctness: At any stage of the algorithm, the rows of the current matrix L i form a subspace of Rows(A), and hence also at the end Rows(B) is a subspace of Rows(A). Thus, if B has (at least) d rows then the final output is (at least) a d-dimensional subspace of Rows(A). If Rank(A) < d then each of the L i, as well as the final B, is row-equivalent to A.

18 Proof of main result – part 2 Proof of the main result- running time: Consider first the case when d  n 2 /m. In this case l 0 = n and the application of the dense lemma on L i requires O(2 -i n 2 d ω-2 ) time. After O(log n) applications the final G.E. to obtain B takes O(nd ω-1 ) time by Lemma 1. The overall runtime is O(n 2 d ω-2 ) as required. Consider next the case when d < n 2 /m. The application of the sparse lemma requires O(ms ω-1 ) time.

19 Proof of main result – part 2 As l 0  dn/s, the dense lemma on L i requires: O(2 -i s -1 n 2 d ω-1 ) time. After O(log n) applications the final G.E. to obtain B takes O(nd ω-1 ) time by Lemma 1. Thus, the overall running time is O(ms ω-1 + s -1 n 2 d ω-1 ) Choosing optimal value for s which is s = (d ω-1 n 2 /m) 1/ω we obtain a runtime of O(n 2-2/ω m 1/ω d ω-2+1/ω ) as required.

20 Thanks