1. CS 290H 31 October and 2 November Support graph preconditioners Final projects: Read and present two related papers on a topic not covered in class Or, experiment with preconditioning in an application of your choice Either way, talk to me by November 9 about your project Laplacian and generalized Laplacian matrices Maximum-weight spanning tree preconditioners Algebraic analysis of congestion/dilation and condition number Augmenting the MST to improve congestion & dilation Extensions: Reverse support – MILU analysis Positive matrix elements – diagonally dominant SPSD and matroid analysis Hierarchical partitioning – Spielman/Teng Remarks on factor width

2. Support Graph Preconditioning +: New analytic tools, some new preconditioners +: Can use existing direct-methods software -: Current theory and techniques limited CFIM: Complete factorization of incomplete matrix Define a preconditioner B for matrix A Explicitly compute the factorization B = LU Choose nonzero structure of B to make factoring cheap (using combinatorial tools from direct methods) Prove bounds on condition number using both algebraic and combinatorial tools

3. Definitions The Laplacian matrix of an n-vertex undirected graph G is the n-by-n symmetric matrix A with a ij = -1 if i ≠ j and (i, j) is an edge of G a ij = 0 if i ≠ j and (i, j) is not an edge of G a ii = the number of edges incident on vertex i Theorem: The Laplacian matrix of G is symmetric, singular, and positive semidefinite. The multiplicity of 0 as an eigenvalue is equal to the number of connected components of G. A generalized Laplacian matrix (more accurately, a symmetric weakly diagonally dominant M-matrix) is an n-by-n symmetric matrix A with a ij ≤ 0 if i ≠ j a ii ≥ Σ |a ij | where the sum is over j ≠ i

4. Edge-vertex factorization of generalized Laplacians A generalized Laplacian matrix A can be factored as A = UU T where U has: a row for each vertex a column for each edge, with two nonzeros of opposite sign a column for each excess-weight vertex, with one nonzero

5. Spanning Tree Preconditioner Spanning Tree Preconditioner [Vaidya] A is generalized Laplacian (symmetric positive semidefinite with negative off-diagonal nzs) B is the gen Laplacian of a maximum-weight spanning tree for A (with diagonal modified to preserve row sums) Form B: costs O(n log n) or less time (graph algorithms for MST) Factorize B = R T R: costs O(n) space and O(n) time (sparse Cholesky) Apply B -1 : costs O(n) time per iteration G(A) G(B)

6. Combinatorial analysis: cost of preconditioning A is generalized Laplacian (symmetric positive semidefinite with negative off-diagonal nzs) B is the gen Laplacian of a maximum-weight spanning tree for A (with diagonal modified to preserve row sums) Form B: costs O(n log n) time or less (graph algorithms for MST) Factorize B = R T R: costs O(n) space and O(n) time (sparse Cholesky) Apply B -1 : costs O(n) time per iteration (two triangular solves) G(A) G(B)

7. Combinatorial analysis : Graphs and sparse Cholesky 10 1 3 2 4 5 6 7 8 9 1 3 2 4 5 6 7 8 9 G(A) G + (A) [chordal] Symmetric Gaussian elimination: for j = 1 to n add edges between j’s higher-numbered neighbors Fill: new nonzeros in factor

8. Numerical analysis: quality of preconditioner support each edge of A by a path in B dilation(A edge) = length of supporting path in B congestion(B edge) = # of supported A edges p = max congestion, q = max dilation condition number κ (B -1 A) bounded by p·q (at most O(n 2 )) G(A) G(B)

9. Numerical analysis: Support numbers Intuition from networks of electrical resistors: graph = circuit; edge = resistor; weight = 1/resistance = conductance How much must you amplify B to provide as much conductance as A? How big does t need to be for tB – A to be positive semidefinite? What is the largest eigenvalue of B -1 A ? The support of B for A is σ(A, B) = min { τ : x T (tB – A)x  0 for all x and all t  τ } If A and B are SPD then σ(A, B) = max{ λ : Ax = λ Bx} = λ max (A, B) Theorem: If A and B are SPD then κ (B -1 A) = σ(A, B) · σ(B, A)

10. Old analysis, splitting into paths and edges Split A = A 1 + A 2 + ··· + A k and B = B 1 + B 2 + ··· + B k such that A i and B i are positive semidefinite Typically they correspond to pieces of the graphs of A and B (edge, path, small subgraph) Theorem: σ(A, B)  max i {σ(A i, B i )} Lemma: σ(edge, path)  (worst weight ratio) · (path length) In the MST case: A i is an edge and B i is a path, to give σ(A, B)  p·q B i is an edge and A i is the same edge, to give σ(B, A)  1

11. New analysis: Algebraic Embedding Lemma New analysis: Algebraic Embedding Lemma vv [Boman/Hendrickson] Lemma: If V·W=U, then σ(U·U T, V·V T )  ||W|| 2 2 (with equality for some choice of W) Proof: take t  ||W|| 2 2 = λ max ( W·W T ) = max y  0 { y T W·W T y / y T y } then y T (tI - W·W T ) y  0 for all y letting y = V T x gives x T (tV·V T - U·U T ) x  0 for all x recall σ(A, B) = min{ τ : x T (tB – A)x  0 for all x, all t  τ } thus σ(U·U T, V·V T )  ||W|| 2 2

12. A B - a 2 - b 2 -a 2 -c 2 -b 2 [ ] a 2 +b 2 - a 2 - b 2 - a 2 a 2 +c 2 - c 2 - b 2 - c 2 b 2 +c 2 [ ] a 2 +b 2 - a 2 - b 2 - a 2 a 2 - b 2 b 2 [ ] a b - a c - b - c [ ] a b - a c - b U V =VV T =UU T [ ] 1 - c / a 1 c / b /b W = x σ(A, B)  ||W|| 2 2  ||W||  x ||W|| 1 = (max row sum) x (max col sum)  (max congestion) x (max dilation)

13. Spanning Tree Preconditioner Spanning Tree Preconditioner [Vaidya] can improve congestion and dilation by adding a few strategically chosen edges to B cost of factor+solve is O(n 1.75 ), or O(n 1.2 ) if A is planar in experiments by Chen & Toledo, often better than drop-tolerance MIC for 2D problems, but not for 3D. G(A) G(B)

14. Complexity of direct methods n 1/2 n 1/3 2D3D Space (fill): O(n log n)O(n 4/3 ) Time (flops): O(n 3/2 )O(n 2 ) Time and space to solve any problem on any well- shaped finite element mesh

15. Support-graph analysis of modified incomplete Cholesky B has positive (dotted) edges that cancel fill B has same row sums as A Strategy: Use the negative edges of B to support both the negative edges of A and the positive edges of B. -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 A A = 2D model Poisson problem.5 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 B B = MIC preconditioner for A

16. Supporting positive edges of B Every dotted (positive) edge in B is supported by two paths in B Each solid edge of B supports one or two dotted edges Tune fractions to support each dotted edge exactly 1/(2  n – 2) of each solid edge is left over to support an edge of A

17. Analysis of MIC: Summary Each edge of A is supported by the leftover 1/(2  n – 2) fraction of the same edge of B. Therefore σ(A, B)  2  n – 2 Easy to show σ(B, A)  1 For this 2D model problem, condition number is O(n 1/2 ) Similar argument in 3D gives condition number O(n 1/3 ) or O(n 2/3 ) (depending on boundary conditions)

18. Open problems I Other subgraph constructions for better bounds on ||W|| 2 2 ? For example [Boman], ||W|| 2 2  ||W|| F 2 = sum(w ij 2 ) = sum of (weighted) dilations, and [Alon, Karp, Peleg, West] show there exists a spanning tree with average weighted dilation exp(O((log n loglog n) 1/2 )) = o(n  ); this gives condition number O(n 1+  ) and solution time O(n 1.5+  ), compared to Vaidya O(n 1.75 ) with augmented spanning tree [Spielman, Teng] recursive partitioning construction gives solution time O(n 1+  ) for all generalized Laplacians! (Uses yet another matrix norm inequality.) Is there a construction that minimizes ||W|| 2 2 directly?

19. Open problems II Make spanning tree methods more effective in 3D? Vaidya gives O(n 1.75 ) in general, O(n 1.2 ) in 2D Issue: 2D uses bounded excluded minors, not just separators Support graph methods for more general matrices? All SPD matrices? ([Boman, Chen, Hendrickson, Toledo]: different matroid for all diagonally dominant SPD matrices) Finite element problems? ([Boman]: Element-by-element preconditioner for bilinear quadrilateral elements) Matrices of bounded factor width?

20. Hierarchy of matrix classes (all real) General nonsymmetric Diagonalizable Normal Symmetric indefinite Symmetric positive (semi)definite = Factor width n Factor width k... Factor width 4 Factor width 3 Diagonally dominant SPSD = Factor width 2 Generalized Laplacian = Symm diag dominant M-matrix Graph Laplacian