1. Numerical Linear Algebra
Mo Mu
March 15,6:30-9:20

2. Linear Algebra Problems
Solving Linear System Equations
A u = b
Matrix Factorizations with Applications in Data Analysis, Eigenvalue/Singular Value Computation,etc

3. Solving Linear System Equations
Direct Methods
Gaussian elimination, pivoting, LU factorization
Band solvers
Sparse solvers
Minimum fill-in
Nested disection
Multi-frontal
Iterative Methods

4. Solving Linear System Equations
Iterative Methods
Basic iterative methods
Simple (one step) linear iterative methods
Optimization based directional searching methods
Advanced iterative techniques
Preconditioning
Polynomial acceleration
Multigrid
Domain decomposition
Subspace decomposition
Others: MinRes, GMRes, etc.

5. Basic Linear Iterative Methods
Simple (one step) linear iterative methods (Linear fixed point iteration with the fixed point being the solution of the original linear system)
un+1 = g(un)
= Gun + k
Residual correction (RF method, with G = I-A)
rn = b – Aun
un+1 = un + rn
= (I - A) un + b
Matrix splitting for fixed point, with A = Q-N, or A = D – L –U specifically, which leads to
Jacobi method: D u = (L+U) u = b, where Q = D
Gauss-Seidel method: (D-L)u = U u + b, where Q = D-L
SOR method, extrapolation with Gauss-Seidel
Error correction, Improved RF, or Preconditioning

6. Basic Linear Iterative Methods, continued
Error correction, Improved RF
Compute rn = b – Aun;
Solve error equation approximately
Aen = rn ,or compute en =A-1 rn, by e = Brn,
with B being an approximation to A-1, called a preconditioner, and easily computable for Brn
Error correction
un+1 = un + e
Preconditioning
un+1= un + B rn
= (I - BA) un + Bb,
G = I – B A = I – Q-1 A (in D. Young, where Q is called a splitting matrix, with A = Q-N) in the linear fixed point iteration;
This may be viewed as by applying B = Q-1 to the original system (preconditioning), then do RF

7. Directional Searching Methods
Optimization
Directional searching
Steepest descent method
Conjugate gradient method
A BRIEF INTRODUCTION TO THE CONJUGATE GRADIENT METHOD

8. Linear Iteration and Preconditioners (J. Xu)
Proposition 2.2 (J. XU) A symmetric iterative scheme gives rise to a preconditioner B for PCG, and the rate of the convergence of the iterative scheme may be accelerated by using PCG
Proposition 2.3 (J. XU) Any preconditioner can also be used to construct a linear iterative scheme by the extrapolated preconditioned RF, with the optimal choice of the parameter:
un+1= un + ωB rn

9. Multigrid Methods

10. Domain Decompsition

11. Subspace Correction

12. Matrix Factorizations