1. Matrix Factorization Lecture #7 EEE 574 Dr. Dan Tylavsky

2. Matrix Factorization © Copyright 1999 Daniel Tylavsky 4 Let’s look at solving Ax=b –A, b are dense. 4 Two possible (general approaches): –Indirect Methods - methods which asymptotically approach (but never reach) the true solution as the number of steps increases. Ex: Point Jacobi, Gauss-Siedel, Successive Over relaxation, Multigrid

3. Matrix Factorization © Copyright 1999 Daniel Tylavsky –Direct Methods - methods that yield the theoretically exact result in a fixed number of steps. (Assuming an infinitely precise computing engine.) Ex: Elimination Methods –Gauss Elimination –Gauss-Jordan Elimination –Cholesky Factorization (Numerically Symmetric) –LU or LDU factorization (a.k.a., Product form of the Inverse),(Variations Include Crout, Doolittle, Banachiewicz) Orthogonalization Methods –QR factorization –Given Rotations

4. Matrix Factorization © Copyright 1999 Daniel Tylavsky Conjugate Gradient Methods - Exact solution in # of steps equal to the number of different eigenvalues. –Various Variations 4 Advantages of Elimination Methods –No convergence criteria (however, pivoting in general non-positive definite case.) –Factors can be used repeatedly. –Factors can be ‘easily’ modified to accommodate matrix/network changes. –Partial factorized matrices give linear network equivalents.

5. Matrix Factorization © Copyright 1999 Daniel Tylavsky –Definitions Symmetric:A=A T Positive Definite: x T Ax>0, (A>0 is short hand notation) Diagonally Dominant:,with inequality for at least one i. Properly Diagonally Dominant:for all i. Rank: Order of the largest sub-matrix with nonzero determinant.

6. Matrix Factorization © Copyright 1999 Daniel Tylavsky –Properties Symmetric pos. def. (A>0) matrix has all positive real eigenvalues. If A>0, then Det(A)=|A|>0. A rank 1 matrix must have the outer-product form, xy T. Symmetric A>0 has a unique Cholesky factorization. A=U T U, A=U’ T DU’ If A is properly diagonally dominant and all of its diagonal elements are positive, then A is positive definite.

7. Matrix Factorization © Copyright 1999 Daniel Tylavsky –Factorizing a non-symmetric matrix. –I claim A can be expanded in the following form:

8. Matrix Factorization © Copyright 1999 Daniel Tylavsky –To find the value of A 2, multiply. –Therefore: Outer product term of first row & first column (absent. diag. element)

9. Matrix Factorization © Copyright 1999 Daniel Tylavsky –Do this repeatedly until we have the product form of A.

10. Matrix Factorization © Copyright 1999 Daniel Tylavsky –To find A3, multiply the result. –We can write:

11. Matrix Factorization © Copyright 1999 Daniel Tylavsky –Continuing on yields: –where: –We can write:

12. Matrix Factorization © Copyright 1999 Daniel Tylavsky –Elementary-Matrix-Multiplication-Superposition Principal. ijij

13. Matrix Factorization © Copyright 1999 Daniel Tylavsky –This implies I can write:

14. Matrix Factorization © Copyright 1999 Daniel Tylavsky –Taking the transpose of the elementary-matrix- multiplication-superposition principal allows me to write: –Hence we can write:

15. Matrix Factorization © Copyright 1999 Daniel Tylavsky –Let’s work though an example: –Divide Top Row by a 11. –Compute –Divide 2nd Row by  2. –Divide 3rd Row by  3. –Compute

16. Matrix Factorization © Copyright 1999 Daniel Tylavsky 4 Individual Problem: Verify that previous answer is correct. 4 Note that result isn’t symmetrical. 4 Individual Problem: Factor A=LU for:

17. Matrix Factorization © Copyright 1999 Daniel Tylavsky 4 To make this symmetric use the L’DL’ T form. –Let –Write: –Division by diag. of row & col. Performed @ same time.

18. Matrix Factorization © Copyright 1999 Daniel Tylavsky 4 Individual Problem: Modify LU factors of A: 4 To get L’DL’ T :

19. Matrix Factorization © Copyright 1999 Daniel Tylavsky 4 Data structure for storing LU factors? 4 For LU factors could use LC-D-UR/X:

20. Matrix Factorization © Copyright 1999 Daniel Tylavsky 4 For L’DL’ T factors could use CR(L)-D/X:

21. Matrix Factorization © Copyright 1999 Daniel Tylavsky –Divide Top Row by a 11. –Compute for 2nd row. –Divide 2nd Row by  2. –If we store & work through A by rows, factorization is done in a slightly different way: –Compute and for 3rd row.

22. Matrix Factorization © Copyright 1999 Daniel Tylavsky 4 Individual Problem: Finish the factorization.

23. Matrix Factorization © Copyright 1999 Daniel Tylavsky –Cholesky Factorization (Symmetric A=LL T.) –A can be expanded in the following form: Etc. –Not used. Calculating square-roots is expensive.

24. The End