1. Math for CSTutorial 2-31 Solution of Systems of Linear Equations Gaussian Elimination LU Decomposition Gram-Schmidt vector orthogonalization Tutorial 2-3. Contents

2. Math for CSTutorial 2-32 Linear Systems in Matrix Form  Linear equation means that every variable is in the power 1 only. If the variables in the equation are ordered x 1, … x n and the missing variables x i are written as … + 0·x i + …, then the linear equation can be written in the matrix notation, according to the matrix multiplication rules. Solution of linear equations was one of the reasons, for matrix notation invention. (1)

3. Math for CSTutorial 2-33 An inconsistent example: Geometric interpretation Rank{A}=1 Rank{A|b}=2≠ Rank{A} ERO:Multiply the first row with -2 and add to the second row

4. Math for CSTutorial 2-34 Full-rank systems If Rank{A}=n Det{A}  0  A -1 exists  Unique solution

5. Math for CSTutorial 2-35 First step of elimination Pivotal element

6. Math for CSTutorial 2-36 Second step of elimination Pivotal element

7. Math for CSTutorial 2-37 Back substitution algorithm

8. Math for CSTutorial 2-38 LU Decomposition A=LU Ax=b  LUx=b Define Ux=y Ly=bSolve y by forward substitution ERO’s must be performed on b as well as A The information about the ERO’s are stored in L Indeed y is obtained by applying ERO’s to b vector Ux=ySolve x by backward substitution

9. Math for CSTutorial 2-39 LU Decomposition by Gaussian elimination Compact storage: The diagonal entries of L matrix are all 1’s, they don’t need to be stored. LU is stored in a single matrix. There are infinitely many different ways to decompose A. Most popular one: U=Gaussian eliminated matrix L=Multipliers used for elimination

10. Math for CSTutorial 2-310 An example: Linear System Rank{A}=n  A -1 exists  Unique solution

11. Math for CSTutorial 2-311 Gaussian Elimination x 3 =3; -x 2 -2x 3 =-8; x 2 =2; x 1 +2*2+3*3=14; x 1 =14-4-9=1.

12. Math for CSTutorial 2-312 LU Factorization L -1 U=L -1 A Check that there were no mistakes: L -1 AU

13. Math for CSTutorial 2-313 LU Factorization Check that there were no mistakes: L -1 AU L -1 bbL -1 x 3 =3; -x 2 -2x 3 =-8; x 2 =2; x 1 +2*2+3*3=14; x 1 =14-4-9=1.

14. Math for CSTutorial 2-314 the Gram-Schmidt procedure for vector orthogonalization The purpose we can construct a set of orthonormal vectors u i from a set of n-dimensional vectors v i. 1≤i≤m And v i can be represented by the linear combination of u i

15. Math for CSTutorial 2-315 G-S procedure (cont.) 1.Select a vector from v i arbitrarily, say v 1 2.By normalizing its length, we obtain the first vector, say 3.Select v 2 and subtract the projection of v 2 onto u 1, we get w 2 =v 2 – (v 2 u 1 )u 1

16. Math for CSTutorial 2-316 G-S procedure (cont.) 4.And normalizing 5.Now we can check, that

17. Math for CSTutorial 2-317 G-S procedure (cont.) 6.The procedure continues by selecting v 3 and subtract its projection on u 1 and u 2, we have w 3 =v 3 – (v 3 u 1 )u 1 – (v 3 u 2 )u 2 7.Then, the orthonormal vector u 3 is 8. By continuing this procedure, we shall construct the set of orthonormal vectors u i

18. Math for CSTutorial 2-318 G-S procedure: Example Consider v 1 =(0,1,1); v 2 =(1,-1,0); v 3 =(1,0,1) They are not orthogonal: (v i,v j )≠0. Following G-S:

19. Math for CSTutorial 2-319 Example (cont.) We can check, that:

20. Math for CSTutorial 2-320 Example (cont.) Why?