1. Chapter 10
LU Decomposition

2. L and U Matrices
Lower Triangular Matrix
Upper Triangular Matrix

3. LU Decomposition Another method for solving matrix equations
Idea behind the LU Decomposition - start with
We know (because we did it in Gauss Elimination) we can write

4. LU Decomposition
Assume there exists [L]
Such that
This implies

5. The Steps of LU Decomposition

6. LU Decomposition LU Decomposition Decomposition Methods (not unique)
* Based on Gauss elimination
*More efficient
Decomposition Methods (not unique)
* Doolittle decomposition lii = 1
* Crout decomposition uii = 1 (omitted)
* Cholesky decomposition (for symmetric matrices) uii = lii

7. LU Decomposition Three Basic Steps
(1) Factor (decompose) [A] into [L] and [U]
(2) given {b}, determine {d} from [L]{d} = {b}
(3) using [U]{x} = {d} and back-substitution, solve for {x}
Advantage: Once we have [L] and [U], we can use many different {b}’s without repeating the decomposition process

8. LU Decomposition LU decomposition / factorization
[ A ] { x } = [ L ] [ U ] { x } = { b }
Forward substitution
[ L ] { d } = { b }
Back substitution
[ U ] { x } = { d }
Forward substitutions are more efficient than elimination

9. Simple Truss
F45 4 5
F35
F14
F24
F25 1 3 H1
F12
F23 2 V1 V2 W

10. Exampe: Forces in a Simple Truss
[A] depends on geometry only; but {b} varies with applied load

11. LU Factorization [ A ] = [ L ] [ U ] Gauss elimination [A]{x}={b}
elimination steps need to be repeated for each different {b}
LU factorization / decomposition
[ A ] = [ L ] [ U ]
does not involve the RHS {b} !!!

12. Doolittle LU Decomposition
Doolittle Algorithm
Get 1’s on diagonal of [L] (lii =1)
Operate on rows and columns sequentially, narrowing down to single element
Identical to LU decomposition based on Gauss elimination (see pp ), but different approach

13. Doolittle LU Decomposition

14. Doolittle LU Decomposition

15. Doolittle LU Decomposition

16. Algorithm of Doolittle Decomposition

17. Very efficient for large matrices !
Forward Substitution
Once [L] is formed, we can use forward substitution instead of forward elimination for different {b}’s
Very efficient for large matrices !

18. Identical to Gauss elimination
Back Substitution
Identical to Gauss elimination

19. Forward Substitution
Example:

20. Back-Substitution

21. Forward and Back Substitutions
Forward-substitution
Back-substitution (identical to Gauss elimination)
Error in textbook (p. 165)

22. Forward and Back Substitutions

23. Example: Forward and Back Substitutions
» [L,U] = LU_factor(A);
L =
U =
» x=LU_solve(L,U,b)
x =
0.2286
0.4714
(without pivoting)
MATLAB M-file
LU_factor
Forward and back substitution

24. Pivoting in LU Decomposition
Still need pivoting in LU decomposition
Messes up order of [L]
What to do?
Need to pivot both [L] and a permutation matrix [P]
Initialize [P] as identity matrix and pivot when [A] is pivoted  Also pivot [L]

25. LU Decomposition with Pivoting
Permutation matrix [ P ]
- permutation of identity matrix [ I ]
Permutation matrix performs “bookkeeping” associated with the row exchanges
Permuted matrix [ P ] [ A ]
LU factorization of the permuted matrix
[ P ] [ A ] = [ L ] [ U ]
Solution
[ L ] [ U ] {x} = [ P ] {b}

26. Permutation Matrix Bookkeeping for row exchanges
Example: [ P1] interchanges row 1 and 3
Multiple permutations [ P ]

27. LU Decomposition with Pivoting
Start with
No need to consider {b} in decomposition

28. Forward Elimination Gauss elimination of first column
Interchange rows 1 & 4
Gauss elimination of first column
Save fi1 in the first column of [L]

29. Forward Elimination Gauss elimination of second column
No interchange required
Gauss elimination of second column
Save fi2 in the second column of [L]

30. Forward Elimination Partial pivoting for [L] and [P]
Interchange rows 3 &4
Partial pivoting for [L] and [P]

31. Forward Elimination Gauss elimination of third column
Save fi3 in third column of [L]

32. LU Decomposition with Pivoting
Gauss elimination with partial pivoting

33. LU Decomposition with Pivoting
Forward substitution
Back substitution

34. LU Decomposition with Pivoting
partial pivoting

35. LU Decomposition with Pivoting
» A=[ ; ; ; ];
» b=[ ]';
» [L,U,P]=LU_pivot(A);
L =
U =
T1 =
T2 =
T1 = [L][U]
T2 = [P][A]
Verify T1= T2

36. LU Decomposition with Pivoting
» A=[ ; ; ; ];
» b=[1; -1; 2; 1];
» [L,U,P]=LU_pivot(A);
» Pb=P*b
Pb = 1 -1 2
» x=LU_Solve(L,U,Pb)
d =
1.0000
0.7143
2.3571
x =
0.4714
0.2286
LU Decomposition
[L][U]{x} = [P]{b}
Forward Substitution
Back Substitution

37. MATLAB’s Methods LU factorization [L,U] = lu (A)
-- returns [L] and [U]
[L, U, P] = lu (A)
-- returns also the permutation matrix [P]
Cholesky factorization: R = chol(A)
-- [A] = [R][R]
Determinant: det (A)

38. Forward and Back Substitutions LU Decomposition without Pivoting
MATLAB function lu
» A=[ ; ; ; ];
» b=[1; -1; 2; 1];
» [L,U]=lu(A)
L =
U =
» L*U
ans =
» d=L\b
d =
1.0000
0.7143
2.3571
» x=U\d
x =
0.2286
0.4714
Forward and Back Substitutions
LU Decomposition without Pivoting

39. Cholesky LU Factorization
If [A] is symmetric and positive definite, it is convenient to use Cholesky decomposition.
[A] = [L][L]T= [U]T[U]
Cholesky factorization can be used for either symmetric or non-symmetric matrices
No pivoting or scaling needed if [A] is symmetric and positive definite (all eigenvalues are positive)
If [A] is not positive definite, the procedure may encounter the square root of a negative number

40. Cholesky LU Factorization
For a general non-symmetric matrix
For symmetric and positive definite matrices

41. Cholesky LU Decomposition

42. Example: Cholesky LU

43. Cholesky LU Factorization
[A] = [U]T[U] = [U] [U]
Recurrence relations

44. Script file for Cholesky Decomposition
Symmetric Matrix L = U
Compute only the upper triangular elements

45. » A=[ ; ; ; ] A = » [L,U] = Cholesky(A) L = U =
Symmetric
[L] = [U]

46. MATLAB Function: chol Forward substitution Back substitution
» U = chol(A)
U =
» U'*U
ans =
» d = U'\b
d = 8 -3 3
» x = U\d
x = 1 -2
Forward substitution
Back substitution