Advanced Computer Graphics Spring 2014 K. H. Ko School of Mechatronics Gwangju Institute of Science and Technology

2 Today’s Topics Linear Algebra  Systems of Linear Equations  Matrices  Vector Spaces

3 Systems of Linear Equations Linear Equation System of Linear Equations (n equations, m unknowns)

4 Systems of Linear Equations Solve a system of n linear equations in m unknown variables  A common problem in applications  In most cases m = n.  The system has three cases No solutions, one solution or infinitely many solutions How to solve the system?  Forward elimination followed by back substitution

5 Systems of Linear Equations A closer look at two equations in two unknowns When the solution method needs to be implemented for a computer, a practical concern is the amount of time required to compute a solution.

6 Systems of Linear Equations Division is more expensive than multiplication and addition. 3 additions 3 multiplications 3 divisions 3 additions 5 multiplications 2 divisions

7 Gaussian Elimination Forward elimination + back substitution = Gaussian elimination

8 Gaussian Elimination Basic Operations for Forward Elimination

9 Gaussian Elimination Basic Operations for Forward Elimination

10 Gaussian Elimination Basic Operations for Forward Elimination

11 Gaussian Elimination Basic Operations for Back Substitution

12 Gaussian Elimination Example

13 Geometry of Linear Systems Consider

14 Geometry of Linear Systems Consider 3 equations and 3 unknowns

15 Numerical Issues If the pivot is nearly zero, the division can be a source of numerical errors.  Use of floating point arithmetic with limited precision is the main cause.

16 Numerical Issues A better algorithm involves searching the entries with the pivot that is largest in absolute magnitude. No division by ε. -> Numerically robust and stable.

17 Numerical Issues However, even the previous approach can be a problem. Swap columns to avoid such problem.  Blackboard!!!

18 Numerical Issues Generally, for a system of n equations in n unknowns…  Full Pivoting: Search the entire matrix of coefficients looking for the entry of largest absolute magnitude to be used as the pivot.  If that entry occurs in row r and column c, then rows r and 1 are swapped followed by a swap of column c and column 1.  After both swaps, the entry in row 1 and column 1 is the largest absolute magnitude entry in the matrix.

19 Numerical Issues Generally, for a system of n equations in n unknowns…  If that entry is nearly zero, the linear system is ill- conditioned and notify the user.  If you choose to continue, the division is performed and forward elimination begins.

20 Iterative Methods for Solving Linear Systems Look for a good numerical approximation instead of the exact mathematical solution.  Useful in sparse linear systems  Approaches Splitting Method Minimization problem

21 Iterative Methods for Solving Linear Systems Splitting Method Issues Convergence Numerical Stability

22 Iterative Methods for Solving Linear Systems Formulate the linear system Ax=b as a minimization problem

23 Matrices Square matrices Identity matrix Transpose of a matrix Symmetric matrix: A = A T Skew-symmetric: A = -A T

24 Matrices Upper echelon matrix  U = [u ij ](nxm) if u ij = 0 for i > j  If m=n, upper triangular matrix Lower echelon matrix  L = [l ij ](nxm) if l ij = 0 for i < j  If m=n, lower triangular matrix

25 Matrices Elementary Row Matrices

26 Matrices Elementary Row Matrices

27 Matrices Elementary Row Matrices  The final result of forward elimination can be stated in terms of elementary row matrices E k, … E 1 applied to the augmented matrix [A|b]. [U|v] = E k … E 1 [A|b]

28 Matrices Inverse Matrix  PA = I: P is a left inverse  A -1 A = I, AA -1 = I.  Inverses are unique  If A and B are invertible, so is AB. Its inverse is (AB) -1 = B -1 A -1

29 Matrices LU Decomposition of the matrix A  The forward elimination of a matrix A produces an upper echelon matrix U. The corresponding elementary row matrices are E k …E 1  U = E k …E 1 A., L = (E k …E 1 ) -1. L is lower triangular.  A = LU: L is lower triangular and U is upper echelon.

30 Matrices LDU Decomposition of the matrix A  L is lower triangular, D is a diagonal matrix, and U is upper echelon with diagonal entries either 1 or 0.

31 Matrices LDU Decomposition of the matrix A

32 Matrices In general the factorization can be written as PA = LDU.

33 Matrices If A is invertible, its LDU decomposition is unique If A is symmetric, U in the LDU decomposition must be U = L T.  A = LDL T.  If the diagonal entries of D are nonnegative, A = (LD 1/2 ) (LD 1/2 ) T

34 Vector Spaces The central theme of linear algebra is the study of vectors and the sets in which they live, called vector spaces. What is the vector???

35 Vector Spaces Definition of a Vector Space (the triple (V,+, ᆞ ) )

36 Q & A?