1. Linear Algebraic Equations
Part Three:
Linear Algebraic Equations

2. Introduction to Matrices
Pages from Chapra and Canale

3. Matrix Notation and Terminology
Figure PT3.2, pg 220
aij = an individual element of the matrix [A] at row i and column j
matrix dimensions = rows by columns (n x m)
square matrix = matrix where the number of rows equals the number of columns rows (n=m)

4. Special Types of Square Matrices
symmetric matrix  aij = aji
diagonal matrix  aij = 0 where i ≠ j
identity matrix  aij = 1 where i = j AND aij = 0 where i ≠ j
for others, see pg 221

5. Matrix Addition/Subtraction
[A] = [B] if aij = bij for all i and j
[A] + [B] = aij + bij for all i and j
[A] - [B] = aij - bij for all i and j
Both addition and subtraction are commutative
[A] + [B] = [B] + [A]
Both addition and subtraction are associative
([A] + [B]) + [C] = [A] + ([B] + [C])

6. Matrix Multiplication by Scalar
Multiplying a matrix [A] by a scalar g

7. Matrix Multiplication
Multiplying two matrices ([C] = [A] [B])
where n = the column dimension of [A] and the row dimension of [B]

8. Matrix Multiplication (Cont.)
Figure PT3.3, pg 223

9. Matrix Multiplication (Cont.)
Example of matrix multiplication
(think of a diving board to help you remember)

10. Matrix Multiplication (Cont.)
Properties of matrix multiplication
Matrix multiplication is generally NOT commutative
[A] [B] ≠ [B] [A]
Matrix multiplication is associative (assuming dimensions are suitable for multiplication)
([A] [B]) [C] = [A] ([B] [C])
Matrix multiplication is distributive (again, assuming dimensions are suitable for multiplication)
[A] ([B] + [C]) = [A] [B] + [A][C]

11. Matrix Division Matrix division is not a defined operation.
HOWEVER, multiplication of a matrix by the inverse of a second matrix is analogous to matrix division.
The inverse of a matrix ( [A]-1 ) is defined as
[A] [A]-1 = [A]-1 [A] =[I]
where [I] is an identify matrix (defined on a previous slide as matrix such that aij = 1 where i = j AND aij = 0 where i ≠ j)

12. Matrix Inverse
Calculating the inverse of a matrix, the matrix must be both square and nonsingular
We defined a square matrix in a previous slide as a matrix with equal dimensions (n = m)
A singular matrix has a determinate, which we will define later, of 0.

13. Matrix Inverse (Cont.)
The inverse of a 2 by 2 matrix can be calculated using the equation below.
Techniques for calculating the inverse of higher-dimension matrices will be covered in Chapters 10 and 11.

14. Matrix Transpose The transpose of a matrix ([B]=[A]T) is defined as
for all i and j
Example if
then

15. Systems of Linear Equations
can be represented using matrix notation as
[A], [x], and [B] are defined on the next slide.

16. Matrix Operation Rules (Cont.)
representation of a system of linear algebraic equations in matrix form

17. Matrix Operation Rules (Cont.)
Given a system of linear algebraic equations such that
find [x].
Solution …
This is one way to solve a system of linear algebraic equations and is covered in more detail in Chapter 10. There are other techniques, however, that are covered in Chapter 9.

18. Chapter 9

19. Linear Algebraic Equations Part 3
An equation of the form ax+by+c=0 or equivalently ax+by=-c is called a linear equation in x and y variables.
ax+by+cz=d is a linear equation in three variables, x, y, and z.
Thus, a linear equation in n variables is
a1x1+a2x2+ … +anxn = b
A solution of such an equation consists of real numbers c1, c2, c3, … , cn. If you need to work more than one linear equations, a system of linear equations must be solved simultaneously.

20. Noncomputer Methods for Solving Systems of Equations
For small number of equations (n ≤ 3) linear equations can be solved readily by simple techniques such as “method of elimination.”
Linear algebra provides the tools to solve such systems of linear equations.
Nowadays, easy access to computers makes the solution of large sets of linear algebraic equations possible and practical.

21. Fig. pt3.5

22. Gauss Elimination Chapter 9
Solving Small Numbers of Equations
There are many ways to solve a system of linear equations:
Graphical method
Cramer’s rule
Method of elimination
Computer methods
For n ≤ 3

23. Graphical Method
For two equations:
Solve both equations for x2:

24. Plot x2 vs. x1 on rectilinear paper, the intersection of the lines present the solution.
Fig. 9.1

25. Figure 9.2

26. Determinants and Cramer’s Rule
Determinant can be illustrated for a set of three equations:
Where [A] is the coefficient matrix:

27. Assuming all matrices are square matrices, there is a number associated with each square matrix [A] called the determinant, D, of [A]. If [A] is order 1, then [A] has one element:
[A]=[a11]
D=a11
For a square matrix of order 3, the minor of an element aij is the determinant of the matrix of order 2 by deleting row i and column j of [A].

29. Cramer’s rule expresses the solution of a systems of linear equations in terms of ratios of determinants of the array of coefficients of the equations. For example, x1 would be computed as:

30. Method of Elimination
The basic strategy is to successively solve one of the equations of the set for one of the unknowns and to eliminate that variable from the remaining equations by substitution.
The elimination of unknowns can be extended to systems with more than two or three equations; however, the method becomes extremely tedious to solve by hand.

31. Naive Gauss Elimination
Extension of method of elimination to large sets of equations by developing a systematic scheme or algorithm to eliminate unknowns and to back substitute.
As in the case of the solution of two equations, the technique for n equations consists of two phases:
Forward elimination of unknowns
Back substitution

33. Pitfalls of Elimination Methods
Division by zero. It is possible that during both elimination and back-substitution phases a division by zero can occur.
Round-off errors.
Ill-conditioned systems. Systems where small changes in coefficients result in large changes in the solution. Alternatively, it happens when two or more equations are nearly identical, resulting a wide ranges of answers to approximately satisfy the equations. Since round off errors can induce small changes in the coefficients, these changes can lead to large solution errors.

34. Singular systems. When two equations are identical, we would loose one degree of freedom and be dealing with the impossible case of n-1 equations for n unknowns. For large sets of equations, it may not be obvious however. The fact that the determinant of a singular system is zero can be used and tested by computer algorithm after the elimination stage. If a zero diagonal element is created, calculation is terminated.

35. Techniques for Improving Solutions
Use of more significant figures.
Pivoting. If a pivot element is zero, normalization step leads to division by zero. The same problem may arise, when the pivot element is close to zero. Problem can be avoided:
Partial pivoting. Switching the rows so that the largest element is the pivot element.
Complete pivoting. Searching for the largest element in all rows and columns then switching.

36. Gauss-Jordan
It is a variation of Gauss elimination. The major differences are:
When an unknown is eliminated, it is eliminated from all other equations rather than just the subsequent ones.
All rows are normalized by dividing them by their pivot elements.
Elimination step results in an identity matrix.
Consequently, it is not necessary to employ back substitution to obtain solution.

37. Chapter 10

38. LU Decomposition and Matrix Inversion Chapter 10
Provides an efficient way to compute matrix inverse by separating the time consuming elimination of the Matrix [A] from manipulations of the right-hand side {B}.
Gauss elimination, in which the forward elimination comprises the bulk of the computational effort, can be implemented as an LU decomposition.

39. If L- lower triangular matrix U- upper triangular matrix Then,
[A]{X}={B} can be decomposed into two matrices [L] and [U] such that
[L][U]=[A]
[L][U]{X}={B}
Similar to first phase of Gauss elimination, consider
[U]{X}={D}
[L]{D}={B}
[L]{D}={B} is used to generate an intermediate vector {D} by forward substitution
Then, [U]{X}={D} is used to get {X} by back substitution.

40. Fig.10.1

41. LU decomposition
requires the same total FLOPS as for Gauss elimination.
Saves computing time by separating time-consuming elimination step from the manipulations of the right hand side.
Provides efficient means to compute the matrix inverse

42. The Matrix Inverse recall …
[A][x]=[B] can be rewritten as [A]-1[A ][x] = [A]-1[B], or more simply [x] = [A]-1[B]
One method for calculating a matrix inverse is using LU Decomposition.
if [B]T = [1 0 0], then solution of [A][X] = [B] gives the first column of [A]-1
if [B]T = [0 1 0], then solution of [A][X] = [B] gives the second column
if [B]T = [0 0 1], then solution of [A][X] = [B] gives the third column of [A]-1