1. Compiled By Raj G. Tiwari
Linear Algebra
Compiled By
Raj G. Tiwari

2. Vector Operations Vector: n×1 matrix
Interpretation: a point or line in n-dimensional space
Dot Product, Cross Product, and Magnitude defined on vectors only y v x

3. Vectors: Dot Product
Think of the dot product as a matrix multiplication
The magnitude is the dot product of a vector with itself

4. Vectors: Cross Product
The cross-product can be computed as a specially constructed determinant
A×B A B

5. What is a Matrix?
A matrix is a set of elements, organized into rows and columns
rows
columns

6. Basic Operations
Transpose: Swap rows with columns

7. Just subtract elements Multiply each row by each column
Basic Operations
Addition, Subtraction, Multiplication
Just add elements
Just subtract elements
Multiply each row by each column

8. Multiplication Is AB = BA? Maybe, but maybe not!
Heads up: multiplication is NOT commutative!
Exceptions
AB=BA iff
B = a scalar,
B = identity matrix I, or
B = the inverse of A, i.e., A-1

9. Matrix multiplication
Stephen Cooke, University of Idaho
Matrix multiplication
Multiplication of matrices require conformability condition
The conformability condition for multiplication is that the column dimensions of the lead matrix A must be equal to the row dimension of the lag matrix B.
An m×n can be multiplied by an n×p matrix to yield an m×p result

10. Symmetric matrix
A symmetric matrix is a square matrix that is equal to its transpose
A=At
The entries of a symmetric matrix are symmetric with respect to the main diagonal (top left to bottom right). So if the entries are written as A = (aij), then
aij=aji
For Example

11. Skew-symmetric matrix
A skew-symmetric (or antisymmetric or antimetric) matrix is a square matrix A whose transpose is also its negative; that is, it satisfies the equation A = −AT. If the entry in the i th row and j th column is aij, i.e. A = (aij) then the symmetric condition becomes aij = −aji. For example, the following matrix is skew-symmetric:

12. Identity and Null Matrices
Stephen Cooke, University of Idaho
Identity and Null Matrices
Identity Matrix is a square matrix and also it is a diagonal matrix with 1 along the diagonals similar to scalar “1”
Null matrix is one in which all elements are zero

13. Determinant of a Matrix
Used for inversion
If det(A) = 0, then A has no inverse
Can be found using factorials, pivots, and cofactors!
6.837 Linear Algebra Review

14. Determinant of a Matrix
If M is our d × d matrix, we define Mi|j to be the (d − 1) × (d − 1) matrix obtained by deleting the ith row and the jth column of M:

15. Determinant of a Matrix
For Matrix A
For a 3×3 matrix:
Sum from left to right
Subtract from right to left
Note: In the general case, the determinant has n! terms

16. example Let's expand our matrix along the first row.
From the sign chart, we see that 1 is in a positive position, 3 is in a negative position and 2 is in a positive position. By putting the + or - in front of the element, it takes care of the sign adjustment when going from the minor to the cofactor.
1 ( ) - 3 ( ) + 2 ( ) = 1 ( -13 ) - 3 ( 2 ) + 2 (18) = = 17

17. Cofactor
Det(A)=
The term Mij is known as the ”minor matrix” and is the matrix you get if you eliminate row i and column j from matrix A.

18. Matrix of minor, Cofactor & Adjoint
Minor matrix calculation
Minor matrix
Cofactor matrix
Adjoint matrix
Adjoint can be found by transposing the matrix of cofactors

19. Inverse of a Matrix Identity matrix: AI = A
Some matrices have an inverse, such that: AA-1 = I
Inversion is tricky: (ABC)-1 = C-1B-1A-1
Derived from non- commutativity property
6.837 Linear Algebra Review

20. Example
Let A be a non-singular matrix. If there exists a square matrix B such that AB = I (identity matrix) then B is called inverse of matrix A and is denoted as A-1.  i.e AA-1 = I  Example:
Matrix A Matrix B = Identity(I)
x =

21. Stephen Cooke, University of Idaho
Matrix inversion
It is not possible to divide one matrix by another. That is, we can not write A/B. This is because for two matrices A and B, the quotient can be written as AB-1

22. Requirements to have an Inverse
The matrix must be square (same number of rows and columns).
The determinant of the matrix must not be zero (determinants are covered in section 6.4). This is instead of the real number not being zero to have an inverse, the determinant must not be zero to have an inverse.
A square matrix that has an inverse is called invertible or non-singular. A matrix that does not have an inverse is called singular.
A matrix does not have to have an inverse, but if it does, the inverse is unique.

23. Trace
The trace of a d × d (square) matrix, denoted tr[M], is the sum of its diagonal elements:

24. Inverse of previous example

25. Eigenvalues and Eigenvectors
Let A be a square matrix. A non-zero vector X is called an eigenvector of A if and only if there exists a number (real or complex)  such that
AX= λ X
If such a number  exists, it is called an eigenvalue of A. The vector C is called eigenvector associated to the eigenvalue . 

26. Eigenvalues and Eigenvectors
Remark. The eigenvector C must be non-zero since we have 
for any number  . 
Rewriting
(A- λI)X=0

27. Computation of Eigenvalues
In linear algebra, the characteristic equation (or secular equation) of a square matrix A is the equation in one variable λ
where det is the determinant and I is the identity matrix. The solutions of the characteristic equation are precisely the eigenvalues of the matrix A

28. Example

29. Computation of Eigenvector
Set
corresponding to an eigenvalue λ, we simply solve the system of linear equations given by
(A- λI)X=0

30. Example Applying characteristic equation
If we develop this determinant using the third column, we obtain
Using easy algebraic manipulations, we get 
which implies that the eigenvalues of A are 0, -4, and 3. 

31. Case 
Rewritten as
By Solving
where c is an arbitrary number

32. THE DERIVATIVES OF VECTOR FUNCTIONS
Let x and y be vectors of orders n and m respectively:
where each component yi may be a function of all the xj , a fact represented by saying that y is a function of x, or
y = y(x).

33. Derivative of a Scalar with Respect to Vector
Derivative of Vector with Respect to Scalar

34. Jacobian matrix
If we have an m-dimensional vector-valued function of a n-dimensional vector x, we calculate the derivatives and represent them as the Jacobian Jacobian matrix
This matrix is also denoted by   and 
The Jacobian determinant (often simply called the Jacobian) is the determinant of the Jacobian matrix.

35. Example