Compiled By Raj G. Tiwari Linear Algebra Compiled By Raj G. Tiwari

Vector Operations Vector: n×1 matrix Interpretation: a point or line in n-dimensional space Dot Product, Cross Product, and Magnitude defined on vectors only y v x

Vectors: Dot Product Think of the dot product as a matrix multiplication The magnitude is the dot product of a vector with itself

Vectors: Cross Product The cross-product can be computed as a specially constructed determinant A×B A B

What is a Matrix? A matrix is a set of elements, organized into rows and columns rows columns

Basic Operations Transpose: Swap rows with columns

Just subtract elements Multiply each row by each column Basic Operations Addition, Subtraction, Multiplication Just add elements Just subtract elements Multiply each row by each column

Multiplication Is AB = BA? Maybe, but maybe not! Heads up: multiplication is NOT commutative! Exceptions AB=BA iff B = a scalar, B = identity matrix I, or B = the inverse of A, i.e., A-1

Matrix multiplication Stephen Cooke, University of Idaho Matrix multiplication Multiplication of matrices require conformability condition The conformability condition for multiplication is that the column dimensions of the lead matrix A must be equal to the row dimension of the lag matrix B. An m×n can be multiplied by an n×p matrix to yield an m×p result

Symmetric matrix A symmetric matrix is a square matrix that is equal to its transpose A=At The entries of a symmetric matrix are symmetric with respect to the main diagonal (top left to bottom right). So if the entries are written as A = (aij), then aij=aji For Example

Skew-symmetric matrix A skew-symmetric (or antisymmetric or antimetric) matrix is a square matrix A whose transpose is also its negative; that is, it satisfies the equation A = −AT. If the entry in the i th row and j th column is aij, i.e. A = (aij) then the symmetric condition becomes aij = −aji. For example, the following matrix is skew-symmetric:

Identity and Null Matrices Stephen Cooke, University of Idaho Identity and Null Matrices Identity Matrix is a square matrix and also it is a diagonal matrix with 1 along the diagonals similar to scalar “1” Null matrix is one in which all elements are zero

Determinant of a Matrix Used for inversion If det(A) = 0, then A has no inverse Can be found using factorials, pivots, and cofactors! 6.837 Linear Algebra Review

Determinant of a Matrix If M is our d × d matrix, we define Mi|j to be the (d − 1) × (d − 1) matrix obtained by deleting the ith row and the jth column of M:

Determinant of a Matrix For Matrix A For a 3×3 matrix: Sum from left to right Subtract from right to left Note: In the general case, the determinant has n! terms

example Let's expand our matrix along the first row. From the sign chart, we see that 1 is in a positive position, 3 is in a negative position and 2 is in a positive position. By putting the + or - in front of the element, it takes care of the sign adjustment when going from the minor to the cofactor. 1 ( 2 - 15 ) - 3 ( 8 - 6 ) + 2 ( 20 - 2 ) = 1 ( -13 ) - 3 ( 2 ) + 2 (18) = -13 - 6 + 36 = 17

Cofactor Det(A)= The term Mij is known as the ”minor matrix” and is the matrix you get if you eliminate row i and column j from matrix A.

Matrix of minor, Cofactor & Adjoint Minor matrix calculation Minor matrix Cofactor matrix Adjoint matrix Adjoint can be found by transposing the matrix of cofactors

Inverse of a Matrix Identity matrix: AI = A Some matrices have an inverse, such that: AA-1 = I Inversion is tricky: (ABC)-1 = C-1B-1A-1 Derived from non- commutativity property 6.837 Linear Algebra Review

Example Let A be a non-singular matrix. If there exists a square matrix B such that AB = I (identity matrix) then B is called inverse of matrix A and is denoted as A-1.  i.e AA-1 = I  Example: Matrix A Matrix B = Identity(I) 1 3 1 x 2 9 -5 = 1 0 0 1 1 2 0 -2 1 0 1 0 2 3 4 -1 -3 2 0 0 1

Stephen Cooke, University of Idaho Matrix inversion It is not possible to divide one matrix by another. That is, we can not write A/B. This is because for two matrices A and B, the quotient can be written as AB-1

Requirements to have an Inverse The matrix must be square (same number of rows and columns). The determinant of the matrix must not be zero (determinants are covered in section 6.4). This is instead of the real number not being zero to have an inverse, the determinant must not be zero to have an inverse. A square matrix that has an inverse is called invertible or non-singular. A matrix that does not have an inverse is called singular. A matrix does not have to have an inverse, but if it does, the inverse is unique.

Trace The trace of a d × d (square) matrix, denoted tr[M], is the sum of its diagonal elements:

Inverse of previous example

Eigenvalues and Eigenvectors Let A be a square matrix. A non-zero vector X is called an eigenvector of A if and only if there exists a number (real or complex)  such that AX= λ X If such a number  exists, it is called an eigenvalue of A. The vector C is called eigenvector associated to the eigenvalue . 

Eigenvalues and Eigenvectors Remark. The eigenvector C must be non-zero since we have  for any number  .  Rewriting (A- λI)X=0

Computation of Eigenvalues In linear algebra, the characteristic equation (or secular equation) of a square matrix A is the equation in one variable λ where det is the determinant and I is the identity matrix. The solutions of the characteristic equation are precisely the eigenvalues of the matrix A

Example

Computation of Eigenvector Set corresponding to an eigenvalue λ, we simply solve the system of linear equations given by (A- λI)X=0

Example Applying characteristic equation If we develop this determinant using the third column, we obtain Using easy algebraic manipulations, we get  which implies that the eigenvalues of A are 0, -4, and 3. 

Case  Rewritten as By Solving where c is an arbitrary number

THE DERIVATIVES OF VECTOR FUNCTIONS Let x and y be vectors of orders n and m respectively: where each component yi may be a function of all the xj , a fact represented by saying that y is a function of x, or y = y(x).

Derivative of a Scalar with Respect to Vector Derivative of Vector with Respect to Scalar

Jacobian matrix If we have an m-dimensional vector-valued function of a n-dimensional vector x, we calculate the derivatives and represent them as the Jacobian Jacobian matrix This matrix is also denoted by   and  . The Jacobian determinant (often simply called the Jacobian) is the determinant of the Jacobian matrix.

Example