Linear algebra: matrix Eigen-value Problems Part 3 Eng. Hassan S. Migdadi

Eigenvalues and eigenvectors Vector spaces Linear transformations Eigenvalue Problems Eigenvalues and eigenvectors Vector spaces Linear transformations Matrix diagonalization

The Eigenvalue Problem Consider a nxn matrix A Vector equation: Ax = lx Seek solutions for x and l l satisfying the equation are the eigenvalues Eigenvalues can be real and/or imaginary; distinct and/or repeated x satisfying the equation are the eigenvectors Nomenclature The set of all eigenvalues is called the spectrum Absolute value of an eigenvalue: The largest of the absolute values of the eigenvalues is called the spectral radius

Determining Eigenvalues Vector equation Ax = lx  (A-lI)x = 0 A-lI is called the characteristic matrix Non-trivial solutions exist if and only if: This is called the characteristic equation Characteristic polynomial nth-order polynomial in l Roots are the eigenvalues {l1, l2, …, ln}

Eigenvalue Example Characteristic matrix Characteristic equation Eigenvalues: l1 = -5, l2 = 2

Eigenvalue Properties Eigenvalues of A and AT are equal Singular matrix has at least one zero eigenvalue Eigenvalues of A-1: 1/l1, 1/l2, …, 1/ln Eigenvalues of diagonal and triangular matrices are equal to the diagonal elements Trace Determinant

Determining Eigenvectors First determine eigenvalues: {l1, l2, …, ln} Then determine eigenvector corresponding to each eigenvalue: Eigenvectors determined up to scalar multiple Distinct eigenvalues Produce linearly independent eigenvectors Repeated eigenvalues Produce linearly dependent eigenvectors Procedure to determine eigenvectors more complex (see text) Will demonstrate in Matlab

Eigenvector Example Eigenvalues Determine eigenvectors: Ax = lx Eigenvector for l1 = -5 Eigenvector for l1 = 2

Matlab Examples >> A=[ 1 2; 3 -4]; >> e=eig(A) e = 2 -5 >> [X,e] = eig(A) X = 0.8944 -0.3162 0.4472 0.9487 2 0 0 -5 >> A=[2 5; 0 2]; >> e=eig(A) e = 2 >> [X,e]=eig(A) X = 1.0000 -1.0000 0 0.0000 2 0 0 2

Vector Spaces Real vector space V Properties of a real vector space Set of all n-dimensional vectors with real elements Often denoted Rn Element of real vector space denoted Properties of a real vector space Vector addition Scalar multiplication

Vector Spaces cont. Linearly independent vectors Basis Elements: Linear combination: Equation satisfied only for cj = 0 Basis n-dimensional vector space V contains exactly n linearly independent vectors Any n linearly independent vectors form a basis for V Any element of V can be expressed as a linear combination of the basis vectors Example: unit basis vectors in R3

Inner Product Spaces Inner product Properties of an inner product space Two vectors with zero inner product are called orthogonal Relationship to vector norm Euclidean norm General norm Unit vector: ||a|| = 1

Linear Transformation Properties of a linear operator F Linear operator example: multiplication by a matrix Nonlinear operator example: Euclidean norm Linear transformation Invertible transformation Often called a coordinate transformation

Orthogonal Transformations Orthogonal matrix A square matrix satisfying: AT = A-1 Determinant has value +1 or -1 Eigenvalues are real or complex conjugate pairs with absolute value of unity A square matrix is orthonormal if: Orthogonal transformation y = Ax where A is an orthogonal matrix Preserves the inner product between any two vectors The norm is also invariant to orthogonal transformation

Similarity Transformations Eigenbasis If a nxn matrix has n distinct eigenvalues, the eigenvectors form a basis for Rn The eigenvectors of a symmetric matrix form an orthonormal basis for Rn If a nxn matrix has repeated eigenvalues, the eigenvectors may not form a basis for Rn (see text) Similar matrices Two nxn matrices are similar if there exists a nonsingular nxn matrix P such that: Similar matrices have the same eigenvalues If x is an eigenvector of A, then y = P-1x is an eigenvector of the similar matrix

Matrix Diagonalization Assume the nxn matrix A has an eigenbasis Form the nxn modal matrix X with the eigenvectors of A as column vectors: X = [x1, x2, …, xn] Then the similar matrix D = X-1AX is diagonal with the eigenvalues of A as the diagonal elements Companion relation: XDX-1 = A

Matrix Diagonalization Example

Matlab Example >> A=[-1 2 3; 4 -5 6; 7 8 -9]; >> [X,e]=eig(A) X = -0.5250 -0.6019 -0.1182 -0.5918 0.7045 -0.4929 -0.6116 0.3760 0.8620 e = 4.7494 0 0 0 -5.2152 0 0 0 -14.5343 >> D=inv(X)*A*X D = 4.7494 -0.0000 -0.0000 -0.0000 -5.2152 -0.0000 0.0000 -0.0000 -14.5343