1. EE611 Deterministic Systems Vector Spaces and Basis Changes Kevin D. Donohue Electrical and Computer Engineering University of Kentucky

2. Matrix Vector Multiplication Let x be an nx1 (column) vector and y be a 1xn (row) vector: Dot (inner) Product: yx=c = |x||y|cos(  ) where c is a scalar (1x1) and  is angle between y and x Projection: Projection of y onto x is denoted by (yx)x = |y|cos(  ) = yx/|x| Outer Product: xy=A where A is an nxn matrix. Matrix-Vector Multiplication: Let x be an nx1 vector and A be an nxn matrix: where ' denotes transpose, and vectors denote a row vector partition in the first expression and a column vector partition in the second expression.

3. Linear Independence Consider an n-dimensional real linear space  n. A set of vectors {x 1, x 2,... x m }  n are linearly dependent (l.d.) iff  a set of real numbers {      m } not identically equal to 0  Otherwise the vectors are linearly independent (l.i.) Show that if a set of vectors are l.d., then at least one of the vectors can be expressed as a linear combination of the others. The dimension of the linear space is the maximal number of l.i. vectors in the space.

4. Basis and Representation A set of l.i. vectors in  n is a basis iff every vector in  n can be expressed as a unique linear combination of these vectors. Given a basis for  n {q 1, q 2,..., q n }, then every vector in  n can be expressed as: where is called the representation of

5. Example Find the representation of noted point with orthonormal basis Q in terms of basis P.

6. Norms The generalization of magnitude or length is given by a metric referred to as a norm. Any real valued function qualifies as a norm provided it satisfies: Non-negative Scalable Consistency Triangular Inequality

7. Popular Norms Given The 1-norm is defined by The 2-norm (Euclidean norm) The infinite-norm Why do you think this is called the infinity norm? Hint: What would a 3-norm, … 100-norm look like?

8. Orthonormal Vector x is normalized, iff its Euclidean norm is 1 (self dot product is 1). Vectors x i and x j are orthogonal iff their dot product is 0. A set of (column) vectors {x 1, x 2,... x m } are orthonormal iff

9. Orthonormalization Given a set of l.i. vectors {e 1, e 2,... e n }, the Schmidt orthonormalization procedure can be used to derive an orthonormal set of vectors {q 1, q 2,... q n } forming a basis for the same linear space: Project and subtract Normalize................................................

10. Linear Algebraic Equations Consider a set of m linear equations with n unknowns: Range space of A is the set all vectors resulting from all possible linear combinations of the columns of A. The rank of coefficient matrix A ( rank(A) ) is equal to its number of l.i columns (or rows). rank(A) is also denoted as  (A) ➢ If rank(A) = n, a unique solution x exists given any y ➢ If rank(A)  m < n, many solutions x exist given any y (underdetermined) ➢ If rank(A)  n < m no solutions x may exist for some y (overdetermined)

11. Nullity The vector x is a null vector of A iff Ax=0 The null space of A is the set of all null vectors. The nullity of A is the maximum number of l.i. vectors in its null space (i.e. dimension of null space). nullity (A) = n -  (A)

12. Conditions for Solution Existence Given mxn matrix A and mx1 vector y, an nx1 solution vector x exists for y=Ax iff y is in range space of A. Given matrix A, a solution vector x exists for y=Ax,  y iff A is full row rank (  (A) = m).

13. Conditions for Unique Solution Given mxn matrix A and mx1 vector y, let x p be a solution for y=Ax. If  (A) = n (nullity k= 0), then x p is unique, and if nullity k > 0 then for any set of real  i 's, x given below is a solution. where vector set {n 1, n 2,... n k } is a basis for the null space. The above solution is also referred to as a parameterization of all solutions.

14. Singular Matrix A square matrix is singular if its determinant is 0. Given nxn non-singular matrix A, then for every y, a unique solution for y=Ax exists and is given by A -1 y=x. The homogeneous equation 0=Ax has a non zero solution iff A is singular, otherwise x=0 is the only solution.

15. Change of Basis Denote a representation of x with respect to (wrt) basis as , and representation wrt As. Note that basis vectors are assumed wrt the orthonormal basis. Find a change of basis transformation such that Show that

16. Similarity Transformation Consider nxn matrix A as a linear operator that maps  n into itself. The vector representations are wrt. Determine the new representation of the linear operator wrt basis Show that: where The operation that changes the basis of the linear operator using a pre and post multiplication of a matrix and its inverse is referred as a similarity transform.