1 Neural Nets Applications Vectors and Matrices

2/27 Outline 1. Definition of Vectors 2. Operations on Vectors 3. Linear Dependence of Vectors 4. Definition of Matrices 5. Operations on Matrices 6. Multiplication of Matrices and Vectors 7. Symmetric Matrices and their Properties 8. Norms, Products, and Orthogonlity of Vectors 9. Linear Systems, Inverse and Pseudo- inverse Matrices

3/27 Definition of Vectors(1/2)  Consider n real numbers (scalars) x 1,x 2, …, x n.  These number can be arranged so as to define a new object X, called a column vector:

4/27 Definition of Vectors(2/2)  Transpose Vector of X (row vector):  n-component,  n-tuple,  n-dimensional vector

5/27 Operations on Vectors(1/2)  Consider the column vectors i.e., adding two component vectors requires adding their respective components.

6/27 Operations on Vectors(2/2)  The product of a column vector X by a scalar c is defined as

7/27 Linear Dependence of Vectors(1/2)  Consider a set of n-dimensional vectors X 1, X 2, …, X m used to construct a vector X such that

8/27 Definition of Matrices(1/2)  A rectangular array A of n  m numbers a ij, where i=1,2,…,n, and j=1,2,…,m, arranged in n rows and m columns is called an nxm matrix

9/27 Definition of Matrices(2/2)  transpose matrix of A is denoted by A t and is defined as

10/27 Operations on Matrices(1/3)  Consider two n  m matrices, A and B with their respective elements a ij and b ij, for i=1,2,…,n and j=1,2,…,m. The sum of matrices A and B is the n  m matrix

11/27 Operations on Matrices(2/3)  The product of an matrix A by a scalar c is defined as an n  m matrix

12/27 Operations on Matrices(3/3)  The product AB of an n  m matrix A and an m  q matrix B is the n  q matrix C for i=1,2,…,n, and k=1,2,…,q. In matrix form

13/27 Multiplication of Matrices and Vectors (1/2)  An m-tuple vector X can be interpreted as a matrix with m rows and one column.  Postmultiplying an n  m A with a vector X In an expanded form

14/27 Multiplication of Matrices and Vectors (2/2)  or

15/27 Symmetric Matrices and their Properties (1/4)  square matrix an n  n matrix with same number of rows and columns  Symmetric matrix

16/27 Symmetric Matrices and their Properties (2/4)  A square matrix A whose elements not on the main diagonal are zero is called a diagonal matrix.

17/27 Symmetric Matrices and their Properties (3/4)  Unity or identity matrix I

18/27 Symmetric Matrices and their Properties (4/4)  determinant of a square matrix is a scalar and is denoted by

19/27 Norms, Products, and Orthogonality of Vectors(1/5)  The Euclidean norm, or length, of an n-tuple vector X is denoted and defined as or

20/27 Norms, Products, and Orthogonality of Vectors(2/5)  The scalar (inner, dot) product of two n-component vectors X and Y is the scalar defined as

21/27 Norms, Products, and Orthogonality of Vectors(3/5) or

22/27 Norms, Products, and Orthogonality of Vectors(4/5)  Vectors X and Y are said to be orthogonal if and only if their scalar product is zero.  If the nonzero vectors X 1, X 2, …, X n, are mutually orthogonal (every vector is orthogonal to each other), then they are linearly independent.

23/27 Norms, Products, and Orthogonality of Vectors(5/5)  The outer product of two n-component vectors X and Y is an n  n matrix as shown below

24/27 Linear Systems, Inverse, and Pseudoinverse Matrices (1/4)  The general form of a linear system of n equations with m unknowns x 1, x 2, …, x m.  In a case where m=n and det A  0, The exact solution of previous equation is where A -1 is the inverse matrix

25/27 Linear Systems, Inverse, and Pseudoinverse Matrices (2/4)  In a case where m<n, it does not have an accurate solution for X.  The approximate solution, which minimize the error.

26/27 Linear Systems, Inverse, and Pseudoinverse Matrices (3/4)  In a case where m>n, it has infinite number of solutions.  The approximate solution, which has the smallest norm, i.e., for each X such that AX=Y. This solution is  A + is called the pseudoinverse of matrix A

27/27 Linear Systems, Inverse, and Pseudoinverse Matrices (4/4)  when m < n,  when m > n,