REGRESSION (CONTINUED) Matrices & Matrix Algebra; Multivariate Regression LECTURE 5 Supplementary Readings: Wilks, chapters 6; Bevington, P.R., Robinson, D.K., Data Reduction and Error Analysis for the Physical Sciences, McGraw-Hill, 1992.

Tutorial on Matrices and Matrix Algebra

VECTORS c is an N-length column vector b is an M-length row vector b T is an N-length column vector

VECTORS Can add two N-length row vectors or two N-length column vectors

VECTORS Can subtract two N-length row vectors or two N-length column vectors

VECTORS Can multiply an N-length vector by a constant

VECTORS Can multiply an K-length row vector by an K-length column vector ‘DOT PRODUCT’ or ‘INNER PRODUCT’ ‘EUCLIDEAN NORM’

VECTORS ‘DOT PRODUCT’ or ‘INNER PRODUCT’ ‘EUCLIDEAN NORM’ Note the close relationship with the linear correlation between two series

VECTORS Can multiply an N-length column vector by an M-length row vector ‘OUTER PRODUCT’

VECTORS Yields an NxM Matrix ‘OUTER PRODUCT’

MATRICES NxM Matrix

TRANSPOSE OF MATRIX NxM Matrix MxN Matrix

Can add two NxM Matrices MATRICES

Can multiply an NxK and KxM Matrix MATRICES

RULES FOR MATRIX ARITHMETIC Associative Laws Commutative Laws

NxN Matrix N IDENTITY MATRIX

DIAGONAL MATRICES

INVERSE OF A (SQUARE) MATRIX

INVERSE OF A MATRIX

Special Cases: 2x2 matrix If det(A)  0 then the matrix is “Invertible” Equivalent to the Matrix being of “full rank” (ie, there are no redundant rows in the matrix)

INVERSE OF A MATRIX Special Cases: diagonal matrix

UNITARY MATRIX Example: 2D Rotation Matrix Note that the inverse represents a rotation in the opposite direction

SOLUTION OF MATRIX EQUATION If A is invertible, We can write

Recall Linear Regression We can write this as a matrix equation,