Special Topic: Matrix Algebra and the ANOVA Matrix properties Types of matrices Matrix operations Matrix algebra in Excel Regression using matrices ANOVA in matrix notation

Definition of a Matrix a matrix is a rectangular array of elements Matrix order (dimensions or size) –m=#rows, n=#columns  m x n a 13 = 6 matrix element order is 2 x 3

Types of Matrices a single value is called a ‘scalar’ B = 6 a single row is called a ‘row vector’ a single column is called a ‘column vector’

Types of Matrices A square matrix has equal numbers of rows and columns In a symmetric matrix, a ij = a ji a 31 = a 13 = 4 In a diagonal matrix, all off-diagonal elements = 0 An identity matrix is a diagonal matrix with diagonals = 1

Common Variance, Independence e ij are independent, with common variance Off-diagonal elements are zero, showing that there is no covariance (there is independence)

Trace The trace of a matrix is the sum of the elements on the main diagonal (  a ii ) tr(A) = = 20

Matrix Addition and Subtraction Add or subtract corresponding elements of each matrix The order (dimensions) of the matrices must be the same

Matrix Multiplication A r x n B n x c M r x c m 11 = 2*4 + 5*1 + 1*9 + 8*5 = 62 Take the sum of crossproducts of rows from the first matrix with columns from the second matrix The number of columns in the first matrix must be the same as the number of rows in the second matrix

Transpose of a Matrix To transpose a matrix, exchange rows and columns A prime () or a ( T ) is used to denote a transpose Note that AA gives the uncorrected sum of squares and crossproducts for the columns of A Sum of squares on the diagonal Sum of crossproducts a 21  a 12 = 5

Inverse of a Matrix Taking the inverse of a matrix is analagous to division in math It’s easy for diagonal matrices Use (-1) as an exponent to denote an inverse

Inverting a 2x2 Matrix D = ad - bcD = 2*9 – 5*3 = 3 Inverting a 2 x 2 matrix is not too hard Find the determinant (D), often written as |M| For larger matrices, use a computer!

Linear Dependence D = ad - bc D = 2*9 – 6*3 = 0 The matrix is singular because one column can be obtained by multiplying another by a constant (3 in this case) |M| = 0 The rank of a matrix = the number of linearly independent columns (1 in this case) A nonsingular matrix is full rank – the rank equals the total number of columns

Properties of Full Rank Matrices A square, nonsingular matrix has a unique inverse For a full rank matrix A -1 A = AA -1 = I If A -1 exists, then (A -1 ) -1 = A If A -1 exists, and B -1 exists then (AB) -1 = B -1 A -1 Using Excel: MDETERM(G3:I5) = 54 The determinant ≠ 0, so there is a unique inverse

Idempotent Matrices A matrix is idempotent if it can be multiplied by itself and the result is the original matrix AA = A Idempotent matrices must be square The trace of an idempotent matrix is equal to its rank

Generalized Inverse A generalized inverse (M – ) can be obtained for any matrix, but the solution is not unique MM – M = M

Matrix Algebra in Excel A little cumbersome, but may be handy for a limited number of calculations Addition and subtraction are the same as always –Use the usual shortcuts: fill down, fill right, copy, paste To transpose a matrix, there are two options: 1.Copy the original matrix, select a single destination cell, use “paste special” and select the option “Transpose” 2.Use the matrix function TRANSPOSE

Matrix Functions in Excel Examples: MMULT, TRANSPOSE, MINVERSE Steps for Matrix operations (on a PC) –Select destination cells (must be the right dimensions) –Enter the matrix formula –Press F2 –Press Ctrl-Shift-Enter

Regression in Matrix Notation Y = X  + ε b = (XX) -1 XY Linear model Parameter estimates SourcedfSSMS Regression ( uncorrected ) p bXYbXY MSR ResidualN-p YY – bXY MSE Total ( uncorrected )NY p = number of parameters estimated in the model N = total number of observations Correction for mean

Regression example Fit a quadratic curve Y = X  + ε Linear model in matrix notation XY Y i = b 0 + b 1 X i + b 2 X i 2 Solution: b = (XX) -1 XY Using Excel, SAS or R Y i = X i -0.5X i 2

Working with Matrices in R Create a matrix Add matrices C <- A + B Multiply matrices D <- A %*% B Transpose a matrix Xp <- t(X) Invert a matrix XpXinv <- solve(XpX)

ANOVA example TRTY CRD with 3 treatment levels, 2 reps Y = X  + ε Linear model   1  2  3 Reparameterize to make a nonsingular matrix (rank = 3) Let  1 = 0

Recalculating the parameters of interest R uses the first treatment as the standard The result is the same as setting  1 =0 R Code for a one-way ANOVA ONE.crd<-lm(Y ~ TRT, data=ONE) summary(ONE.crd)