Matrix Algebra (and why it’s important!) Methods for Dummies FIL October 2007 Steve Fleming & Verity Leeson Need to add: division, and why its important. Graphical model of determinant. Animate slide 4.

Sources and further information Jon Machtynger & Jen Marchant’s slides! Human Brain Function textbook (for GLM) SPM course http://www.fil.ion.ucl.ac.uk/spm/course/ Web Guides http://mathworld.wolfram.com/LinearAlgebra.html http://www.maths.surrey.ac.uk/explore/emmaspages/option1.html http://www.inf.ed.ac.uk/teaching/courses/fmcs1/ (Formal Modelling in Cognitive Science course) http://www.wikipedia.org

Scalars, vectors and matrices Scalar: Variable described by a single number – e.g. Image intensity (pixel value) Vector: Variable described by magnitude and direction – e.g. Image intensity at a particular time Linear algebra had its beginnings in the study of vectors in Cartesian 2-space and 3-space. A vector, here, is a directed line segment, characterized by both its magnitude, represented by length, and its direction. Vectors can be used to represent physical entities such as forces, and they can be added to each other and multiplied with scalars, thus forming the first example of a real vector space. Modern linear algebra has been extended to consider spaces of arbitrary or infinite dimension. Four dimensional vector is image at time point in 3d space Matrix: Rectangular array of vectors defined by number of rows and columns Square (3 x 3) Rectangular (3 x 2) d r c : rth row, cth column 3 2 (Roman Catholic)

Matrices in Matlab Vector formation: [1 2 3] Matrix formation: ‘;’ is used to signal end of a row ‘:’ is used to signify all rows or columns Subscripting – each element of a matrix can be addressed with a pair of numbers; row first, column second (Roman Catholic) e.g. X(2,3) = 6 X(3, :) = X( [2 3], 2) = “Special” matrix commands: zeros(3,1) = ones(2) = magic(3) more to come… Magic is where rows, columns and diagonals all sum to same total. Has to be square.

Matrix addition Addition (matrix of same size) Commutative: A+B=B+A Associative: (A+B)+C=A+(B+C) Subtraction (consider as the addition of a negative matrix)

Matrix multiplication Scalar multiplication: Rule for multiplication of vectors/matrices: n l a11 a12 a13 b11 b12 a21 a22 a23 X b21 b22 a31 a32 a33 b31 b32 a41 a42 a43 m k Matrix multiplication rule: “When A is a mxn matrix & B is a kxl matrix, AB is only viable if n=k. The result will be an mxl matrix” b11 b12 a11 a12 a13 b21 b22 x a21 a22 a23 b31 b32 a31 a32 a33 a41 a42 a43

Multiplication method Sum over product of respective rows and columns For larger matrices, following method might be helpful: m m Define output matrix X = r c Sum over crc = Matlab does all this for you! Simply type: C = A * B N.B. If you want to do element-wise multiplication, use: A .* B =

Transposition column → row row → column Mrc = Mcr In Matlab: AT = A’

Outer and inner products of vectors Two vectors: Inner product = scalar (1xn)(nx1)  (1X1) Outer product = matrix (nx1)(1xn)  (nXn)

Identity matrices Is there a matrix which plays a similar role as the number 1 in number multiplication? Consider the nxn matrix: A square nxn matrix A has one A In = In A = A An nxm matrix A has two!! In A = A & A Im = A Worked example A In = A for a 3x3 matrix: 1 2 3 1+0+0 0+2+0 0+0+3 4 5 6 X = 4+0+0 0+5+0 0+0+6 7 8 9 7+0+0 0+8+0 0+0+9 In Matlab: eye(r, c) produces an r x c identity matrix

Inverse matrices Definition. A matrix A is nonsingular or invertible if there exists a matrix B such that: worked example: 1 X 2 3 -1 3 = 2 + 1 3 3 -1 + 1 3 3 -1 2 1 3 -2+ 2 3 3 1 + 2 3 3 Common notation for the inverse of a matrix A is A-1 The inverse matrix A-1 is unique when it exists. If A is invertible, A-1 is also invertible  A is the inverse matrix of A-1. Matrix division: A/B = AB-1 If A is an invertible matrix, then (AT)-1 = (A-1)T In Matlab: A-1 = inv(A)

Determinants Determinant is a function: In Matlab: det(A) = det(A) Input is nxn matrix Output is a single number (real or complex) called the determinant A matrix A has an inverse matrix A-1 if and only if det(A)≠0 (see next slide) In Matlab: det(A) = det(A)

Calculation of inversion using determinants Or you can just type inv(A)! thus Note: det(A)≠0 More complex matrices can be inverted using methods such as the Gauss-Jordan elimination, Gaussian elimination or LU decomposition

Applications

SEM http://www.maths.soton.ac.uk/~jav/soton/MATH1007/workbook_8/8_2_inv_mtrx_sim_lin_eqnpdf.pdf Neural Networks http://csn.beckman.uiuc.edu/k12/nn_matrix.pdf SPM http://imaging.mrc-cbu.cam.ac.uk/imaging/PrinciplesStatistics

Solving simultaneous equations For one linear equation ax=b where the unknown is x and a and b are constants 3 possibilities

With >1 equation and >1 unknown Can use solution from the single equation to solve For example In matrix form AX = B

Need to find determinant of matrix A (because X =A-1B) From earlier (2 x -2) – (3 x 1) = -4 – 3 = -7 So determinant is -7 To find A-1

if B is So

Neural Networks Neural networks create a mathematical model of the connections in a neural system Connections are the excitatory and inhibitory synapses between neurons Excitatory Connection Inhibitory Connection Input Neuron Output Neuron

Scenario 1 Input Neuron Output Neuron then If then If

Scenario 2 + = The combination of both an active excitatory and active inhibitory input will cancel out No net activity

Matrix Representations of Neural Connections –Scenario 2 again #1 #3 #2 + = -1 +1 1 Excitatory = positive influence on post synaptic cell Inhibitory = negative influence With the synapses labelled (1-3) and activity level specified we can translate this information into a set of vectors (1 row matrices)

Input vector = (1 1) relates to activity (#1 #2) #3 #2 + = -1 +1 1 Input vector = (1 1) relates to activity (#1 #2) Weight vector = (1 -1) relates to connection weight (#1 #2) Activity of Neuron 3 Input x weight With varying input (activity) and weight, neuron 3 can take on a wide range of values

How are matrices relevant to fMRI data? Consider that data measured includes Response variable e.g BOLD signal at a particular voxel Many scalars for this one voxel Explanatory variables These are assumed to be measured without error May be continuous May be dummy indicating levels of an experimental factor

With a single explanatory variable Y = X . β + ε Observed = Predictors * Parameters + Error BOLD = Design Matrix * Betas + Error

Y = X . β + ε Y is a matrix of BOLD signals Preprocessing ... Y Y is a matrix of BOLD signals Each column represents a single voxel sampled at successive time points. Each voxel is considered as independent observation Analysis of individual voxels over time, not groups over space Time Intensity

Design Matrix Matrix Rows : values of X for a single predictor Y = X . β + ε Y X1 X2 X1 X2 Most –ve nearest black, most +ve nearest white Matrix Rows : values of X for a single predictor Columns : different predictors

A complex version = + Y = X × b + e data vector (Voxel) design matrix Solve equation for β – tells us how much of the BOLD signal is explained by X data vector (Voxel) design matrix parameters error vector a m b3 b4 b5 b6 b7 b8 b9 = + Y = X × b + e

The End… Any (easy) questions?!