Multiple testing Justin Chumbley Laboratory for Social and Neural Systems Research University of Zurich With many thanks for slides & images to: FIL Methods group

Overview of SPM – Random field theory RealignmentSmoothing Normalisation General linear model Statistical parametric map (SPM) Image time-series Parameter estimates Design matrix Template Kernel Gaussian field theory p <0.05 Statisticalinference

t = contrast of estimated parameters variance estimate t Error at a single voxel

t = contrast of estimated parameters variance estimate t  Error at a single voxel

Decision: H 0, H 1 : zero/non-zero activation t = contrast of estimated parameters variance estimate t  h Error at a single voxel

Decision: H 0, H 1 : zero/non-zero activation t = contrast of estimated parameters variance estimate t  h  Error at a single voxel

Decision: H 0, H 1 : zero/non-zero activation t = contrast of estimated parameters variance estimate t  h Error at a single voxel

Decision: H 0, H 1 : zero/non-zero activation t = contrast of estimated parameters variance estimate t  h Decision rule (threshold) h, determines related error rates, Convention: Penalize complexity Choose h to give acceptable under H 0 Error at a single voxel

Types of error Reality H1H1 H0H0 H0H0 H1H1 True negative (TN) True positive (TP) False positive (FP)  False negative (FN) specificity: 1- = TN / (TN + FP) = proportion of actual negatives which are correctly identified sensitivity (power): 1- = TP / (TP + FN) = proportion of actual positives which are correctly identified Decision

Multiple tests t = contrast of estimated parameters variance estimate t  h  t  h   h  t  h  What is the problem?

Multiple tests t = contrast of estimated parameters variance estimate t  h  t  h   h  t  h  Penalize each independent opportunity for error.

Multiple tests t = contrast of estimated parameters variance estimate t  h  t  h   h  t  h  Convention: Choose h to limit assuming family-wise H 0

Issues 1. Voxels or regions 2. Bonferroni too harsh (insensitive) Unnecessary penalty for sampling resolution (#voxels/volume) Unnecessary penalty for independence

intrinsic smoothness –MRI signals are aquired in k-space (Fourier space); after projection on anatomical space, signals have continuous support –diffusion of vasodilatory molecules has extended spatial support extrinsic smoothness –resampling during preprocessing –matched filter theorem  deliberate additional smoothing to increase SNR –Robustness to between-subject anatomical differences

1.Apply high threshold: identify improbably high peaks 2.Apply lower threshold: identify improbably broad peaks 3.Total number of regions? Acknowledge/estimate dependence Detect effects in smooth landscape, not voxels

Null distribution? 1. Simulate null experiments 2. Model null experiments

Use continuous random field theory Discretisation (“lattice approximation”) Smoothness quantified: resolution elements (‘resels’) similar, but not identical to # independent observations computed from spatial derivatives of the residuals

Euler characteristic

General form for expected Euler characteristic  2, F, & t fields E [  h (  )] =  d R d (  )  d ( h ) Small volumes: Anatomical atlas, ‘functional localisers’, orthogonal contrasts, volume around previously reported coordinates… Unified Formula R d (  ):d-dimensional Minkowski functional of  – function of dimension, space  and smoothness: R 0 (  )=  (  ) Euler characteristic of  R 1 (  )=resel diameter R 2 (  )=resel surface area R 3 (  )=resel volume  d (  ):d-dimensional EC density of Z(x) – function of dimension and threshold, specific for RF type: E.g. Gaussian RF:  0 (h)=1-  (h)  1 (h)=(4 ln2) 1/2 exp(-h 2 /2) / (2  )  2 (h)=(4 ln2) exp(-h 2 /2) / (2  ) 3/2  3 (h)=(4 ln2) 3/2 (h 2 -1) exp(-h 2 /2) / (2  ) 2  4 (h)=(4 ln2) 2 (h 3 -3h) exp(-h 2 /2) / (2  ) 5/2 

Euler characteristic (EC) for 2D images

Spatial extent: similar

Voxel, cluster and set level tests e u h

ROI VoxelField FWEFDR * voxels/volume Height Extent ROI VoxelField Height Extent There is a multiple testing problem (‘voxel’ or ‘blob’ perspective). More corrections needed as.. Detect an effect of unknown extent & location

Further reading Friston KJ, Frith CD, Liddle PF, Frackowiak RS. Comparing functional (PET) images: the assessment of significant change. J Cereb Blood Flow Metab Jul;11(4): Genovese CR, Lazar NA, Nichols T. Thresholding of statistical maps in functional neuroimaging using the false discovery rate. Neuroimage Apr;15(4): Worsley KJ Marrett S Neelin P Vandal AC Friston KJ Evans AC. A unified statistical approach for determining significant signals in images of cerebral activation. Human Brain Mapping 1996;4:58-73.