Inference on SPMs: Random Field Theory & Alternatives

Inference on SPMs: Random Field Theory & Alternatives
Thomas Nichols, Ph.D. Director, Modelling & Genetics GlaxoSmithKline Clinical Imaging Centre Zurich SPM Course Feb 12, 2009

normalisation image data parameter estimates design matrix kernel
Thresholding & Random Field Theory realignment & motion correction General Linear Model model fitting statistic image smoothing normalisation Statistical Parametric Map anatomical reference Corrected thresholds & p-values

Outline Orientation Assessing Statistic images
The Multiple Testing Problem Random Field Theory & FWE Permutation & FWE False Discovery Rate

Assessing Statistic Images
Where’s the signal? High Threshold Med. Threshold Low Threshold t > 5.5 t > 3.5 t > 0.5 Good Specificity Poor Power (risk of false negatives) Poor Specificity (risk of false positives) Good Power ...but why threshold?!

Blue-sky inference: What we’d like
Don’t threshold, model the signal! Signal location? Estimates and CI’s on (x,y,z) location Signal magnitude? CI’s on % change Spatial extent? Estimates and CI’s on activation volume Robust to choice of cluster definition ...but this requires an explicit spatial model space

Blue-sky inference: What we need
Need an explicit spatial model No routine spatial modeling methods exist High-dimensional mixture modeling problem Activations don’t look like Gaussian blobs Need realistic shapes, sparse Some initial work Hartvig et al., Penny et al., Xu et al. Not part of mass-univariate framework

Real-life inference: What we get
Signal location Local maximum – no inference Center-of-mass – no inference Sensitive to blob-defining-threshold Signal magnitude Local maximum intensity – P-values (& CI’s) Spatial extent Cluster volume – P-value, no CI’s

Voxel-level Inference
Retain voxels above -level threshold u Gives best spatial specificity The null hyp. at a single voxel can be rejected u space Significant Voxels No significant Voxels

Cluster-level Inference
Two step-process Define clusters by arbitrary threshold uclus Retain clusters larger than -level threshold k uclus space Cluster not significant Cluster significant k k

Cluster-level Inference
Typically better sensitivity Worse spatial specificity The null hyp. of entire cluster is rejected Only means that one or more of voxels in cluster active uclus space Cluster not significant Cluster significant k k

Here c = 1; only 1 cluster larger than k
Set-level Inference Count number of blobs c Minimum blob size k Worst spatial specificity Only can reject global null hypothesis uclus space k k Here c = 1; only 1 cluster larger than k

Hypothesis Testing Null Hypothesis H0 Test statistic T  level P-value
t observed realization of T  level Acceptable false positive rate Level  = P( T>u | H0 ) Threshold u controls false positive rate at level  P-value Assessment of t assuming H0 P( T > t | H0 ) Prob. of obtaining stat. as large or larger in a new experiment P(Data|Null) not P(Null|Data) u  Null Distribution of T t P-val Null Distribution of T

Multiple Testing Problem
Which of 100,000 voxels are sig.? =0.05  5,000 false positive voxels Which of (random number, say) 100 clusters significant? =0.05  5 false positives clusters t > 0.5 t > 1.5 t > 2.5 t > 3.5 t > 4.5 t > 5.5 t > 6.5

MTP Solutions: Measuring False Positives
Familywise Error Rate (FWE) Familywise Error Existence of one or more false positives FWE is probability of familywise error False Discovery Rate (FDR) FDR = E(V/R) R voxels declared active, V falsely so Realized false discovery rate: V/R

FWE MTP Solutions: Bonferroni
For a statistic image T... Ti ith voxel of statistic image T ...use  = 0/V 0 FWE level (e.g. 0.05) V number of voxels u -level statistic threshold, P(Ti  u) =  Conservative under correlation Independent: V tests Some dep.: ? tests Total dep.: 1 test

SPM approach: Random fields…
Consider statistic image as lattice representation of a continuous random field Use results from continuous random field theory  lattice represtntation

FWE MTP Solutions: Controlling FWE w/ Max
FWE & distribution of maximum FWE = P(FWE) = P( i {Ti  u} | Ho) = P( maxi Ti  u | Ho) 100(1-)%ile of max distn controls FWE FWE = P( maxi Ti  u | Ho) =  where u = F-1max (1-) .  u

FWE MTP Solutions: Random Field Theory
Euler Characteristic u Topological Measure #blobs - #holes At high thresholds, just counts blobs FWE = P(Max voxel  u | Ho) = P(One or more blobs | Ho)  P(u  1 | Ho)  E(u | Ho) Threshold Random Field No holes Never more than 1 blob Suprathreshold Sets

RFT Details: Expected Euler Characteristic
E(u)  () ||1/2 (u 2 -1) exp(-u 2/2) / (2)2   Search region   R3 (  volume ||1/2  roughness Assumptions Multivariate Normal Stationary* ACF twice differentiable at 0 Stationary Results valid w/out stationary More accurate when stat. holds Only very upper tail approximates 1-Fmax(u)

Random Field Theory Smoothness Parameterization
E(u) depends on ||1/2  roughness matrix: Smoothness parameterized as Full Width at Half Maximum FWHM of Gaussian kernel needed to smooth a white noise random field to roughness  Autocorrelation Function FWHM

Random Field Theory Smoothness Parameterization
RESELS Resolution Elements 1 RESEL = FWHMx FWHMy FWHMz RESEL Count R R = ()  || = (4log2)3/2 () / ( FWHMx FWHMy FWHMz ) Volume of search region in units of smoothness Eg: 10 voxels, 2.5 FWHM 4 RESELS Beware RESEL misinterpretation RESEL are not “number of independent ‘things’ in the image” See Nichols & Hayasaka, 2003, Stat. Meth. in Med. Res. . 1 2 3 4 6 8 10 5 7 9

Random Field Theory Smoothness Estimation
Smoothness est’d from standardized residuals Variance of gradients Yields resels per voxel (RPV) RPV image Local roughness est. Can transform in to local smoothness est. FWHM Img = (RPV Img)-1/D Dimension D, e.g. D=2 or 3

Random Field Intuition
Corrected P-value for voxel value t Pc = P(max T > t)  E(t)  () ||1/2 t2 exp(-t2/2) Statistic value t increases Pc decreases (but only for large t) Search volume increases Pc increases (more severe MTP) Smoothness increases (roughness ||1/2 decreases) Pc decreases (less severe MTP)

RFT Details: Unified Formula
General form for expected Euler characteristic 2, F, & t fields • restricted search regions • D dimensions • E[u(W)] = Sd Rd (W) rd (u) Rd (W): d-dimensional Minkowski functional of W – function of dimension, space W and smoothness: R0(W) = (W) Euler characteristic of W R1(W) = resel diameter R2(W) = resel surface area R3(W) = resel volume rd (W): d-dimensional EC density of Z(x) – function of dimension and threshold, specific for RF type: E.g. Gaussian RF: r0(u) = 1- (u) r1(u) = (4 ln2)1/2 exp(-u2/2) / (2p) r2(u) = (4 ln2) exp(-u2/2) / (2p)3/2 r3(u) = (4 ln2)3/2 (u2 -1) exp(-u2/2) / (2p)2 r4(u) = (4 ln2)2 (u3 -3u) exp(-u2/2) / (2p)5/2 

Random Field Theory Cluster Size Tests
5mm FWHM 10mm FWHM 15mm FWHM Expected Cluster Size E(S) = E(N)/E(L) S cluster size N suprathreshold volume ({T > uclus}) L number of clusters E(N) = () P( T > uclus ) E(L)  E(u) Assuming no holes

RFT Cluster Inference & Stationarity
Problem w/ VBM Standard RFT result assumes stationarity, constant smoothness Assuming stationarity, false positive clusters will be found in extra-smooth regions VBM noise very non-stationary Nonstationary cluster inference Must un-warp nonstationarity Reported but not implemented Hayasaka et al, NeuroImage 22:676– 687, 2004 Now available in SPM toolboxes Gaser’s VBM toolbox (w/ recent update!) VBM: Image of FWHM Noise Smoothness Nonstationary noise… …warped to stationarity

Random Field Theory Cluster Size Distribution
Gaussian Random Fields (Nosko, 1969) D: Dimension of RF t Random Fields (Cao, 1999) B: Beta distn U’s: 2’s c chosen s.t. E(S) = E(N) / E(L)

Random Field Theory Cluster Size Corrected P-Values
Previous results give uncorrected P-value Corrected P-value Bonferroni Correct for expected number of clusters Corrected Pc = E(L) Puncorr Poisson Clumping Heuristic (Adler, 1980) Corrected Pc = 1 - exp( -E(L) Puncorr )

Random Field Theory Limitations
Lattice Image Data  Continuous Random Field Sufficient smoothness FWHM smoothness 3-4× voxel size (Z) More like ~10× for low-df T images Smoothness estimation Estimate is biased when images not sufficiently smooth Multivariate normality Virtually impossible to check Several layers of approximations Stationary required for cluster size results

Real Data fMRI Study of Working Memory Second Level RFX ...
yes UBKDA Active fMRI Study of Working Memory 12 subjects, block design Marshuetz et al (2000) Item Recognition Active:View five letters, 2s pause, view probe letter, respond Baseline: View XXXXX, 2s pause, view Y or N, respond Second Level RFX Difference image, A-B constructed for each subject One sample t test ... N no XXXXX Baseline

Real Data: RFT Result Threshold Result S = 110,776
2  2  2 voxels 5.1  5.8  6.9 mm FWHM u = 9.870 Result 5 voxels above the threshold minimum FWE-corrected p-value -log10 p-value

Nonparametric Inference
Parametric methods Assume distribution of statistic under null hypothesis Needed to find P-values, u Nonparametric methods Use data to find distribution of statistic under null hypothesis Any statistic! 5% Parametric Null Distribution 5% Nonparametric Null Distribution

Permutation Test Toy Example
Data from V1 voxel in visual stim. experiment A: Active, flashing checkerboard B: Baseline, fixation 6 blocks, ABABAB Just consider block averages... Null hypothesis Ho No experimental effect, A & B labels arbitrary Statistic Mean difference A B 103.00 90.48 99.93 87.83 99.76 96.06

Under Ho Consider all equivalent relabelings AAABBB ABABAB BAAABB BABBAA AABABB ABABBA BAABAB BBAAAB AABBAB ABBAAB BAABBA BBAABA AABBBA ABBABA BABAAB BBABAA ABAABB ABBBAA BABABA BBBAAA

Under Ho Consider all equivalent relabelings Compute all possible statistic values AAABBB ABABAB BAAABB BABBAA AABABB ABABBA BAABAB BBAAAB AABBAB ABBAAB BAABBA BBAABA AABBBA ABBABA BABAAB BBABAA ABAABB ABBBAA BABABA BBBAAA

Under Ho Consider all equivalent relabelings Compute all possible statistic values Find 95%ile of permutation distribution AAABBB ABABAB BAAABB BABBAA AABABB ABABBA BAABAB BBAAAB AABBAB ABBAAB BAABBA BBAABA AABBBA ABBABA BABAAB BBABAA ABAABB ABBBAA BABABA BBBAAA

Under Ho Consider all equivalent relabelings Compute all possible statistic values Find 95%ile of permutation distribution -8 -4 4 8

Controlling FWE: Permutation Test
Parametric methods Assume distribution of max statistic under null hypothesis Nonparametric methods Use data to find distribution of max statistic under null hypothesis Again, any max statistic! 5% Parametric Null Max Distribution 5% Nonparametric Null Max Distribution

Permutation Test Other Statistics
Collect max distribution To find threshold that controls FWE Consider smoothed variance t statistic To regularize low-df variance estimate

Permutation Test Smoothed Variance t
Collect max distribution To find threshold that controls FWE Consider smoothed variance t statistic mean difference t-statistic variance

Permutation Test Smoothed Variance t
Collect max distribution To find threshold that controls FWE Consider smoothed variance t statistic mean difference Smoothed Variance t-statistic smoothed variance

Permutation Test Strengths
Requires only assumption of exchangeability Under Ho, distribution unperturbed by permutation Allows us to build permutation distribution Subjects are exchangeable Under Ho, each subject’s A/B labels can be flipped fMRI scans not exchangeable under Ho Due to temporal autocorrelation

Permutation Test Limitations
Computational Intensity Analysis repeated for each relabeling Not so bad on modern hardware No analysis discussed below took more than 3 hours Implementation Generality Each experimental design type needs unique code to generate permutations Not so bad for population inference with t-tests

Permutation Test Example
... D yes UBKDA Active fMRI Study of Working Memory 12 subjects, block design Marshuetz et al (2000) Item Recognition Active:View five letters, 2s pause, view probe letter, respond Baseline: View XXXXX, 2s pause, view Y or N, respond Second Level RFX Difference image, A-B constructed for each subject One sample, smoothed variance t test ... N no XXXXX Baseline

Permute! 212 = 4,096 ways to flip 12 A/B labels For each, note maximum of t image . Permutation Distribution Maximum t Maximum Intensity Projection Thresholded t

Compare with Bonferroni  = 0.05/110,776 Compare with parametric RFT 110,776 222mm voxels 5.15.86.9mm FWHM smoothness 462.9 RESELs

t11 Statistic, Nonparametric Threshold
uPerm = sig. vox. uRF = 9.87 uBonf = sig. vox. t11 Statistic, Nonparametric Threshold t11 Statistic, RF & Bonf. Threshold Test Level vs. t11 Threshold Smoothed Variance t Statistic, Nonparametric Threshold 378 sig. vox.

Does this Generalize? RFT vs Bonf. vs Perm.

RFT vs Bonf. vs Perm.

Reliability with Small Groups
Consider n=50 group study Event-related Odd-Ball paradigm, Kiehl, et al. Analyze all 50 Analyze with SPM and SnPM, find FWE thresh. Randomly partition into 5 groups 10 Analyze each with SPM & SnPM, find FWE thresh Compare reliability of small groups with full With and without variance smoothing . Skip

SPM t11: 5 groups of 10 vs all 50 5% FWE Threshold
10 subj 10 subj 10 subj T>10.69 T>10.10 T>4.66 10 subj 10 subj all 50

SnPM t: 5 groups of 10 vs. all 50 5% FWE Threshold
10 subj 10 subj 10 subj Arbitrary thresh of 9.0 T>9.00 T>6.49 T>6.19 T>4.09 10 subj 10 subj all 50

SnPM SmVar t: 5 groups of 10 vs. all 50 5% FWE Threshold
10 subj 10 subj 10 subj Arbitrary thresh of 9.0 T>9.00 all 50 T>4.84 T>4.64 10 subj 10 subj

MTP Solutions: Measuring False Positives
Familywise Error Rate (FWE) Familywise Error Existence of one or more false positives FWE is probability of familywise error False Discovery Rate (FDR) FDR = E(V/R) R voxels declared active, V falsely so Realized false discovery rate: V/R

rFDR = V0R/(V1R+V0R) = V0R/NR
False Discovery Rate For any threshold, all voxels can be cross-classified: Realized FDR rFDR = V0R/(V1R+V0R) = V0R/NR If NR = 0, rFDR = 0 But only can observe NR, don’t know V1R & V0R We control the expected rFDR FDR = E(rFDR) Accept Null Reject Null Null True V0A V0R m0 Null False V1A V1R m1 NA NR V

False Discovery Rate Illustration:
Noise Signal Signal+Noise

Control of Per Comparison Rate at 10%
11.3% 12.5% 10.8% 11.5% 10.0% 10.7% 11.2% 10.2% 9.5% Percentage of Null Pixels that are False Positives Control of Familywise Error Rate at 10% FWE Occurrence of Familywise Error Control of False Discovery Rate at 10% 6.7% 10.4% 14.9% 9.3% 16.2% 13.8% 14.0% 10.5% 12.2% 8.7% Percentage of Activated Pixels that are False Positives

Benjamini & Hochberg Procedure
Select desired limit q on FDR Order p-values, p(1)  p(2)  ...  p(V) Let r be largest i such that Reject all hypotheses corresponding to p(1), ... , p(r). JRSS-B (1995) 57: p(i)  i/V  q 1 p(i) p-value i/V  q i/V 1

Adaptiveness of Benjamini & Hochberg FDR
Ordered p-values p(i) P-value threshold when no signal: /V P-value threshold when all signal:  Fractional index i/V

Benjamini & Hochberg Procedure Details
Standard Result Positive Regression Dependency on Subsets P(X1c1, X2c2, ..., Xkck | Xi=xi) is non-decreasing in xi Only required of null xi’s Positive correlation between null voxels Positive correlation between null and signal voxels Special cases include Independence Multivariate Normal with all positive correlations Arbitrary covariance structure Replace q by q/c(V), c(V) = i=1,...,V 1/i  log(V) Much more stringent Benjamini & Yekutieli (2001). Ann. Stat. 29:

Benjamini & Hochberg: Key Properties
FDR is controlled E(rFDR)  q m0/V Conservative, if large fraction of nulls false Adaptive Threshold depends on amount of signal More signal, More small p-values, More p(i) less than i/V  q/c(V)

Controlling FDR: Varying Signal Extent
p = z = Signal Intensity 3.0 Signal Extent 1.0 Noise Smoothness 3.0 1

p = z = 3.48 Signal Intensity 3.0 Signal Extent 5.0 Noise Smoothness 3.0 4

Controlling FDR: Benjamini & Hochberg
Illustrating BH under dependence Extreme example of positive dependence 8 voxel image 1 32 voxel image (interpolated from 8 voxel image) p(i) p-value i/V  q/c(V) i/V 1

Real Data: FDR Example Threshold Result Indep/PosDep u = 3.83
Arb Cov u = 13.15 Result 3,073 voxels above Indep/PosDep u < minimum FDR-corrected p-value FWE Perm. Thresh. = voxels FDR Threshold = ,073 voxels

Conclusions Must account for multiplicity FWE FDR
Otherwise have a fishing expedition FWE Very specific, not so sensitive Random Field Theory Great for single-subject fMRI, EEG Nonparametric / SnPM Much better power voxel-wise than RFT for small DF FDR Less specific, more sensitive Interpret with care! FP risk is over whole set of surviving voxels

References Most of this talk covered in these papers
TE Nichols & S Hayasaka, Controlling the Familywise Error Rate in Functional Neuroimaging: A Comparative Review. Statistical Methods in Medical Research, 12(5): , 2003. TE Nichols & AP Holmes, Nonparametric Permutation Tests for Functional Neuroimaging: A Primer with Examples. Human Brain Mapping, 15:1-25, 2001. CR Genovese, N Lazar & TE Nichols, Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate. NeuroImage, 15: , 2002.

Inference on SPMs: Random Field Theory & Alternatives

Similar presentations

Presentation on theme: "Inference on SPMs: Random Field Theory & Alternatives"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Inference on SPMs: Random Field Theory & Alternatives

Similar presentations

Presentation on theme: "Inference on SPMs: Random Field Theory & Alternatives"— Presentation transcript:

Similar presentations

About project

Feedback