Presentation is loading. Please wait.

Presentation is loading. Please wait.

Correction for multiple comparisons in FreeSurfer

Published byHugo Mosley Modified over 6 years ago

Similar presentations

Presentation on theme: "Correction for multiple comparisons in FreeSurfer"— Presentation transcript:

1 Correction for multiple comparisons in FreeSurfer

2 Problem of Multiple Comparisons
p value is probability that a voxel is falsely activated Threshold too liberal: many false positives Threshold too restrictive: lose activation (false negatives)

3 Clusters p < 0.10 p < 0.01 p < 10-7
- True signal tends to be clustered - False Positives tend to be randomly distributed in space - Cluster – set of spatially contiguous voxels that are above a given threshold.

4 Cluster-forming Threshold
p<.00001 sig>5 p<.0001 sig>4 Unthresholded p<.001 sig>3 As threshold lowers, clusters may expand or merge and new clusters can form. There is no way to say what the threshold is best.

5 Cluster Table, Uncorrected
p<.0001 sig>4 38 clusters ClusterNo Area(mm2) X Y Z Structure Cluster superiorfrontal Cluster insula Cluster superiorparietal Cluster precentral Cluster supramarginal … How likely is it to get a cluster of a certain size under the null hypothesis?

6 Clusterwise Correction
Supramarginal Gyrus Cluster mm2 How likely is it to get a cluster mm2 or bigger by chance? How likely is it to get a cluster of a certain size under the null hypothesis?

7 Cluster-based Correction for Multiple Comparisons
Simulate data under Null Hypothesis: Synthesize Gaussian noise and then smooth (Monte Carlo), like random field theory Permute rows of design matrix (Permutation, orthog.) Analyze, threshold, cluster, get MaxClusterSizeNull Repeat 10,000 times – gives a list of MaxClusterSizeNulls under the null Analyze real data, get ClusterSize (eg, mm2) Count number of times MaxClusterSizeNull > ClusterSize P(cluster) = #(MaxClusterSizeNull > ClusterSize) /10000 Histogram of MaxClusterSizeNull mri_glmfit-sim

8 Cluster Table, Corrected
p<.0001 sig>4 22 clusters out of 38 have cluster p-value < .05 ClusterNo Area(mm2) X Y Z Structure Cluster P Cluster superiorfrontal Cluster insula Cluster superiorparietal Cluster precentral Cluster supramarginal … Note the difference between the Cluster Forming Threshold (p<.0001) and the Clusterwise p-value (.05).

9 Clusterwise Correction
Supramarginal Gyrus Cluster mm2 Probability of getting a cluster mm2 or bigger by chance is p=.04 This is the clusterwise p-value.

10 Surface-based Correction for Multiple Comparisons
2D Cluster-based Correction at p < .05 mri_glmfit-sim --glmdir lh.gender_age.glmdir --cache 2 pos --2spaces --cwpvalthresh .05

11 Surface-based Correction for Multiple Comparisons
Original mri_glmfit command: mri_glmfit --y lh.thickness.sm10.mgh --fsgd gender_age.txt --C age.mtx –C gender.mtx --surf fsaverage lh --cortex --glmdir lh.gender_age.glmdir 2D Cluster-based Correction at p < .05 mri_glmfit-sim --glmdir lh.gender_age.glmdir --cache 2 pos --2spaces --cwpvalthresh .05 lh.gender_age.glmdir/ beta.mgh – parameter estimates rvar.mgh – residual error variance age/ sig.mgh – -log10(p), uncorrected gamma.mgh, F.mgh gender/

12 Surface-based Correction for Multiple Comparisons
2D Cluster-based Correction at p < .05 Use pre-computed simulation results positive contrast voxelwise threshold thres = 2 (p<.01) Gaussian assumption Can do another simulation or permutation Not related to recon-all -qcache mri_glmfit-sim --glmdir lh.gender_age.glmdir --cache 2 pos --2spaces --cwpvalthresh . 05

13 Surface-based Correction for Multiple Comparisons
2D Cluster-based Correction at p < .05 Cluster-wise threshold p<.05 cw = cluster-wise pval = p value thresh = threshold mri_glmfit-sim --glmdir lh.gender_age.glmdir --cache 2 pos --cwpvalthresh .05 --2spaces

14 Surface-based Correction for Multiple Comparisons
2D Cluster-based Correction at p < .05 Doing analysis with left hemi but right hemi will be done separately. Need to correct for full search space. mri_glmfit-sim --glmdir lh.gender_age.glmdir --cache 2 pos --cwpvalthresh .05 --2spaces

15 Correction for Multiple Comparisons Output
mri_glmfit-sim --glmdir lh.gender_age.glmdir --cache 2 pos --cwpvalthresh .05 --2spaces lh.gender_age.glmdir age gender sig.mgh – pre-existing uncorrected p-values cache.th20.pos.sig.cluster.mgh – map of significance of clusters cache.th20.pos.sig.ocn.annot – annotation of significant clusters cache.th20.pos.sig.cluster.summary – text file of cluster table (clusters, sizes, MNI305 XYZ, and their significances) Only shows clusters p<.05, change –cwpvalthesh to a larger value to get more (ie, less sig) clusters

16 Corrected Outputs ClusterNo Area(mm2) X Y Z Structure Cluster P
cache.th20.pos.sig.ocn.annot – annotation of significant clusters cache.th20.pos.sig.cluster.summary -- text file of cluster table ClusterNo Area(mm2) X Y Z Structure Cluster P Cluster superiorfrontal Cluster insula Cluster superiorparietal Cluster precentral Cluster supramarginal …

17 Eklund, et al, 2016 PNAS High false positive rates using resting state fMRI analyzed as task data Caused by non-Gaussian smoothness Applies to parametric (random field theory) and Monte Carlo simulations Effect present in thickness analysis, but not as bad Eklund fMRI results were in the range of 10-60% false positive rate. Cluster Failure: Why fMRI inferences for spatial extent have inflated false-positive rates. Eklund, Nichols, Knutsson, PNAS

18 Output files are the same (prepended with “perm.”)
Permutation If there is no effect of group, then group membership can be randomly changed. Repeat this many times to get NULL distribution Makes no assumptions about Gaussianity of the data. v6 Requires designs without nuisance vars (age) Patch to allow for non-orthogonal designs using the ter Braak method ftp://surfer.nmr.mgh.harvard.edu/pub/dist/freesurfer/6.0.0-patch/mri_glmfit-sim Example command: Output files are the same (prepended with “perm.”) mri_glmfit-sim –perm pos --perm-resid Note: use “--perm-resid” with non-orthogonal designs (patch)

19 False Discover Correction Possible
False Discovery Rate (FDR) Built into tksurfer & QDEC (Genovese, et al, NI 2002) mri_fdr --help

20 Tutorial Command-line Stream QDEC – same data set
Create an FSGD File for a thickness study Age and Gender Run mris_preproc mri_surf2surf mri_glmfit mri_glmfit-sim tksurfer QDEC – same data set

21 QDEC – An Interactive Statistical Engine GUI
Query – Select subjects based on Match Criteria Design – Specify discrete and continuous factors Estimate – Fit Model Contrast – Automatically Generate Contrast Matrices Interactive – Makes easy things easy (that used to be hard) …a work in progress No Query yet Two Discrete Factors (Two Levels) Two Continuous Factors Surface only

22 recon-all -s <sid> -qcache
For QDEC to work interactively, you need to run: recon-all -s <sid> -qcache (or as additional flag in your regular processing) This will map and smooth thickness maps to fsaverage, use –target <id> to specify your own target and –measure <surfmeas> to specify curv, area, sulc etc.

23 QDEC – Spreadsheet Discrete Factors need a factorname.level file
qdec.table.dat – spreadsheet with subject information – can be huge! fsid gender age diagnosis Left-Cerebral-White-Matter-Vol 011121_vc Female Demented 021121_ Female Demented 010607_vc Female Nondemented 021121_vc Male Demented 020718_ Male Demented 020322_vc Male Nondemented gender.levels diagnosis.levels Discrete Factors need a factorname.level file Female Male Demented Nondemented

24 QDEC GUI Load QDEC Table File List of Subjects
List of Factors (Discrete and Cont) Choose Factors Choose Input (cached): Hemisphere Measure (eg, thickness) Smoothing Level “Analyze” Builds Design Matrix Builds Contrast Matrices Constructs Human-Readable Questions Analyzes Displays Results

Download ppt "Correction for multiple comparisons in FreeSurfer"

Similar presentations

Surface-based Group Analysis in FreeSurfer

Surface-based Group Analysis in FreeSurfer
SPM 2002 C1C2C3 X =  C1 C2 Xb L C1 L C2  C1 C2 Xb L C1  L C2 Y Xb e Space of X C1 C2 Xb Space X C1 C2 C1  C3 P C1C2  Xb Xb Space of X C1 C2 C1 

SPM 2002 C1C2C3 X =  C1 C2 Xb L C1 L C2  C1 C2 Xb L C1  L C2 Y Xb e Space of X C1 C2 Xb Space X C1 C2 C1  C3 P C1C2  Xb Xb Space of X C1 C2 C1 
FMRI Data Analysis: I. Basic Analyses and the General Linear Model

FMRI Data Analysis: I. Basic Analyses and the General Linear Model
Multiple testing Justin Chumbley Laboratory for Social and Neural Systems Research University of Zurich With many thanks for slides & images to: FIL Methods.

Multiple testing Justin Chumbley Laboratory for Social and Neural Systems Research University of Zurich With many thanks for slides & images to: FIL Methods.
Topological Inference Guillaume Flandin Wellcome Trust Centre for Neuroimaging University College London SPM Course London, May 2014 Many thanks to Justin.

Topological Inference Guillaume Flandin Wellcome Trust Centre for Neuroimaging University College London SPM Course London, May 2014 Many thanks to Justin.
Classical inference and design efficiency Zurich SPM Course 2014

Classical inference and design efficiency Zurich SPM Course 2014
Multiple testing Justin Chumbley Laboratory for Social and Neural Systems Research Institute for Empirical Research in Economics University of Zurich With.

Multiple testing Justin Chumbley Laboratory for Social and Neural Systems Research Institute for Empirical Research in Economics University of Zurich With.
07/01/15 MfD 2014 Xin You Tai & Misun Kim

07/01/15 MfD 2014 Xin You Tai & Misun Kim
Multiple testing Justin Chumbley Laboratory for Social and Neural Systems Research Institute for Empirical Research in Economics University of Zurich With.

Multiple testing Justin Chumbley Laboratory for Social and Neural Systems Research Institute for Empirical Research in Economics University of Zurich With.
Multiple comparison correction Methods & models for fMRI data analysis 18 March 2009 Klaas Enno Stephan Laboratory for Social and Neural Systems Research.

Multiple comparison correction Methods & models for fMRI data analysis 18 March 2009 Klaas Enno Stephan Laboratory for Social and Neural Systems Research.
Comparison of Parametric and Nonparametric Thresholding Methods for Small Group Analyses Thomas Nichols & Satoru Hayasaka Department of Biostatistics U.

Comparison of Parametric and Nonparametric Thresholding Methods for Small Group Analyses Thomas Nichols & Satoru Hayasaka Department of Biostatistics U.
Multiple comparison correction Methods & models for fMRI data analysis 29 October 2008 Klaas Enno Stephan Branco Weiss Laboratory (BWL) Institute for Empirical.

Multiple comparison correction Methods & models for fMRI data analysis 29 October 2008 Klaas Enno Stephan Branco Weiss Laboratory (BWL) Institute for Empirical.
Surface-based Analysis: Intersubject Registration and Smoothing

Surface-based Analysis: Intersubject Registration and Smoothing
General Linear Model & Classical Inference Guillaume Flandin Wellcome Trust Centre for Neuroimaging University College London SPM M/EEGCourse London, May.

General Linear Model & Classical Inference Guillaume Flandin Wellcome Trust Centre for Neuroimaging University College London SPM M/EEGCourse London, May.
2nd Level Analysis Jennifer Marchant & Tessa Dekker

2nd Level Analysis Jennifer Marchant & Tessa Dekker
Multiple Comparison Correction in SPMs Will Penny SPM short course, Zurich, Feb 2008 Will Penny SPM short course, Zurich, Feb 2008.

Multiple Comparison Correction in SPMs Will Penny SPM short course, Zurich, Feb 2008 Will Penny SPM short course, Zurich, Feb 2008.
Random Field Theory Will Penny SPM short course, London, May 2005 Will Penny SPM short course, London, May 2005 David Carmichael MfD 2006 David Carmichael.

Random Field Theory Will Penny SPM short course, London, May 2005 Will Penny SPM short course, London, May 2005 David Carmichael MfD 2006 David Carmichael.
Basics of fMRI Inference Douglas N. Greve. Overview Inference False Positives and False Negatives Problem of Multiple Comparisons Bonferroni Correction.

Basics of fMRI Inference Douglas N. Greve. Overview Inference False Positives and False Negatives Problem of Multiple Comparisons Bonferroni Correction.
With a focus on task-based analysis and SPM12

With a focus on task-based analysis and SPM12
Multiple testing Justin Chumbley Laboratory for Social and Neural Systems Research Institute for Empirical Research in Economics University of Zurich With.

Multiple testing Justin Chumbley Laboratory for Social and Neural Systems Research Institute for Empirical Research in Economics University of Zurich With.

Similar presentations

Ads by Google