NA-MIC National Alliance for Medical Image Computing Non-Parametric Statistical Permutation Tests for Local Shape Analysis Martin Styner, UNC Dimitrios Pantazis, Richard Leahy, USC LA Tom Nichols, University of Michigan Ann Arbor

National Alliance for Medical Image Computing TOC Motivation local shape analysis Local shape difference/distance measures Statistical significance maps Problem: Multiple correlated comparisons 1 st approach: It’s a hack! 2 nd approach: Let’s do it right! Template free - Hotelling T 2 measures Example Results Conclusions & Outlook

National Alliance for Medical Image Computing Motivation Shape Analysis Anatomical studies of brain structures –Changes between patient and healthy controls –Detection, Enhanced understanding, course of disease, pathology –Normal neuro-development interest in diseases with brain changes –Schizophrenia, autism, fragile-X, Alzheimer's Information additional to volume Both volumetric and shape analysis Shape analysis: where and how?

National Alliance for Medical Image Computing Shape Distances Shape description: –SPHARM-PDM –M-rep Normalization: –Rigid Procrustes, brain size normalized Local scalar distance –Euclidean distance –“Radius” difference –Signed vs absolute

National Alliance for Medical Image Computing Local Shape Analysis Distance to template Distance between subject pairs Sets of distance-maps Significance map –Statistical test at each point –Mean difference test P-values Significance threshold

National Alliance for Medical Image Computing Multiple Comparisons Lots of correlated statistical tests → Overly optimistic –M-rep: 2x24 tests, SPHARM: 2252 tests –Same problem with other shape descriptions and other difference analysis schemes Correction needed, overly optimistic –Test locally at given level (e.g. α = 0.05) –Globally incorrect false-positive rate Bonferroni correction, worst case, assumption: 0% correlation –Correct False-Positive rate at α/n = 0.05/4000 = –Correct False-Positive rate at 1-(1- α) 1/n =

National Alliance for Medical Image Computing 1 st Approach: SnPM Statistical non-Parametric Maps in SPM (SPIE 2004) Decomposition of distance map into separate images for processing in SnPM 75% overlap necessary due to distortions Each image is tested separately in SnPM ONE BIG HACK: –6 correlated tests –Averaging in overlap

National Alliance for Medical Image Computing 2 nd Approach: Permutations Non-parametric permutation test using spatially summarized statistics, ISBI 2004 Correct false positive control (Type II) Summary: –Random permutations of the group labels –Metric for difference between populations –Spatial normalization for uniform spatial sensitivity –Summarize statistics across whole shape –Choose threshold in summary statistic

National Alliance for Medical Image Computing Statistical Problem 2 groups: a & b, #member n a, n b Each member: p-features (e.g. 4000) Test: Is the mean of each feature in the 2 populations the same? –Null hypothesis: The mean of each feature is the same –Permutations of group label leave distributions unchanged under null hypothesis –M permutations Specific test –Correct false positive rate

National Alliance for Medical Image Computing Non-parametric Permutation Tests Goal: significance for a vector with 4’000 correlated variables 50’000 to 100’000 permutations Extrema statistic: controls false-positive diff norm Summary Statistic Min/Max Histogram diff norm

National Alliance for Medical Image Computing Single Feature Example Feature f A,1 -f A,n1 vs f B,1 -f B,n1 Compute difference: T 0 =|  A -  B| Permute group label → A’ i,B’ I → T i Make Histogram of T i Histogram = pdf Sum histogram = cdf Cdf at 1-α = Threshold α

National Alliance for Medical Image Computing Multiple features Testing a single feature → no problem Testing multiple features together as a whole, NOT individually Summary is necessary of all features across the surface For correct Type II, use an extrema measurement –Right sided distance metrics → Maxima –Left sided distance metrics → Minima

National Alliance for Medical Image Computing Spatial Normalization Extremal summary is most influenced by regions with higher variance Assume 2 regions with same difference, but one has larger variance –Region with larger variance contributes more to extremal statistics and thus sensitivity in that region is higher Normalization of local statistical distributions is necessary for spatially uniform sensitivity

National Alliance for Medical Image Computing Spatial Normalization A) local p-values, non-parametric –Minimum, (1-α) thresh B) standard deviation, parametric –Maximum, α thresh C) q-th quantile, non-parametric –q = 68% ~  if Gaussian –Maximum, α thresh Assumptions: A > C > B Uniform sensitivity: A > C ~ B Numerical pdf: C > B > A Use A –Many permutations –High computation + space costs Extrema statistics Shape difference metric α1-α Norm  Max-stat Norm p-value Min-stat

National Alliance for Medical Image Computing Raw vs Corrected P-values Raw significance map: –4000 elements, 5% → 200 will be significant at 5% by pure chance, if locations are uncorrelated. Corrected significance map –Correct control of false negative –Single location significant → whole shape significant No assumption over local covariance –Overly pessimistic –There is room for improvement!

National Alliance for Medical Image Computing Raw vs Corrected P-values Raw p-values are comparable But visualization of raw p-value map is misleading even without statement about significance –Too optimistic, often viewed using linear colormap –P-value correction is non-linear ! Correction factor: F = Raw-P / Corr-P

National Alliance for Medical Image Computing Metric for Group difference Scalar Local difference: –Signed/Unsigned Euclidean distance –Thickness difference –Pairs, Template Difference of mean metric → Statistical feature T = |  A -  B | Needed: Positive scalar  + shape difference metric between populations PDM: Mean difference of Euclidean distance at a selected point Gaussian, passed Lilliefors test 0.01

National Alliance for Medical Image Computing Template Free Stats No need for a scalar value at each location for each subject Positive scalar difference value between populations SPHARM-PDM –So far: Signed/absolute Euclidean distance at each location to template → Scalar field analysis –New: Difference vectors to template → Vector field analysis –Better: Location vector at each location → Template free analysis → Length of difference vector between mean vectors of populations → Hotelling T 2 distance between populations = Hotelling T 2 is mean difference 2 vector weighted with the pooled Covariance matrix T 2 = (μ a – μ b ) Σ a,b (μ a –μ b ) Σ a,b = ( (n a - 1) Σ a + (n b -1) Σ b ) / (n a +n b - 2)

National Alliance for Medical Image Computing Hotelling T2 histogram Hotelling T 2 distance of locations (template free) →  2

National Alliance for Medical Image Computing Results SnPM hack vs Correct permutation tests Sample Hippocampus study: Stanley study, resp/non-resp SZ (56) vs Cnt (26) –Both M-rep & PDM Other example tests

National Alliance for Medical Image Computing SnPM-Hack vs Correct Stat SnPM too optimistic –relatively good agreement L R SnPM

National Alliance for Medical Image Computing Hippocampus SZ Study Left Right

National Alliance for Medical Image Computing M-rep Shape Analysis Left Right

National Alliance for Medical Image Computing Vector Field Analysis T 2 location T 2 template difference Abs template distance (scalar) Raw Significance Maps Corr Significance Maps

National Alliance for Medical Image Computing Conclusions of Methods Multiple comparison correction scheme for local shape analysis –Non-parametric, Permutation-based –Globally correct for false-positive across whole object –Applicable to scalar, vectors, any Euclidean space measures –Black box –Pessimistic estimate

National Alliance for Medical Image Computing NAMIC kit StatNonParamTestPDM –Command line tool, Win/Linux/MacOSX –E.g. StatNonParamTestPDM -out -surfList -numPerms signLevel signSteps 1000 Output (for meshes) –P-value of global shape difference between the populations (mean T 2 across surface) –Mean difference map (effect size) –Hotelling T 2 map using robust T 2 formula –Raw significance map –Corrected significance map –Mean surfaces of the 2 groups

National Alliance for Medical Image Computing StatNonParamTestPDM Input: File with list of ITK mesh files Generic features also supported using customizable text-file input option Currently in NAMIC-Sandbox (open) Next: submission to Insight Journal MeshVisu, combination of Mesh and maps ….. Map Txt

National Alliance for Medical Image Computing That’s it folks… Questions

National Alliance for Medical Image Computing Corrected Analysis – Spatial Normalization Without normalization → incorrect, unless uniformity is assumed –High variability → overestimation of significance –Low variability → underestimation of significance  -normalization ~ 68% normalization No norm max stat L R