Statistics for fMRI

Statistics and science Quick poll: When was statistics (p-values for rejecting a hypothesis, t-tests etc.) invented? A: Early 1700s B: Early 1800s C: Early 1900s

Statistics and science Answer: Early 1900s t-test: 1908 (Gosset / “Student” ) General principles: 1925 “Statistical Methods for Research Workers” Ronald A. Fisher

Statistics and science A puzzle: Lots of great science was done before statistics came along! t-test: 1908 (Gosset / “Student” ) General principles: 1925 “Statistical Methods for Research Workers” Ronald A. Fisher Some discoveries which (today) seem unthinkable without statistics: Development of vaccines (Jenner, 1796) Genetics and inheritance (Mendel, 1866) Almost all of the rest of science!

A metaphor: doing stats right is like putting on your seatbelt What really matters in science is finding a true underlying regularity in nature Gold standard test: independent replication Neither necessary nor sufficient, but helpful: doing the stats right What really matters in driving is getting home safely Neither necessary nor sufficient, but helpful: wearing your seatbelt For a long time, cars didn’t even have seatbelts. But people still got home safely (usually)

Multiple comparisons test There are lots of voxels in the brain (~30,000) You can test each one separately to see whether it is active If you run lots of tests, some of them will give positive results even if nothing is there

http://xkcd.com/882/

http://xkcd.com/882/

http://xkcd.com/882/

http://xkcd.com/882/

Mouse-float-over title text: http://www.explainxkcd.com/wiki/index.php/882:_Significant Mouse-float-over title text: So, uh, we did the green study again and got no link. It was probably a-- "RESEARCH CONFLICTED ON GREEN JELLY BEAN/ACNE LINK; MORE STUDY RECOMMENDED!"

http://xkcd.com/552/ Correlation does not imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing "look over there."

What does “Result X is statistically significant (p<0 What does “Result X is statistically significant (p<0.05)” actually mean? Take a vote! A: Given the data that we found, there is only a 5% chance that Result X is false B: If Result X were false, then there would only be a 5% chance of us finding the data that we did find C: If some other people now go off and look for Result X, there is only a 5% chance that they will fail to find it D: All of the above answers all mean basically the same thing

Base rates and false positives Sensitivity: probability of detecting a real effect Specificity: prob. of not getting a false positive Jacob Cohen “The Earth is round (p<0.05)”

What does “Result X is statistically significant (p<0 What does “Result X is statistically significant (p<0.05)” actually mean? A: Given the data that we found, there is only a 5% chance that Result X is false p( Hypothesis | Data ) B: If Result X were false, then there would only be a 5% chance of us finding the data that we did find p( Data | Hypothesis ) C: If some other people now go off and look for Result X, there is only a 5% chance that they will fail to find it D: All of the above answers all mean basically the same thing

Example: testing for a rare disease Suppose 0.1% of population has bubonic plague Suppose test for plague has 99% specificity (false positive rate) Test 1000 people: 1 person has plague 10 people test positive 9 of those are false positives!

What does “probability” actually mean? True or false? There is a single widely-agreed upon answer to the question of what exactly it means to say “The probability that it will rain tomorrow is 0.1” A1: True, there is an agreed upon answer A2: False, lots of sensible people disagree on this

Different ways of correcting for multiple comparisons Bonferroni correction Family-Wise Error False Discovery Rate

The infamous “dead salmon” study Bennett et al. (2009) If you scan a dead salmon and don’t do any multiple comparison correction, then you get some active voxels This is not exactly news, but it is at least memorable Like saying “Big discovery in traffic safety: remember to wear your seatbelt!”

Different ways of correcting for multiple comparisons Raw signal intensity Uncorrected results Family-Wise Error correction False Discovery Rate correction

Voodoo correlations – Vul et al (2009) ROI analysis Correlation with behaviour within that ROI Problem: It is tempting (and easy) to select the ROI, and to look for correlations within the ROI, using the same data: non-independent

An example highlighted in the voodoo correlations paper Eisenberger, Naomi I., Matthew D. Lieberman, and Kipling D. Williams. "Does rejection hurt? An fMRI study of social exclusion." Science 302.5643 (2003): 290-292.

Problem: correlations become inflated Vul, E. et al. (2009) "Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition." Perspectives on Psychological Science, 4(3): 274-290. Similar argument made simultaneously in: Kriegeskorte, N. et al. (2009). Circular analysis in systems neuroscience: the dangers of double dipping. Nature Neuroscience, 12(5), 535-540.

The solution: Cross-validation The solution: Cross-validation. Splitting up your data into independent parts Training set and testing set Independent parts of the data Just like in using classifiers

This Thursday Computer Lab: Meliora Rm 178. Registered students only (sorry) Explore hands-on the preprocessing steps involved in fMRI, using SPM: Motion correction Spatial normalisation to a template Smoothing You are welcome to bring your own laptop, but you’ll need to pre-load the Haxby data, Matlab and SPM8