1. A (very) brief introduction to multivoxel analysis “stuff” Jo Etzel, Social Brain Lab j.a.etzel@med.umcg.nl

2. Mass univariate (spm) Test every voxel separately Fit a linear model to each voxel Look for brain structures where the blobs occur Obtain a p-value for each voxel Use parametric (or permutation) statistics to evaluate significance Multivariate Test groups of voxels (ROIs) at once Use machine learning algorithms Structures first (no blobs) Obtain a classification accuracy for each ROI Evaluate significance of accuracy by permutation testing and perhaps subsetting (parametric statistics also possible)

3. Classification yes no yes from www.cac.science.ru.nl/people/ustun/index.html What sort of flower is this? Is this example type 1 or type 2?

4. from http://spect.yale.edu/displaying_results.html listening to hand action soundslistening to mouth action sounds Can the activity in the right premotor cortex classify the volumes into hand and mouth action sounds? - or - Is the brain activity in this ROI different while listening to hand and mouth action sounds?

5. subjectstimulusexamplevoxel 1voxel 2 1mouth10.8464-0.8932 1mouth20.57760.0494 1hand11.49550.2002 1hand2-0.3844-0.3322 2mouth10.5432-0.9855 www.cs.sunysb.edu/~mueller/research/brainMiner/ 0.3543-0.17342.34440.3832-0.98440.98640.0033 for each ROI in each volume (or summary volume) temporal compression; extract GLM parameter estimates to classifiers

6. subjectstimulusexamplevoxel 1voxel 2 1mouth10.8464-0.8932 1mouth20.57760.0494 1hand11.49550.2002 1hand2-0.3844-0.3322 1hand30.5432-0.9855 subjectstimulusexamplevoxel 1voxel 2 1mouth10.8464-0.8932 1mouth20.57760.0494 1hand11.49550.2002 1hand2-0.3322.1134 subjectstimulusexamplevoxel 1voxel 2 1hand51.49550.2002 1hand6-0.3844-0.3322 1mouth30.5432-0.9855 1hand42.31451.3342 testing set training set classifier accuracy (from test set) 1 st train: mouth vs. hand? 2 nd test: mouth vs. hand? classify each subject separately, average accuracy across subjects  subjects can have their own patterns classify subjects together, test left-out subject  subjects need the same patterns all data

7. ROIs * P < 0. 0042, permutation test (Bonferroni correction) of 0.05 for 12 ROIs. So, have which ROIs allowed significant classification accuracy for separating mouth and hand action sounds: preML, preMR, M1L, S2R, audL, audR.

8. Permutation Test What? null hypothesis is arbitrary labeling  If labeling is arbitrary (random), then there is no relationship between the labels and the activity, so classification should not be possible. real data classified about the same as arbitrary data: no evidence for a relationship between the labels and the activity: not significant. real data is classified better than the arbitrary data: it is “unusual:” significant. How? classify the real data make many randomized-label data sets (arbitrary or “fake” data). classify each fake data set in the same way as the real data proportion of fake data sets classified better than the real data is the p-value

9. 1. Analyze the real data (get accuracy). M1 L = 0.6000 2. Make lots of permuted-label data sets (all if possible, at least 1,000). 0.4944 0.510 0.499 0.5211 0.480 0.5002 0.498 0.519 0.5720 0.4789 … 3. Analyze the fake data sets (get accuracies). 4. Count how many of the fake data sets were classified more accurately than the real data. Of 1000 fake data sets, none had accuracy higher than 0.60. 5. Divide and get the p-value. 1/1001 = 0.000999, so M1 L p = 0.001 (50/1001 = 0.0499) Done!

10. Permutation test results, showing true results with uncorrected p-value cutoff lines. True mean accuracy lines that fall above the range of the permuted lines are highly significant (p = 0.002).

11. Black box? Random Forests (RFs) first fMRI use, I think algorithm makes 10,000 CARTs, all try to classify, majority vote final class algorithm makes random variable selections for the “forest” of “trees” support vector machines (svms) previously used with fMRI data algorithm converts data to a higher dimension to try to find a linear separating line (hyperplane) between the classes. classifier yes no yes no yes no yes

12. k-nearest neighbor Linear discriminant analysis Gaussian Naïve Bayes Hidden Markov Models Partial Least Squares For more info/review:  Mitchell, Machine Learning, 2004, 145-175;  O’toole, Journal of Cognitive Neuroscience, 2007, 1735-1753. Other classification-type algorithms classifier Neural Networks A generalization of linear regression functions; many variations. Each node calculates a weighted sum, and does a binary input to the next layer, so simple networks can be expressed as (long) equations. Training the network for each person involves setting the weights on each voxel.

13. Idea: brain activity pattern during remembering an item should be similar to the pattern when learning the item. Subjects learned 3 lists of 10 items each; items were labeled pictures of famous people, famous locations, and everyday objects. After learning, subjects tried to remember all of the items; saying them aloud as they remembered them.

14. 2: testing volumes when recalling 1: training volumes when learning items in each category classifier (neural network) 3: accuracy (sort of) “For each brain scan, the classifier produced an estimate of the match between the current testing pattern and each of the three study contexts.”

15. “ However, follow-up analyses indicated that voxels outside of peak category-selective areas are also important for establishing this result.”

16. Take-Home Message Classification (multivariate methods) can answer questions and find patterns not available with spm-type methods.  spm is still useful, though! Method is (for me) ROI-focused: start with hypotheses about which ROIs can classify (or not) which stimuli. Differences in activation to stimuli can be restated as classification predictions.

17. That’s It!

18. Permutation vs. t-test p-values The permutation and t-test p-values are very highly correlated, though not always identical. Calculated correlation line in solid, perfect in dotted.