1. Learning to Decode Cognitive States from Brain Images Tom Mitchell et al. Carnegie Mellon University Presented by Bob Stark

2. Outline ● Experiment Setup ● Machine learning ● 3 case studies ● Results ● Analysis

3. Experiment Setup ● Functional Magnetic Resonance Imaging (fMRI) used – Regions of Interest (ROIs) ● fMRI scans taken constantly, once per second for 20 minutes: – each image =~ 15,000 voxels – tens of millions of observations total – each voxel a machine learning feature

4. Experiment Setup (2)

5. Machine Learning ● Find f: fMRI-sequence(t 1, t 2 ) -> CognitiveState – Supervised – Cross-validation ● Gaussian Naive Bayes (GNB) ● Support Vector Machine (SVM) ● k Nearest Neighbor (kNN)

6. Gaussian Naive Bayes ● Estimate distribution of P(c|x) ● P(x|c) := Gaussian ● P(c) := Bernoulli ● Naive? ● SharedVariance, DistinctVariance ● Parameters estimated with MLE

7. Super Vector Machine ● n-1-dimensional hyperplane classifier ● Maximum-margin hyperplane ● Linear kernel SVM

8. k Nearest Neighbor ● A new sample is classified based on its nearest “neighbor” in the training set ● Used Euclidean distance (nearest brain location) ● Most frequent class among k training samples – k := {1,3,5,7,9}

9. 3 Case Studies ● Picture vs. Sentence ● Syntactic Ambiguity ● Semantic Categories

10. Picture vs. Sentence ● “Does this sentence correctly describe this picture?” ● Classify (differentiate between) examining picture and sentence ● Picture (or sentence) 4 seconds then blank 4 seconds = 8 ● f: fMRI-sequence(t 0, t 0 + 8) -> {Picture, Sentence}

11. Syntactic Ambiguity ● “The experienced soldiers warned about the dangers conducted the midnight raid” ● “The experienced soldiers spoke about the dangers before the midnight raid” ● f: fMRI-sequence(t 0 + 4.5, t 0 + 15) -> {Ambiguous, Unambiguous}

12. Semantic Categories ● Quickly presented with categories, then words and asked to vote to tell if it was in one of the categories or not – Fish, four-legged animals, trees, flowers, fruits, vegetables, family members, occupations, tools, kitchen items, dwellings, building parts ● Classify the category from one fMRI image ● f: fMRI(t) -> WordCategory – Notice: no interval ● Image on slide 4

13. Results ● Listed: cross-validation classification error ● GNB & SVM > kNN ● kNN is best with larger k

14. Analysis ● It's feasible! ● But only for specific cognitive states from specific tasks ● Not true/false, correct/incorrect, negative/positive! – Maybe more training data? ● Most uses or types = open question ● Single-subject classifiers were used – Implications of multi-subject classifiers?

15. References ● "Learning to Decode Cognitive States from Brain Images," T.M. Mitchell, R. Hutchinson, R.S. Niculescu, F.Pereira, X. Wang, M. Just, and S. Newman, Machine Learning, Vol. 57, Issue 1-2, pp. 145-175. October 2004. ● http://en.wikipedia.org/wiki/Support_vector_machine