What functional brain imaging reveals about the neuroarchitecture of object knowledge Kai-min Kevin Chang, Vicente Malave, Svetlana Shinkareva, and Marcel Adam Just Center for Cognitive Brain Imaging, Carnegie Mellon University CategoriesExemplars body parts leg, arm, eye, foot, hand furniture chair, table, bed, desk, dresser vehicles car, airplane, train, truck, bicycle animals horse, dog, bear, cow, cat kitchen utensils glass, knife, bottle, cup, spoon tools chisel, hammer, screwdriver, pliers, saw buildings apartment, barn, house, church, igloo part of building window, door, chimney, closet, arch clothing coat, dress, shirt, skirt, pants insects fly, ant, bee, butterfly, beetle vegetables lettuce, tomato, carrot, corn, celery man made objects refrigerator, key, telephone, watch, bell refrigerator celery ant X X 3s 7s time Overview We used functional Magnetic Resonance Imaging to study the cortical systems that underpin semantic representation of object knowledge. Method Participant was presented with black and white line drawings of 60 objects from a range of categories (see below) and were instructed to think of the same properties consistently during each presentation. fMRI procedure: Cerebral activation was measured using BOLD contrast (Kwong et al., 1992; Ogawa et al., 1990). Functional images were acquired on a Siemens Allegra 3.0T scanner at the Brain Imaging Research Center. EPI acquisition sequence, 17 oblique axial slices, TR=1000ms, TE=30ms, 64x64 acquisition matrix, 5 mm thickness, 1 mm gap, flip angle 60°. Image were corrected for slice acquisition timing, and motion-corrected with Statistical Parametric Mapping software (SPM99, Wellcome Department of Cognitive Neurology, London, UK). Data were normalized to the Montreal Neurological Institute (MNI) template, and resampled to 3x3x6 mm voxels for Machine Learning analysis. Analyses of a single brain region at a time used region definitions derived from the Anatomical Automatic Labeling (AAL) system (Tzourio-Mazoyer et al., 2002). Machine Learning Evaluation Classification results were evaluated using k-fold cross validation, where one example per class was left out for each fold. Both raw and rank accuracies are reported. 4CAPS Modeling We implemented a set of neural processing regions in the 4CAPS neural architecture (Just & Varma, in press), along with their corresponding specialization to process each feature. The 4CAPS model was used to simulate human performing the picture-naming task. Theoretical Each object is associated with a set of distinguishing sensory/functional features (e.g. visual-motion, visual-parts, function, etc.) according to Cree & McRae (2003)'s semantic feature norming studies. The sensory/functional features are thought to be processed by a set of neural processing regions that are mostly agreed by the current state of the literature. Correlation = Data-driven Can we instead learn the set of activated neural processing centers from our data? Yes, but we need to check if it is generailzable, otherwise we may overfit to the data. Evaluation The activation pattern of the neural processing regions in 4CAPS is correlated to human's cortical activation pattern reading the same word. Mean PSC matrix words x voxels Preprocessed fMRI data time x voxels Construct mean Percent Signal Change (PSC) matrix per stimulus event averaged over 4 successive images Full training set words training x voxels Full test set words test x voxels Training set words training x voxels select Test set words test x voxels select Partition data into training and test sets Select Voxels (features) Evaluate classification Subset of voxels selected from Training set Train classifier to identify categories based on selected features Get classification accuracy Alternative Feature (voxel) Selection Methods Select individually most discriminating voxels: Threshold Number of Misclassifications select voxels with the minimum number of misclassifications made by the best threshold chosen for each voxel (Ben-Dor et al., 2000) Wilcoxon select voxels with highest statistic value Select conjointly most discriminating voxels: Logistic regression select voxels with highest absolute value of the regression weights Select most stable voxels : Stability of differential response for words select voxels with the largest average pairwise correlations across the word presentation 2 Alternative Classifiers: Gaussian Naïve Bayes Uses Bayes rule to estimate the probability distribution from the training set. Makes conditional independence assumption of the features. Logistic Regression Uses parametric form to directly fit the distribution P(Y|X). Classification performance is evaluated using k-fold cross validation Conclusion We have shown that high classification accuracies can be obtained for distinguishing the 12 categories of objects and even the 60 exemplars. Furthermore, the neural processing regions associated with the sensory/functional features are modeled in 4CAPS. Simulation of 4CAPS shows comparable activation pattern to human. ExperimentCorrelations Across trials Across words Across studies on all categories on tools and dwellings Across subjects N/A ExperimentChanceRawRank Category (12-way) 8.3%52%88% Exemplar (60-way) 1.6%37%91% Machine Learning Flowchart Correlation Between Neural and 4CAPS Activation Vector 4CAPS Modeling Hierarchy