What do object-sensitive regions show tuning to? fMRI Object classes Haxby et al. (2001) Grey-scale photographs; MVPA Schwarzlose, Swisher, Dang & Kanwisher (2008) Grey scale photographs; MVPA [tuning to class & location] Shape Grill-Spector et al (1999) Grey-scale photographs; adaptation Kim, Biederman, Lescroart & Hayworth (2009) Line drawings of basic level objects; adaptation Drucker & Aguirre (2009) Synthetic 2d shapes; adaptation & MVPA Op de Beeck, Torfs & Wagemans (2008) Synthetic 3d shapes, MVPA Haushofer et al. (2008a,b) Synthetic 2d shapes, adaptation/MVPA

What do object-sensitive regions show tuning to? Electrophysiology Sparse response with tuning to a narrow range of stimuli Monkeys - Rolls & Tovee (1995), Humans - Quiroga et al. (2005) Waydo et al. (2006) [medial temporal] – 0.2-1% of stimuli Computational modelling Ullman, Vidal-Naquet & Sali (2002) Sparse response tuned to object parts Parts chosen for each class to be selective but invariant to within-class variation

Investigate tuning to many feature dimensions simultaneously Aims Investigate tuning to many feature dimensions simultaneously New real-time MVPA imaging method Allow for tuning to parts of real objects Use rich sets of (colour) photographic stimuli Kriegeskorte et al. (2008); Tyler & colleagues [Data from Naci] No need to… Assume pattern of selectivity vs. invariance across features is same for all objects Pre-specify a set of conditions Region-of-interest LOC & posterior fusiform From localiser scan (objects-scrambled objects, Naci PhD)

Dynamically Adaptive Imaging (DAI) Stimulus delivery Modify stimuli Real time analysis Pre-processing MVPA Scanner console

DAI similarity search Similarity search Kriegeskorte (2008) Choose a referent object Find the “neural neighbourhood” of items that evoke a similar pattern of response Kriegeskorte (2008) Iterative procedure Present the items for this generation (initially, all of them) Use real-time MVPA to compare the similarity of the pattern evoked by each object to that evoked by the referent Choose the best items and repeat Details Generation sizes 91-24-20-16-13 Referent is repeated every 7 items MVPA using correlation across voxels on beta values. Each object modelled separately.

Bonferroni corrected across features (***) Bonferroni corrected across features

(**)

Quantifying relationship between features and neural neighbourhood Train LDA classifier to predict from their features which objects will be in the neural neighbourhood Leave-one-out across subjects Classification performance good Tuning to different features defines the neural neighbourhoods of different referents

Assess number of features while remaining agnostic about what they are Which features? Assess number of features while remaining agnostic about what they are

Dissimilarity search Reverse selection rule for search Number of features encoded… …small, easy to find opposite …high, dissimilarity search will not converge because: (1) Many ways for something to be dissimilar (2) Exact opposite may not be in set (or even exist) Feature space is not very low dimensional But can we quantify the dimensionality?

Assessing dimensionality Two or three DAI sessions in each subject First generations of each all contain the same set of 92 objects Use PCA to assess dimensionality of the response

Assessing dimensionality Do dimensions reflect neural features, or might some reflect artefacts in the data? Test by quantifying information content of pattern as a function of the number of components using MVPA In each subject: (1) Pick an object, train classifier to recognise activity pattern using all but one session (2) Test on the remaining session Repeat for all objects and subjects

Summary: Object processing Object-selective cortex reflects both perceptual and semantic features Pattern of tuning versus invariance to these features is different in the neighbourhood of different referents Estimate of at least 25 distinct feature dimensions Summary: Methodological results Real-time MVPA with DAI Similarity Search can effectively search complex stimulus spaces Neural response evoked by a single presentation can be reliably classified Can classify single objects from a set of 92 Thanks to… Michele Veldsman, Daniel Mitchell, Annika Linke & other lab. members Lorina Naci, Niko Kriegeskorte & the CSL; the CBU radiographers rhodri.cusack@mrc-cbu.cam.ac.uk