Neuronal basis of natural textures coding in area V4 of the awake monkey: texture analysis P.Girard, C. Jouffrais, F. Arcizet, J. Bullier Insight2+ IST– D shape and material properties for recognition

Aim of the study (WP3) Coding of material properties In area V4 of awake macaque monkey Performing a visual fixation task Stimuli from the CURET database: 12 textures + 12 scrambled textures Frontal viewing direction 3 illumination directions (22.5, 45 and 67.6 deg.)  72 stimuli

Stimuli Terrycloth Sand paper PlasterAluminum foil Salt crystalsRoof shinglePlaster (zoom) Lettuce leaf Linen ConcreteWhite bread Soleirolia plant

Experimental setup Control of the experiment and real time analog and digital acquisition: CORTEX (courtesy of NIH) 5 independent microelectrodes (TREC) Sorting software: MSD (Alpha-Omega) Eye monitoring: IScan eye-tracker (120 Hz, 0.2 DVA)

Protocol Mapping of the Receptive Field (RF) Hand-moved bars M-sequences of black and white dots Recording of response to the 72 stimuli (10 trials per stimulus) Control: 36 original textures moved 1 deg apart

Recording sites.

Database and statistics Database: 167 cells (42 with unshaped stimuli, 98 with shaped stimuli, 27 with new set of textures) Statistics ANOVA 3-factors (Texture, Illum. Dir., Type) Population (Rank analysis, MDS, comparison V4/IT)

V4 neuron sharply selective to textures Plaster (zoom)Lettuce leaf 0.5s Spikes/s OnOff 22.5 deg. 45 deg deg.

]]]]] ] Texture neuron selective to illumination direction Example of a V4 cell whose discharge is systematically increased for a lighting direction of 67.6 deg.

V4 neuron selective to original and “moved” textures Example of a V4 cell whose selectivity is the same for ‘original’ and ‘moved’ conditions. No response to scrambled sitmuli.

Statistics 3 factors ANOVA (main effect + interaction, P<0.05) shows that: 82% of the cells are selective to textures 69% of the cells have a different response to original and random-phase textures 69% of the cells are selective to lighting direction

82% selective to texture

Multidimensional Scaling (MDS) – originals Dimension 2 MDS analysis performed on 68 cells. Original textures only, final configuration, 3 dimensions (Alienation:0.108, Stress: 0.099).

Correlations of neuronal responses with first,second,third and fourth order parameters Median luminance Rms contrast skewness kurtosis

Texture analysis Is there a match between V4 cell population and a set of filters that could be used to classify the textures? Are there other interesting parameters that characterize the textures and are coded in V4?

Texture analysis: methodology Sets of 2D GABOR filters (several sizes, spatial frequencies and 8 orientations (0°:22.5:157.5°) 3 different types of quantification of outputs - thresholds -energy -Spectral histograms

Example of filter and computations (thresholds) Size= 12 pixels, freq: 9.5 c/°, sigma 4 pixels, orientation 0 Size= 12 pixels, freq: 14 c/°, sigma 4 pixels, orientation 0

Example of cluster analysis with filters and neurons

Example of filter and computations (energy)

Cluster analysis based upon energy N=56 filters: Size 12 pixels, freq: 2 to 28 c/°, sigma 3 pixels, orientations 0:22.5:157.5°

MDS based upon energy

Spectral Histogram N=29

Spectral Histogram vs ENERGY energy Spectral histogram

MDS over different epochs after the stimulus onset filters: Size 12 pixels, freq: 2 to 28 c/°, sigma 3 pixels, orientations 0:22.5:157.5°

MDS with images (filters/neurons)

New textures

SNR is an important parameter Mean2/std2 (of image, not of filtered image)

Snr : 1 possible dimension N=27 filters: Size 12 pixels, freq: 2 to 28 c/°, sigma 3 pixels, orientations 0:22.5:157.5°

SNR another example

Mds with images of the textures

Luminance?

Conclusions Coding of material properties in V4 and IT Is this indeed texture classification or identification? We need expert advice to use better texture characterization (Spatial frequency…) or classification (Varma and Zisserman, Geusebroek and Smeulders) Do neurons perform such expert classification? Need to use a comparable behavioural task?

Not shown