1. Neuronal basis of natural textures coding in area V4 of the awake monkey: texture analysis P.Girard, C. Jouffrais, F. Arcizet, J. Bullier Insight2+ IST–2000-29688 3D shape and material properties for recognition

2. Aim of the study (WP3) Coding of material properties In area V4 of awake macaque monkey Performing a visual fixation task Stimuli from the CURET database: 12 textures + 12 scrambled textures Frontal viewing direction 3 illumination directions (22.5, 45 and 67.6 deg.)  72 stimuli

3. Stimuli Terrycloth Sand paper PlasterAluminum foil Salt crystalsRoof shinglePlaster (zoom) Lettuce leaf Linen ConcreteWhite bread Soleirolia plant

4. Experimental setup Control of the experiment and real time analog and digital acquisition: CORTEX (courtesy of NIH) 5 independent microelectrodes (TREC) Sorting software: MSD (Alpha-Omega) Eye monitoring: IScan eye-tracker (120 Hz, 0.2 DVA)

5. Protocol Mapping of the Receptive Field (RF) Hand-moved bars M-sequences of black and white dots Recording of response to the 72 stimuli (10 trials per stimulus) Control: 36 original textures moved 1 deg apart

6. Recording sites.

7. Database and statistics Database: 167 cells (42 with unshaped stimuli, 98 with shaped stimuli, 27 with new set of textures) Statistics ANOVA 3-factors (Texture, Illum. Dir., Type) Population (Rank analysis, MDS, comparison V4/IT)

8. V4 neuron sharply selective to textures Plaster (zoom)Lettuce leaf 0.5s 100 0 Spikes/s OnOff 22.5 deg. 45 deg. 67.6 deg.

9. ]]]]] ] Texture neuron selective to illumination direction Example of a V4 cell whose discharge is systematically increased for a lighting direction of 67.6 deg.

10. V4 neuron selective to original and “moved” textures Example of a V4 cell whose selectivity is the same for ‘original’ and ‘moved’ conditions. No response to scrambled sitmuli.

11. Statistics 3 factors ANOVA (main effect + interaction, P<0.05) shows that: 82% of the cells are selective to textures 69% of the cells have a different response to original and random-phase textures 69% of the cells are selective to lighting direction

12. 82% selective to texture

13. Multidimensional Scaling (MDS) – originals Dimension 2 MDS analysis performed on 68 cells. Original textures only, final configuration, 3 dimensions (Alienation:0.108, Stress: 0.099).

14. Correlations of neuronal responses with first,second,third and fourth order parameters Median luminance Rms contrast skewness kurtosis

15. Texture analysis Is there a match between V4 cell population and a set of filters that could be used to classify the textures? Are there other interesting parameters that characterize the textures and are coded in V4?

16. Texture analysis: methodology Sets of 2D GABOR filters (several sizes, spatial frequencies and 8 orientations (0°:22.5:157.5°) 3 different types of quantification of outputs - thresholds -energy -Spectral histograms

17. Example of filter and computations (thresholds) Size= 12 pixels, freq: 9.5 c/°, sigma 4 pixels, orientation 0 Size= 12 pixels, freq: 14 c/°, sigma 4 pixels, orientation 0

18. Example of cluster analysis with filters and neurons

19. Example of filter and computations (energy)

20. Cluster analysis based upon energy N=56 filters: Size 12 pixels, freq: 2 to 28 c/°, sigma 3 pixels, orientations 0:22.5:157.5°

21. MDS based upon energy

22. Spectral Histogram N=29

23. Spectral Histogram vs ENERGY energy Spectral histogram

24. MDS over different epochs after the stimulus onset filters: Size 12 pixels, freq: 2 to 28 c/°, sigma 3 pixels, orientations 0:22.5:157.5°

25. MDS with images (filters/neurons)

26. New textures

27. SNR is an important parameter Mean2/std2 (of image, not of filtered image)

28. Snr : 1 possible dimension N=27 filters: Size 12 pixels, freq: 2 to 28 c/°, sigma 3 pixels, orientations 0:22.5:157.5°

29. SNR another example

30. Mds with images of the textures

31. Luminance?

32. Conclusions Coding of material properties in V4 and IT Is this indeed texture classification or identification? We need expert advice to use better texture characterization (Spatial frequency…) or classification (Varma and Zisserman, Geusebroek and Smeulders) Do neurons perform such expert classification? Need to use a comparable behavioural task?

33. Not shown