Grenoble Images Parole Signal Automatique Modeling of visual cortical processing to estimate binocular disparity Introduction - The objective is to estimate binocular disparity through a physiologically consistent model on the basis of actual knowledge about 3D perception. Basis : Energy model A physiologically relevant model 1 Linear combination of Gabor filter outputs acting as cortical simple cells: Frequency and orientation tuned Final non linear output to simulate binocular complex cells One preferred frequency and disparity per cell ; Filtering of stereoscopic images through a complete distribution of cells with different combinations of frequencies and disparities. Contribution : - Retina model 9 Non linear contrast adaptation, spectral whitening - Size/Disparity correlation 2 Smaller disparity range for finer scales - Non uniform distribution of disparity tuned cells within scales 3,4 Better acuity for smaller disparities and for finer scales - “Coarse to Fine” process 5,6 Coarse scales constrain depth direction estimation of finer scales - Intra/Inter scales disparity interactions Energy normalization of different scales Excitatory and inhibitory influence in function of disparity difference with the preferred disparity of a “cell” 7 Fine to coarse: Excitatory influence of the closest disparity of a finer scale 8  Simulation results and conclusion MAGGIA Christophe, GUYADER Nathalie, GUERIN DUGUE Anne Schematic representation of the model architecture. The stereoscopic image is processed by retina model then by disparity energy model at the coarsest scale. Spatial pooling and orientation averaging are used as in the literature. Then, interaction within and between scales are applied to the response of each disparity cell. Estimation use winner takes all. According to the disparity estimated, process stops or continue with finer scales. (log) References : 1-Ohzawa I. Mechanisms of stereoscopic vision: the disparity energy model. Curr. Opin. Neurobiol. 1998;8(4): Smallman HS, MacLeod DI. Size-disparity correlation in stereopsis at contrast threshold. J Opt Soc Am A Opt Image Sci Vis. 1994;11(8): Farell B, Li S, McKee SP. Coarse scales, fine scales, and their interactions in stereo vision. J Vis. 2004;4(6): Schor CM, Wood I. Disparity range for local stereopsis as a function of luminance spatial frequency. Vision Res. 1983;23(12): Marr D, Poggio T. A computational theory of human stereo vision. Proc. R. Soc. Lond., B, Biol. Sci. 1979;204(1156): Chen Y, Qian N. A coarse-to-fine disparity energy model with both phase-shift and position-shift receptive field mechanisms. Neural Comput. 2004;16(8): Julesz B., Chang JJ.,Interaction between pools of binocular disparity detectors tuned to different disparities. Biol.Cybernetics. 1976; 22: Smallman HS., Fine-to-coarse scale disambiguation in stereopsis. Vision Res ; 23(8): Hérault J. Retinal sampling. Intern report, Gipsa-Lab, 2009 Natural image in gray level took with a stereo camera superimposed with its computed depth map ( disparity color scale in pixel) Random Dot Stereogram computed to display a square in depth with a disparity of 10 pixels and its computed depth map (disparity color scale in pixel) - Interactions within scales stabilize disparity estimation and reduce false matches. - Disparity estimation depends on scales and disparity cell distribution as in perception. - Perception of disparity frontiers does not depend on scales or disparity magnitude. - The interaction fine to coarse are not optimized yet to refine disparity edges for high disparity magnitude. - Efficiency of interactions are highly dependent on weight coefficient, optimization is needed. - Low texture surface are not borne by the model. Parvocellular pathway, global high pass filter Spectral whitening, non linear contrast adaptation 9 Disparity Cell organized according treshold value found in literature 3,4 Literature shows that close disparity level stimulate each other whereas farther disparityhave an inhibitory effect 7