A model for detecting illusory contours Caveat: Response decreases gradually with increasing intermeshing of gratings. This does not correspond with subjective percepts. (Peterhans and von der Heydt, 1989)

Convergence of cues in contour coding (Studies with Tilt AfterEffect) Adapting stimulus Test stimulus TAE is orientation specific and exhibits inter-ocular transfer. Therefore, TAE is likely to be of cortical origin. Does the TAE transfer from one kind of contour (say ‘real’) to another (say ‘illusory’, or ‘motion defined’)? Yes. (Berkley, DeBruyn & Orban, 1994; Paradiso, Shimojo & Nakayama, 1989)

Convergence of cues in contour coding

From zero-crossings to a raw primal sketch (collection of tokens) Zero crossings at a coarse scale Zero crossings at a fine scale ‘Blobs’ Regions where Zero crossings Form closed contours ‘Edge segments’ Regions of coincidence of ZCs across scales

From a raw primal sketch to a full primal sketch Requires contour linking and extracting globally salient structures

What assumptions does the visual system use to link Edge fragments and identify ‘important contours’? 1.Collinearity (aligned fragments typically belong to the same contour) 2.Length (longer contours are more salient) 3.Low curvature 4.Completion (closed contours are more salient) 5.Object recognizability (controversial) Ideas from psychophysical studies

Computing saliency How to assign a saliency measure to each edge fragment? S (p) =  w   ii i  i = 1 if element present = 0 otherwise w = e i -c i Ci is the total curvature of the contour from p upto the ith element    p i Wi is 1 for a straight line and decreases as the curvature increases -penalizes high curvature -Weights distant elements less than closer ones [Ullman and Shashua, 1988]

Final saliency at p S(p) = max S (p)  Very high computational requirements, but can be implemented Highly efficiently in a locally connected network. [Ullman, 1995]    p i 

Computing saliency – some results

Psychophysical and physiological correlates Of saliency computations -Fields et al -Polat and Sagi Facilitation of detection by aligned neighboring elements Gilbert and Wiesel found long range connections between similarly tuned hypercolumns that might subserve the perceptual facilitation.

From a raw primal sketch to a full primal sketch Requires segregating regions with different textures

Moving beyond the first stages of image-processing… Information about object shape and surface properties

Open questions about early processes Do texture segregation processes work on the 2D image or at the level of Surface representation?

Processing Framework Proposed by Marr Recognition Shape From stereo Shape From stereo Motion flow Shape From motion Color estimation Shape From contour Shape From shading Shape From texture 3D structure; motion characteristics; surface properties Edge extraction Image Emphasis on ‘Bottom-up’ processing

Stereopsis Challenges: 1. Trigonometric calculations 2. Correspondence problem The process of perceiving the relative distance to objects based on their Lateral displacement in the two retinal images (aka binocular disparity). Other potential cues to depth (convergence and accommodation) do not seem to be too Important for humans.

The Stereo Correspondence Problem “During binocular regard of an objective image, each uniocular Mechanism develops independently a sensual image of considerable Completeness. The singleness of binocular perception results from The union of these elaborated uniocular sensations. The singleness id Therefore the product of a synthesis that works with already elaborated Sensations contemporaneously proceeding.” - Sherrington, 1906

The Stereo Correspondence Problem Hand

However, different looking images could also be fused… “Although a perfect stranger to you, and living on the reverse side of the globe, I have taken the liberty of writing to you on a small discovery I have made in Binocular vision in the stereoscope. I find by taking two ordinary photos of two Different persons’ faces, the portraits being about the same sizes, and looking About the same direction, and placing them in a stereoscope, the faces blend into One in a most remarkable manner, producing in the case of some ladies’ Portraits, in every instance, a decided improvement in beauty.” - From a letter to Charles Darwin by A. L. Austin of New Zealand

Is monocular shape analysis a necessary pre-requisite for stereo correspondence?

Open questions: 1. Does RDS stereopsis conclusively prove that the shape-first theory is incorrect? 2. Are RDSs representative of real-world scenes? 3. Are RDSs completely devoid of monocular shape cues?