Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 1 Computational Architectures in Biological Vision, USC, Spring 2001 Lecture 11: Visual Illusions. Reading Assignments: None

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 2 What Can Illusions Teach Us? They exacerbate the failure modes of our visual system, and show artifacts/byproducts of normal visual processing. Hence, they can help us constrain computational models of visual processing. How relevant is this to computer vision? It can be if we believe that the closer to biological systems our algorithm will be, the best it will perform in real-life situations.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 3 The Story so Far Intricate hierarchy of visual areas. Neurons respond to increasingly complex stimuli as we ascent the hierarchy. Interactions are: - feedforward - intra-area - feedback and non-linear.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 4 Early Processing Center-surround and oriented filters. Columnar organization and topographic maps.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 5 Columns

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 6

7 Non-Linear Interactions Feedforward model does not explain many properties of early visual processing. Some of the non-linear interactions are well studied.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 8

9 Complex Non-linear Interactions

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 10 Higher-Level Vision Similar to lower-level vision, we can derive several general principles for higher-level processing.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 11 Some Dedicated Circuitry e.g., looming neurons in insects directly compute time-to-contact from a wide array of simple inputs.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 12 Convergence of Cues A given aspect of vision (e.g., depth perception) rarely relies on only one set of cues (e.g., stereo disparity). Rather, evidence is accumulated about the visual world in several parallel processing streams, which partially overlap in their selectivity. E.g., distance to an object can also be estimated from motion parallax, vergence, shape from shading, size constancy, occlusions, etc. In normal vision, all these mechanisms contribute to our percept.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 13 Gestalt Psychology Founded by Max Wertheimer in the early 1900s as a revolt against Wundt’s “molecular” program for psychology. Gestalt = “unified” or “meaningful whole” Basic observation: often our experience is richer than our simple sensations. E.g., rather than perceiving a succession of static frames, we see motion in a movie. So, a rapid sequence of elementary static sensory events yields a different experience altogether, that of motion. Gestalt Psychologists call this the “phi phenomenon.”

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 14 Gestalt Psychology A set of basic “laws” was empirically defined. At the basis, the law of “Pragnanz” stipulates that not only are we built to experience the whole rather than the multiple individual elementary stimuli, but we are also naturally inclined to do so. See letter “B” rather than collection of curve segments?

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 15 Gestalt Laws Law of closure: if something missing from otherwise complete figure, our percept will consider that the missing part is present. Law of similarity: group similar items together. XOOOOO OXOOOO OOXOOO OOOXOO OOOOXO OOOOOX

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 16 Gestalt Laws Law of proximity: things close together belong together. ********** **********3 groups of 10 stars or 10 groups of 3? ********** Law of symmetry: we tend to group parts into objects according to symmetry. [ ][ ][ ] (here, despite proximity)

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 17 Gestalt Laws Law of continuity: a partially occluded line appears to continue behind the occluder (rather than perceiving 2 line segments ending at the occluder) Figure-ground: we usually perceive as figure the smaller part of a B/W image.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 18 Gestalt Psychology Is not restricted to perception, but also proposes theories for memory, dreams, etc. Nowadays, Gestalt Theory has lost much of its popularity mainly because of its failure to explain (rather than simply observe) the peculiarities of perception. Nevertheless, its basic laws and principles remain true and are omnipresent in perception.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 19 Now, on to the Illusions! The following web site showcase the illusions described in the (web- based) rest of the lecture:

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 20 Illusions - low-level illusions: often based on the fact that we tend to underestimate acute angles and overestimate obtuse angles, to perceive nearly-right angles as being right, etc. - constancy illusions: based on the tendency that, in the absence of low- level cues (e.g., a 3D scene seen on a 2D screen), higher-level cues become dominant. - aftereffects: if we “burn in” (i.e., adapt to) a given image and then remove it, a “reverse” percept occurs. This makes sense in terms of selectively adapting a subpopulation of neurons. - impossible figures: built on consistent local cues but organized as an inconsistent whole.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 21 Illusions We will see that several simple illusions can be explained as ambiguities in basic computational problems which we studied: e.g., constancy illusions & color/contrast constancy mach bands&contrast gain control barber pole&aperture problem but many remain a mystery!