Neurodynamics of figure-ground organization Dražen Domijan University of Rijeka, Rijeka, Croatia 8th Alps-Adria Psychology Conference LJubljana, October 2-4, 2008

Figure-ground assignment Complex process that could be analysed at different levels: Psychological (Phenomenological) level Neurophysiological level Computational level

Psychological level The Rubin vase/face stimulus When the black center region is seen as the figure, it appears to have a definite shape, and the object it portrays, a vase, can be recognized The adjacent white region, appears locally shapeless

Psychological level Figure-ground organization is an important step in visual processing which separate structured input to which processing efforts should be devoted (figure) from less structured background Gestalt psychologists identified several variables which influence figure-ground assignment (Palmer, 1999)

Gestalt principles of figure - ground organization (Palmer, 1999) Surroundedness Size Contrast Convexity Horizontal-vertical orientation Symmetry Parallelism

New cues to figure-ground assignment Spatial frequency (Klymenko & Weisstein, 1986) Lower region (Vecera et al., 2002) Top-bottom polarity (Hullemann & Humphreys, 2004) Extremal edges (Palmer & Ghose, 2008)

Spatial frequency Higher spatial frequencies appear to be a figure (Klymenko & Weisstein, 1986)

Lower region Lower region refers to the tendency to assign figure to the surface in the lower part of the visual field (Vecera et al., 2002)

Top-bottom polarity Top-bottom polarity refers to the tendency to assign figure to the surface with wide base and narrow top (Hullemann & Humphreys, 2004)

Extremal edges Luminance or texture gradients produced by smooth convex surfaces. Strong cue which overrides classical cues such as size, or surroundedness (Palmer & Ghose, 2008)

Ecological validity of Gestalt principles Fowlkes et al. (2007) Size, convexity and lower region in real digital images

The role of top-down processes Knowledge or object recognition (Peterson, 1994, 1999) Endogenous orienting of attention (Baylis & Driver, 1996) Exogenous orienting of attention (Vecera et al., 2004)

Knowledge or object recognition (Peterson, 1994, 1999)

Exogenous orienting of attention (Vecera et al., 2004)

Neurophysiological level Neurons with oriented receptive fields (simple, complex and hypercomplex) found in primary visual cortex (Hubel & Wiesel, 1962; 1965; 1968) Interior enhancement in V1 (Lamme, 1995; Zipser et al., 1996; Lamme et al., 1996) Border ownership in V2 (Zhou et al., 2000; Qiu & von der Heydt, 2005)

Neurophysiological investigations Hubel & Wiesel (1962, 1965, 1968) Neuronal orientation and directional selectivity Neuronal receptive field is indicated by broken rectangle

Interior enhancement Lamme (1995), Zipser et al. (1996), Lamme et al. (1996) Figure is coded with enhanced firing rate for neurons in the interior of figural surface Slow dynamics – maximal modulation at about 150 ms

Border ownership Zhou et al. (2000), Qiu & von der Heydt (2005) Figure is coded with enhanced firing rate for one side of the border Fast dynamics – response ocurs at ms from stimulus onset

Parallel streams Ventral stream (What?): V1 Blob - V2 (Thin stripe) - V4 - IT Function: object recognition Dorsal stream (Where?): V1 4B - V2 Thick stripe - MT - Parietal Function: localization and action

Computational level Border ownership models: - recurrent processing (Li, 2005; Sakai & Nishimura, 2006) - feedback processing (Craft et al., 2007; Jehee et al., 2007) Filling-in models: - integration of depth, form and color (Grossberg, 1994, 1997) - figural filling-in (Domijan & Šetić, 2008)

Border ownership models Recurrent or intra-cortical processing (Li, 2005; Sakai & Nishimura, 2006) Figural status is assigned to one side of the border due to the local excitation and inhibition within neural network

Li (2005) Examples of mutual excitation and mutual inhibition between pairs of model neural elements coding border segments Mutual excitation/inhibition is more likely and stronger between neurons signaling border segments that are more consistent/inconsistent with belonging to a single figure

Border ownership models Feedback processing from higher visual areas (Craft et al., 2007; Jehee et al., 2007) Figural status is assigned to one side of the border due to the global integration and interactive activation in multi-scale neural network

Jehee et al. (2007)

Craft et al. (2007)

Border ownership models Problems: Cannot explain new principles of figure-ground assignment (spatial frequency, lower region, top-bottom polarity, extremal edges) Do not explicate how attention alters figure-ground assignment Do not explicate how surfaces are constructed Border ownership models do not relate precisely to perceptual experience

Filling-in models Boundary Contour System and Feature Contour System (Grossberg & Mingolla, 1985; Grossberg & Todorović, 1988) The model is based on the idea that filling-in ocur in the cortex as a diffusion process which spread neural activity within interior of the surface (isomorphic to the perceptual filling-in) Activation spreading is blocked by the boundary signals which encode surface borders

Perceptual filling-in A) Blind spot B) Neon color spreading C) Craik-O’Brien- Cornsweet effect D) Phantom illusion

Grossberg (1994) Multi-scale neural architecture for 3-D vision Figure-ground relations are resolved as part of depth segregation Near depth plane encodes figure Farther depth plane encodes background

Grossberg (1994)

Figural filling-in (Domijan & Šetić, 2008) Boundary computation: finding oriented luminance discontinuities (edges) Parietal cortex: saliency computation Surface representation: recurrent network with object-based lateral inhibition

Figural filling-in Figure-ground relation is resolved independently from depth stratification. Figure is denoted by the maximal firing rate. Concept of V1 and V2 as “active black-boards” – information retroinjected from parietal cortex is used to guide processing in ventral stream (Bullier, 2001) Base grouping and incremental grouping – feedforward and feedback modes of visual processing (Roelfsema, 2006)

Simulation of surroundedness, contrast and size

Simulation of lower region and top-bottom polarity

Filling-in models Problems: Cannot explain border ownership responses in V2 Exhibit only slow dynamics Neurophysiological mechanisms of filling-in are not clear Filling-in models do not relate precisely to neurophysiological findings

Conclusion Border ownership models could not explain many of the identified Gestalt principles of figural assignment Filling-in models are based on the assumptions that are not supported by the neurophysiology Important problem for future work: How to reconcile filling-in and border ownership models in order to provide a unified account of the figure-ground organization?

Thank you for your attention!