2. EE141 1 Perception - Vision Janusz A. Starzyk http://grey.colorado.edu/CompCogNeuro/index.php/CECN_CU_Boulder_OReilly http://grey.colorado.edu/CompCogNeuro/index.php/Main_Page Based on book Cognition, Brain and Consciousness ed. Bernard J. Baars courses taught by Prof. Randall O'Reilly, University of Colorado, and Prof. Włodzisław Duch, Uniwersytet Mikołaja Kopernika and http://wikipedia.org/ Cognitive Architectures

3. EE141 2 Motivation Perception is comparatively the easiest to understand although for many specific questions there are no clear answers. General questions:  Why does the primary visual cortex react to oriented edges?  Why does the visual system separate information into the dorsal stream, connected to motion and representation of object locations, and the ventral stream, connected to object recognition?  Why does damage to the parietal cortex lead to spatial orientation and attention disorders?  In what way do we recognize objects in different places, orientations, distances, with different projections of the image onto the retina?

4. EE141 3 Introduction  The purpose of vision: ‘to know what is where’ (David Marr)  The visual perception is far more complicated than simply taking a picture with digital camera  The camera doesn’t really do anything with this image and doesn’t have any knowledge about what is stored in the image

5. EE141 4  Knowing what: perceiving features, groups and objects  Studies of human visual perception and neuroscience suggest that there are many levels of perception.  The human brain appears to process basic visual features, such as color, orientation, motion, texture and stereoscopic depth.  Neurons are highly tuned to specific features like a line at particular angle, a particular color, or particular motion detection.  The activity of each neuron represents only a small part of the visual field. How is the brain able to combine this information across many neurons? The brain is able to organize basic feature elements into organized perceptual groups. Psychologists proposed the Gestalt laws of perceptual grouping, such as the laws of similarity, proximity, good continuation, common fate and so forth Introduction

6. EE141 5 Perceptual grouping  Grouping by similarity: White dots grouped with white dots, squares with squares  Grouping by proximity: Here we perceive two separate groups of dots that are near each other  Grouping by good continuation: On the left we perceive a single object. When the same lines are separated we do not

7. EE141 6 Visual pathways Visual pathways: retina => lateral geniculate nucleus (LGN) of the thalamus => visual radiation => area of the primary cortex V1 => higher levels of the visual system => associative and multimodal areas. V1 cells are organized in ocular dominance columns and orientation columns, retinoscopic. Simple layer 4 cells react to bands with a specific slant, contrasting edges, stimulus from one eye. A substantial part of the central V1 area reacts to signals from fovea, where the density of receptors is the greatest.

8. EE141 7 Functional organization of the visual system  Objects in environment are projected to the back of the eye – the retina.  Retina contains millions of photoreceptors

9. EE141 8 Cones cone-shaped less sensitive operate in high light color vision Rods rod-shaped highly sensitive operate at night gray-scale vision Two types of light-sensitive photoreceptors The retina

10. EE141 9 The signals from photoreceptors are processed by a collection of intermediary neurons, bipolar cells, horizontal cells and amacrine cells, before they reach the ganglion cells The retina http://www.iit.edu/~npr/DrJennifer/visual/retina.html

11. EE141 10 Retina  The retina is not a passive camera registering images.  Crucial rule: enhancing contrasts underlining changes in space and time, strengthening edges, uniformly lit areas are less important.  Photoreceptors in rods and cones,  3-layer network, ganglion cells =>LGN.  Receptive fields: areas, which stimulate a given cell.  The combination of signals in the retina gives center-surround receptive fields (on-center) and vice versa, detects edges.  Each individual field of cells can be modeled as a Gaussian model, so these fields are obtained as a difference of Gaussians (DOG).

12. EE141 11 On-center off surround ganglion cells  No stimuli:  both fire at base rate  Stimuli in center:  ON-center-OFF- surround fires rapidly  OFF-center-ON- surround doesn’t fire  Stimuli in both regions:  both fire slowly  Stimuli in surround:  OFF-center-ON- surround fires rapidly  ON-center-OFF- surround doesn’t fire

13. EE141 12 David Hubel & Torsten Wiesel Received Nobel price for their discovery of on-center off-surround cells in retina http://www.physiology.wisc.edu/yin/public/ on-center cell On-center off surround ganglion cells

14. EE141 13  Retina ganglion cells receive both excitatory and inhibitory inputs from bipolar neurons  In the figure shown, the ganglion cell receives excitatory inputs from cells corresponding to the on-center region, and inhibitory inputs from the off-center region On-center off surround ganglion cells

15. EE141 14  Lateral inhibition is important in enhancing neural representation of edges, where the light intensity changes sharply and indicate a presence of contours, shapes, or objects.  Uniform parts of a picture are less interesting. On-center off surround ganglion cells Original image Image based on edges

16. EE141 15  Sometimes this later inhibition leads to a surprising visual illusions as shown on this figure.  Notice black dots appear on intersection of white lines. On-center off surround ganglion cells

17. EE141 16 Retinal ganglion cells  There are two types of ganglion cells in the retina:  Large magnocellular ganglion cells, or M cells, carry information about: –Movement –Location –depth perception.  Smaller parvocellular ganglion cells, or P cells, transmit signals that pertain to: –Colour –Form –texture of objects in the visual field.

18. EE141 17 Lateral geniculate nucleus (LGN)  From the eye, retinal ganglion cells send their axons to a structure in the thalamus called lateral geniculate nucleus (LGN)  The inputs from the nasal portion of each retina must cross at the optic chiasm to project to the opposite LGN

19. EE141 18 Pathways in visual system  Propagation of the visual input from the left and right visual fields.  Signals propagate through eye, retina, optic nerve, chiasm, optic tract, LGN to visual cortex V1

20. EE141 19  Neurons in V1 are sensitive to a whole host of visual features, not seen in the LGN, like orientation, direction of motion, color differences, or binocular disparities.  Orientation helps to detect edges and contours.  Direction of motion is important to determine dangerous moves of an attacker.  Color helps to differentiate and identify objects particularly in a camouflage environment.  Binocular disparities between images in two eyes allow us to perceive stereo-depth when we look at object with both eyes. Primary visual cortex V1

21. EE141 20 Primary visual cortex V1  Neurons in V1 respond with different strength to orientation edges, depending on location of their receptive fields.  Neuron’s response is strongest if the excitation aligns with its receptive field.

22. EE141 21 Edge detectors Contrasting signals points from the LGN are organized by the V1 cortex into edge detectors oriented at a specific angle. Simple V1 cells combine into edge detectors, enabling the determination of shapes, other cells react to color and texture. Properties of edge detectors: different orientation; high frequency = fast changes, narrow bands; low frequency = gentle changes, wide bands; polarization = dark-light or vice-versa, dark-light-dark or vice-versa.

23. EE141 22 Representation in the V1 cortex Oriented edge detectors can be created by correlational Hebbian learning based on natural scenes. What happens with information about color, texture, motion?

24. EE141 23 Retinoscopic maps in V1 The spatial position of the ganglion cells within the retina is preserved by the spatial organization of the neurons within the LGN layers. The posterior LGN contains neurons whose receptive field are near the fovea.

25. EE141 24 Area V1: The Primary Visual Cortex  V1 is made up of 6 layers (no relation to 6 layers in LGN).  LGN sends axons to layer IV of V1.  M and P cells are separate.  Right and Left eye are separate.

26. EE141 25 Hierarchy of visual processing  From retina, LGN, V1, through V4 and to ventral temporal cortex (VTC) neurons gradually respond to more complex stimuli:  Retina and LGN extract small dots  In V1 small dots are combined into edges,  In V4 edges are combined into simple shapes and color features  In VTC simple shapes are combined into objects

27. EE141 26 Extrastriate visual areas – outside of V1  V1 sends signals to many higher visual areas, including areas such as V2, V3, V4 and motion- sensitive area MT.  Area V4 is important for the perception of color and some neurons in V4 respond to more complex features or their combination (like corners or curves).  The middle-temporal area (MT), is important for motion perception.  Almost all of the neurons in MT are direction-selective, and respond selectively to a certain range of motion directions or patterns of motion. Flattened map of higher visual areas

28. EE141 27 depth direction “where” pathway “what” pathway orientation shape color Recognized Object ready for perception  “Where" = large-celled pathway, heading for the parietal lobe.  "What"= small-celled pathway heading for the temporal lobe (IT). Two streams where?/what?

29. EE141 28  Lateral occipital complex (LOC)  The lateral occipital complex seems to have a general role in object recognition and responds strongly to a variety of shapes and objects.  Presumably neurons in this region respond best to different kinds of objects.  Fusiform face area (FFA)  Human neuroimaging studies have shown that there is a region in the fusiform gyrus, called the fusiform face area (FFA) that responds more strongly to faces than to just about any other category of objects.  This region responds more to human, animal and cartoon faces than to a variety of non-face stimuli.  Neurons in this area specialize in facial expression, particular identity or viewpoint (e.g. profile)  Parahippocampal place area (PPA)  The parahippocampal place area is another strongly category-selective region that responds best to houses, landmarks, indoor and outdoor scenes.  This area responds weakly to faces, body parts, and animals. Areas involved in object recognition

30. EE141 29 Hierarchical and interactive theories of vision  Hierarchical and interactive theories of vision  According to hierarchical theory, visual consciousness is organized in a hierarchical fashion with increasingly higher visual areas being more closely related to our internal conscious experience. But if this is the case how to explain awareness of all details in the observed image?  The interactive theory of visual consciousness emphasizes interactions between lower and higher visual areas where higher areas send feedback signals down to early visual area.  Is it possible to invoke visual experience without seeing?  Yes we can bypass retina and LGN and stimulate area V1.  However, it seems impossible to recover full visual experience from higher visual areas bypassing primary visual cortex.

31. EE141 30 Some answers  Why does the primary visual cortex react to oriented edges? Because correlational learning in a natural environment leads to this type of detector.  Why does the visual system separate information into the dorsal pathway and the ventral pathway? Because signal transformations extract qualitatively different information, strengthening some contrasts and weakening others.  Why does damage to the parietal cortex lead to disorders of spatial orientation and attention (neglect)? Because attention is an emergent property of systems with competition.  How do we recognize objects in different locations, orientations, distances, with different images projected on the retina? Thanks to transformations, which create distributed representations based on increasingly complex and spatially invariant features.