Chapter 4: Object Recognition What do various disorders of shape recognition tell us about object recognition? What do various disorders of shape recognition tell us about object recognition? - Apperceptive visual agnosia (Ch. 2) - Associative visual agnosia - Perceptual categorization deficit What do neuroimaging studies tell us about object recognition? What do neuroimaging studies tell us about object recognition? The computational interpretation: What does cognitive evidence suggest about constraints on the nature of the neural representations underlying object recognition? The computational interpretation: What does cognitive evidence suggest about constraints on the nature of the neural representations underlying object recognition?

Object Recognition Dr. Harley has filled you in on the neuroscience side of the miracle… Dr. Harley has filled you in on the neuroscience side of the miracle… Farah (2000) – Fig. 4.1

Visual Agnosias Visual Agnosia: A blanket term for impaired visual object recognition following brain damage when elementary visual functions (acuity, visual fields) are adequate. Visual Agnosia: A blanket term for impaired visual object recognition following brain damage when elementary visual functions (acuity, visual fields) are adequate. Apperceptive visual agnosia: Inability to group the local features into a coherent perceptual representation (Ch 2) Apperceptive visual agnosia: Inability to group the local features into a coherent perceptual representation (Ch 2) Associative visual agnosia: Inability to recognize visually presented objects, despite having a coherent perceptual representation Associative visual agnosia: Inability to recognize visually presented objects, despite having a coherent perceptual representation

Visual Agnosias: Copying and drawing from memory. Apperceptive agnosia Associative agnosia Copying Memory Copying

Associative visual agnosia: Criteria Patients can form percepts (do perceptual grouping) unlike apperceptive agnosia patients. Patients can form percepts (do perceptual grouping) unlike apperceptive agnosia patients. They can see an object well enough to describe its appearance, to draw it, or to succeed in a same/different test of appearances. They can see an object well enough to describe its appearance, to draw it, or to succeed in a same/different test of appearances. They have difficulty recognizing visually presented objects (can’t name or sort objects by category) They have difficulty recognizing visually presented objects (can’t name or sort objects by category) They can demonstrate knowledge of objects from other sensory modalities They can demonstrate knowledge of objects from other sensory modalities

Associative agnosia behavior: An impairment of shape perception They can copy complex objects. BUT their copying behavior is abnormal – HJA took 6 hours to do the cathedral. They can copy complex objects. BUT their copying behavior is abnormal – HJA took 6 hours to do the cathedral. They are very sensitive to visual quality of stimuli. They recognize real objects better than line drawings; face recognition of unfamiliar people is impaired by changing lighting conditions. They are very sensitive to visual quality of stimuli. They recognize real objects better than line drawings; face recognition of unfamiliar people is impaired by changing lighting conditions. They make visual shape recognition errors; they might call a baseball bat, a paddle, knife, baster, etc. They make visual shape recognition errors; they might call a baseball bat, a paddle, knife, baster, etc.

Associative Visual Agnosia The human analog of the IT-lesioned monkeys. The human analog of the IT-lesioned monkeys. –They fail to recognize objects because they fail to represent shape normally. Slow, slavish copying Slow, slavish copying Sensitivity to visual quality of the stimuli Sensitivity to visual quality of the stimuli Visual shape errors Visual shape errors

Regions of brain damage associated with different types of visual agnosia Apperceptive Agnosia: Diffuse damage to the occipital lobe and surrounding areas Apperceptive Agnosia: Diffuse damage to the occipital lobe and surrounding areas Associative Agnosia: Occipitotemporal regions of both hemispheres Associative Agnosia: Occipitotemporal regions of both hemispheres From Banich (2004)

Perceptual categorization deficit Difficulty recognizing objects viewed from unusual perspectives or uneven illumination conditions (Warrington & Taylor, 1973) Difficulty recognizing objects viewed from unusual perspectives or uneven illumination conditions (Warrington & Taylor, 1973) Initially assumed to reflect an impairment of viewpoint-invariant object recognition Initially assumed to reflect an impairment of viewpoint-invariant object recognition

BUT: Is Perceptual Categorization Deficit really about loss of shape representations? Not impaired in the real world. Not impaired in the real world. Not impaired under all viewing conditions per se. Not impaired under all viewing conditions per se. Mainly impaired when matching an USUAL to an UNUSUAL view; normal people have similar, less serious, type of difficulty. Mainly impaired when matching an USUAL to an UNUSUAL view; normal people have similar, less serious, type of difficulty. Associated with unilateral RH lesions, usually in parietal cortex, not inferotemporal cortex Associated with unilateral RH lesions, usually in parietal cortex, not inferotemporal cortex

Perceptual categorization deficit Demonstrates value of examining evidence carefully and trying to link it with what is known to see whether patterns fit or not. Demonstrates value of examining evidence carefully and trying to link it with what is known to see whether patterns fit or not. In this case, the pattern differs greatly from other visual agnosias. In this case, the pattern differs greatly from other visual agnosias.

Functional Neuroimaging Studies: Goal of PET, fMRI studies: To localize the psychological process(es) of interest Goal of PET, fMRI studies: To localize the psychological process(es) of interest Research design: Measure brain activity in at least two conditions: a control (baseline) condition and an experimental condition. Research design: Measure brain activity in at least two conditions: a control (baseline) condition and an experimental condition. –Psychological process localized by subtracting brain activity in the control (baseline) condition from brain activity in the experimental condition.

PET Imaging ( Posner & Raichle, 1994 ) Upper row: Control PET scan (resting while looking at static fixation point) is subtracted from looking at a flickering checkerboard stimulus positioned 5.5° from fixation point. Middle row: Subtraction procedure produces a somewhat different image for each of 5 subjects. Bottom row: The 5 images are averaged to eliminate noise, producing the image at the bottom.

Where in the brain does visual recognition occur? Visual recognition localized to the posterior half of the brain – really? Visual recognition localized to the posterior half of the brain – really? Whoops!!! Why did these studies fail to localize visual recognition? Whoops!!! Why did these studies fail to localize visual recognition? Poor methodology! Poor methodology!

Neuroimaging methods: The subtraction technique Using the subtraction technique effectively requires that the baseline and experimental conditions differ only in the process of interest. Using the subtraction technique effectively requires that the baseline and experimental conditions differ only in the process of interest. –Requires careful logical analysis of the task to determine the best comparison conditions. If there are many differences between the baseline and experimental conditions, it is difficult to interpret the results of the study. If there are many differences between the baseline and experimental conditions, it is difficult to interpret the results of the study.

Neuroimagining Studies of Object Recognition: Comparing baseline and experimental conditions Sequence of trial events –Fixation point –Stimulus presentation –Task (two types) Passive Viewing Task: Are stimuli comparable (e.g., size, complexity) Mind is not a vacuum Active Viewing Task: What is task? What is response?

Sergent et al (1992a): Active viewing task. Baseline condition Baseline condition - View fixation point - View fixation point Experimental condition Experimental condition - View fixation point - See line drawing of object - Decide whether the object is living or non- living - Make Yes/No response

Localizing Visual Recognition Is human visual recognition supported by one general purpose recognition system or are there specialized modules for recognizing objects, faces, printed words? Is human visual recognition supported by one general purpose recognition system or are there specialized modules for recognizing objects, faces, printed words? Good functional neuroimaging studies do exist to test these possibilities and will be discussed in Chapters 5 & 6. Good functional neuroimaging studies do exist to test these possibilities and will be discussed in Chapters 5 & 6.

Constraints on the nature of shape representations in IT : Coordinate systems Cannot be simple viewer-centered or environmentally-centered Cannot be simple viewer-centered or environmentally-centered – IT cells respond to a given shape over changes in the position, size, and picture plane orientation of object –Impairments of IT lesioned monkeys: suggest they have lost abstract shape representation

Constraints: Coordinate systems cont. Two possibilities: Two possibilities: 1. Object-centered coordinate system 2. A cluster of multiple viewer-centered object representations plus the ability to transform one representation to another as necessary (a la Multiple Views model of Tarr, 1995).

Constraints: Coordinate systems cont. Farah prefers the Multiple Viewer-Centered representation option because: Farah prefers the Multiple Viewer-Centered representation option because: –Position, size, and orientation invariance shown by IT cells is not perfect; consistent with some views being better learned than others. –IT neurons can selectively learn arbitrary associations between pairs of stimuli, a prerequisite for deriving invariance from viewer-centered representations. –Consequently, because cell activity takes time to decay and because seeing different views of an object tends to be clustered in time, the correlation (association) between several different retinotopic views of the same object could be learned.

Constraints: Primitives and Organization Primitives Primitives –Cannot be contours –Could be either surface-based (2D) or volume-based (e.g., 3D geon-like parts). Organization Organization –Hierarchical? We really don’t know much about how multipart objects are represented

Constraints: Implementation Symbolic Model vs Neural net Is a perceptual representation created and then compared to a memory representation? (Implies that the memory representation can be destroyed, yet the perceptual representation remains intact). Unlikely, since no such evidence exists. Is a perceptual representation created and then compared to a memory representation? (Implies that the memory representation can be destroyed, yet the perceptual representation remains intact). Unlikely, since no such evidence exists. Does the input get coded and recoded in a succession of neural nets as it is processed through the visual system? (Implies that impairment in test of object memory is associated with perceptual impairment). Yes, this seems true in associative agnosia. Does the input get coded and recoded in a succession of neural nets as it is processed through the visual system? (Implies that impairment in test of object memory is associated with perceptual impairment). Yes, this seems true in associative agnosia.

Constraints: Implementation Distributed representation more likely than local representation Distributed representation more likely than local representation –Single unit recording –Graceful degradation following damage to IT