Why do things look the way they do? Because that’s they way they really are!? Because they had to look some way or other? In this class we will look at a somewhat related question:  What does the way things appear to us in our conscious experience tell us about how vision and mental imagery works and about the nature of the representations we construct when we see or visualize? La Grande Illusion is the belief that we have access to the form of representation we construct in episodes of cognition Recent example: Wegner, in The Illusion of Conscious Will, Argues that what we are conscious of is not what causes our voluntary acts  ZP: “…the ‘illusion’ is really that the experience we have of causing certain things to happen explains why it happened”

This is what our conscious experience suggests goes on in vision…

This is what the demands of explanation suggests must be going on in vision…

Subjective experience suggests that when we see, we are examining an inner display The completion of percepts from partial information Blind spot, resolution and color sensitivity ‘amodal completion’ (most scene objects are occluded) Completions have real behavioral consequences ‘illusory contours’ and amodal completions act in many ways like real figures Visual representations appear to be panoramic, stable, and in an allocentric frame of reference The superposition theory has a lot of intuitive appeal Are there any reasons (other than conscious experience) for taking this view seriously?

Completions …

Standard view of saccadic integration by superposition

The superposition view fails O'Regan, J. K., & Lévy-Schoen, A. (1983). Integrating visual information from successive fixations: Does trans-saccadic fusion exist? Vision Research, 23(8),

Visual interpretation is local Partly AmbiguousUnambiguous #1Unambiguous #2

Visual interpretation is local

Coordinating the interpretations among parts is more difficult if they are not both visible at the same time

When temporal integration occurs is shows the effect of memory load Higher memory loadLower memory load

When temporal integration occurs is shows the effect of memory load

When temporal integration occurs order of presentation is important

Errors in recall suggest how visual information is encoded Errors in relative orientation often take a canonical form Errors in reproducing a 3D image preserve 3D information Children have very good visual memory, yet often make egregious errors of recall

Errors in recall suggest how visual information is encoded Children more often confuse left-right than rotated forms Errors in imitating actions is another source of evidence

Ability to manipulate and recall patterns depends on their conceptual, not geometric, complexity Difficulty in superimposing shapes depends on they are conceptualized Look at first two shapes and superimpose them in your mind; then draw (or select one) that is their superposition

Many studies have shown that memory for shapes is dependent on the conceptual vocabulary available for encoding them e.g., recall of chess positions by beginners and masters

Finally, the phenomenology of seeing (including its completeness, its filled-out details, and its panoramic scope) turns out to be an illusion! We see far less, and with far less detail and scope, than our phenomenology suggests Objectively, we are colorblind – and outside a small region called the fovea, our sight is so bad we are legally blind. The rest of the visual field is seriously distorted and even in the fovea not all colors are in focus at once. More importantly, we register surprisingly little of what is in our field of view. Despite the subjective impression that we can see a great deal that we cannot report, recent evidence suggests that we cannot even tell when things change as we watch.

What changes between flashes? Airplane Helicopter Dinner Farm scene Paris corner

Where does this leave us? Should we conclude that seeing is a process of constructing conceptual descriptions? Most cognitive scientists and AI people would say yes, although there would be several types of exception.  There remains the possibility that for very short durations (e.g sec) there is a form of representation very like visual persistence – sometimes called an ‘iconic storage’ (e.g., Sperling, 1960).  From a neuroscience perspective there is evidence of a neural representation in early vision – in primary visual cortex – that is retinotopic and therefore “pictorial.” Doesn’t this suggest that a ‘picture” is available in the brain in vision?  We shall see later that this evidence is seriously misleading and does not support a picture theory of working memory or LTM  A major theme of this course will be to show that an important mechanism of vision is not conceptual but causal: Visual Indexes  Many people continue to hold a version of the “picture theory” of mental representation in mental imagery. More on this later.

Conceptual & Methodological Problems in the study of Visual Perception There is a problem about what it means to see: Ordinary usage confounds: seeing X, seeing X as R, seeing that P, and believing that P. Distinguishing these is a cottage industry in philosophy.  Some examples of the prescientific sense of see and why it is a problem (Droodles, “looks like”) Examples of droodlesExamples of droodles  Ordinary language also confounds the content and the vehicle of perception: what perceptual representations are about and what properties the representations themselves have  There is a serious problem about what to make of phenomenology – about the conscious experience of seeing. In view of how misleading subjective experience can be, and in view of the fact that what progress there is cognitive science has been made by ignoring the distinction between processes of which we are consciously aware and those of which we have no conscious experience, cognitive science, unlike philoeophy, has tended to shy away from the evidence of introspection.

Conceptual & Methodological Problems in the study of Visual Perception Part of the puzzle about vision rests on the failure to make certain distinctions (such as those alluded to earlier: between seeing and believing, between content and form). These two distinctions will play a major role in our later discussion. An important distinction that I will address only briefly is between seeing and coming to have a belief about what one is seeing. This is crucial to my claim about the Cognitive Impenetrability of Vision or the view of vision as an architectural module. This is the topic of Chapters 2 and 3 and will be summarized very briefly.  The advent of signal detection theory alerted is to the fact that each instance of signal detection involves at least two stages: a detection stage and a response selection or decision stage which is cognitively penetrable.