Arizona State University Memory and Cognition PSY 324 Topic: Visual Imagery Dr. Ellen Campana Arizona State University

Visual Imagery

What is Imagery? Think about the following questions…. How many windows are on front of where you live? How is the furniture arranged in your bedroom? Are an elephant’s ears rounded or pointy? What does a pumpkin pie smell like? You use mental imagery to answer these Mental imagery = experiencing a sensory impression in the absence of sensory input

Uses of Imagery Creative process of music Sports Psychology Paul McCartney’s “Yesterday” Conductors use “inner audition” to practice without orchestras Imagine sounds Imagine locations of instruments Sports Psychology Swimming – imagining all of the strokes Time to mentally rehearse linked to real times Jack Nicklaus, golfing, discovered an error

Uses of Imagery Visual Imagery (mental imagery of visual info) useful in scientific discoveries Einstein’s theory of relativity came from imagining himself riding along a beam of light

The Study of Imagery Early ideas Aristotle: thought is impossible without an image Wundt: images are one of three basic elements of consciousness (others: sensations, feelings) Strong link between images and thought Gave rise to imageless-thought debate Can people think without images? Galton said yes: people who have difficulty forming images can still think Wundt still said no

The Study of Imagery Behaviorism Rise of behaviorism shut down the imageless-thought debate Study of imagery is unproductive because visual images are invisible to everyone else Watson (1928) called visual imagery “unproven” and “mythological” 1920’s-1950’s: no study of imagery in mainstream psychology

The Study of Imagery The Cognitive Revolution Cognitive psychologists developed ways to measure behavior to infer cognitive processes (like visual imagery) Memory (Paivio, 1963, 1965) Mental chronometry (Shepard & Meltzer, 1971) Study of imagery returned to mainstream psychology Research continues today

Visual Imagery Research

Demonstrating that Imagery Exists Paivio (1963, 1965): concrete vs. abstract nouns Method: paired-associate learning Training: participants learn pairs of words Testing: participants presented with first word of pair, task is to recall the second (recognition task) Two types of word pairs (within-Ss design) Concrete: truck – tree (both easy to form image of) Abstract: truth – idea (both hard to form image of) Result: concrete pairs were easier to recall Interpretation: Conceptual-peg hypothesis

Demonstrating that Imagery Exists Shepard and Metzler (1971) Mental chronometry – general idea of measuring how long it takes to carry out various cognitive tasks Task: participants saw two objects, had to indicate quickly whether the two objects were the same or different Results: Time it took to indicate that they were the same object was directly related to how far the object had to be rotated Interpretation: imagery and perception share some of the same mechanisms

Shepard & Metzler (1971)

Imagery and Perception Major question: do imagery and perception share the same mechanisms? Kosslyn (1973): mental scanning Task: Memorize an image, then answer questions about whether certain parts appear in the image Time it takes to say yes is related to distance between initial focus and correct part

Kosslyn (1973)

Imagery and Perception Major question: do imagery and perception share the same mechanisms? Kosslyn (1973): mental scanning Task: Memorize an image, then answer questions about whether certain parts appear in the image Time it takes to say yes is related to distance between initial focus and correct part Lea (1975): alternative explanation for result Perhaps as people scan they are distracted by other parts Kosslyn (1978): ruled out alternative explanation

Imagery and Perception

Imagery and Perception Kosslyn’s experiments were convincing, but there was another alternative explanation Pylyshyn (1973) – results are based on propositional mechanisms, not on spatial representation Ushered in the imagery debate (still going on)

Propositional vs. Spatial Basic idea: just because the experience is spatial doesn’t mean the underlying representation is Spatial experience of mental images could be an epiphenomenon (side effect) Epiphenomenon: related to / co-occurring with the mechanism, but not actually part of the mechanism Information could be encoded with language (propositional representation) or with images (depective representations) – can’t tell Propositional rep. is like a semantic network

Propositional vs Spatial

How Does Phlyshyn Explain Kosslyn’s First Set of Results? Tacit-knowledge explanation People in the mental scanning task behave based on what happens in a real scene (tacit knowledge) In the real world it takes longer to travel greater distances Simulate this behavior in the experiment This explanation seems overly complicated and doesn’t seem to scale to complex scenes Phylyshyn still makes these arguments, so they have to be addressed

Propositional vs Spatial

Against Propositional Representation Finke and Pinker (1982) Short presentation of simple display with 4 random dots, followed (after 2-sec delay) by an arrow Participants had to say whether the arrow pointed to one of the dots in the first display (gone now)

Finke and Pinker (1982)

Against Propositional Representation Finke and Pinker (1982) Short presentation of simple display with 4 random dots, followed (after 2-sec delay) by an arrow Participants had to say whether the arrow pointed to one of the dots in the first display (gone now) Results: time it took to respond “yes” was directly related to distance from arrow Why couldn’t this be propositional? No time to convert to propositions, no meaning in the dots (except spatial relations)

Propositional vs. Spatial Phylyshyn refuses to cede, responding to each study with tacit knowledge explanations He is in the minority Most researchers think there’s overwhelming evidence that mental imagery is spatial Let’s assume (like most people) that mental imagery is spatial, and go on talking about how it relates to perception

Imagery and Perception

Size in the Visual Field As you move closer to objects in the real world Object fills more of your visual field Details are easier to see Kosslyn (1978) used these facts to investigate mental imagery

Size in Visual Field

Kosslyn (1978) Task Conditions (within-subjects) Imagine two objects, moving close enough so that the bigger object fills most of visual field Answer questions about one of the animals Conditions (within-subjects) Question about the animal when it was bigger than the other animal Question about the animal when it was smaller than the other animal Result: questions answered faster when it was bigger Same as with a similar perceptual task

Kosslyn (1978)

Kosslyn (1978) In addition to the question-answering experiment, participants did a mental walk task Imagine an animal Walk toward it until it “overflows” the visual field How far away is it? (estimate distance) Let’s try: Cat This building Result: Greater distances for larger objects

Interactions Between Perception and Imagery Visual images can interact with perception Perky (1910) – subliminal presentation Participants imagined an object and described their mental image to the experimenter While they did this, an image of the same object was projected very dimly No participants reported seeing the image Descriptions of mental images matched details of real image Farah (1985) – priming task Mental images prime performance in a perception task

H Farah (1985) Imagine the letter ___ Indicate whether it was in the first or second white square H Results: participants were more accurate when the letter that appeared matched the letter they had imagined

Imagery and the Brain

Brain’s Response to Imagery Imagery neurons (Kreiman & coworkers, 2000) Respond to both perceiving and imagining an object Single cell recording, but in humans fMRI & PET most studies show overlap for perceiving and imagining (visual cortex, frontal lobes) More activity toward the back of the brain for perception than imagery Nonvisual areas deactivated (Amedi & coworkers, 2005)

Brain’s Response to Imagery Transcranial Magnetic Stimuluation Method for temporarily disrupting function in part of the brain Lets researchers demonstrate causality, not just correlation (if you disrupt it and people can still do the task it cannot be causal) Kosslyn and coworkers (1999) used this method to show that areas involved in both perception and imagery were causally linked to imagery (not an epiphenomenon)

Brain’s Response to Imagery Neuropsychological case studies Some evidence of related mechanism MGS: mental imagery changed after removal of occipital lobe tissue Perceptual problems accompanied by problems with imagery Unilateral neglect (Map of part of Milan study) Some evidence of independence Dissociations between imagery and perception R.M. Could see and draw but not imagine and draw, etc. C.K. Visual agnosia (could not name pictures) but OK drawing from imagination

Putting it all Together Some evidence for common mechanism BUT also some evidence for separate mechanisms Behrmann & Coworkers (1994) have an account that brings them together (top-down vs. bottom-up) No true “answer,” just continuing debate

Other stuff…

Using Images to Improve Memory Method of Loci “Placing” objects at places within a familiar location to remember them – walking through that location when remembering Peg-word Technique Learning a rhyme (one is a bun, two is a shoe…) and then imagining things you want to remember asssociated with those things

Mental Representation of Mechanical Systems People can use visual imagery to solve problems Five-gear problem Water-pouring problem Pulley problem It has also been important for scientific discovery Einstien’s theory of relativity

The End