Langston, PSY 4040 Cognitive Psychology Notes 5 Imagery

An Exercise Imagine a dinner plate. It has spaghetti around the top rim. There is a carrot in the middle, pointing down. There are two fried eggs under the spaghetti. There’s a banana along the bottom rim. What do you see?

Questions What is a mental image? How is it like reality, how is it different? Does an image use the same neural hardware as experience? Depending on how we answer the first set of questions, how do we know what’s real?

Architecture Recall our box model: Sensory Store Filter Pattern Recognition Selection STM LTM Input (Environment) Response

Working Memory We modified short-term memory into working memory: Articulatory Loop Visuo-Spatial Sketchpad Central Executive

Imagery Imagery is going to cover more than one box. In working memory, it’s going to be in the visuo-spatial sketchpad. It’s also going to influence long-term memory.

Plan of Attack How can we assess imagery (mostly a working memory issue)? How does imagery affect memory (mostly a long-term memory issue)? What are images (a test of the cognitive psychology paradigm)? How do we maintain contact with reality?

Assessing Imagery Spatial ability is somewhat independent of verbal ability. We can test it using methods similar to operation span or digit span. I have two examples of operation span-type tasks:

Assessing Imagery Cube folding: Will the arrows touch if you fold these into cubes?

Assessing Imagery Cube task: Where does the X end up?

Assessing Imagery Note how both of these involve storage and processing.

Imagery and Memory Images in working memory: Mental rotation tasks suggest that you use images in working memory in an analog way. Shepard and Metzler (1972): Rotate images, look at response time.

Imagery and Memory Shepard & Metzler (1972): A: “Same” rotated in picture plane. B: “Same” rotated in depth. C: Different. Shepard & Metzler (1972, p. 702)

Imagery and Memory Shepard & Metzler (1972): Shepard & Metzler (1972, p. 702)

Imagery and Memory Images in working memory: We can look at our CogLab results here…

Imagery and Memory Images in working memory: Other tasks also suggest analog use of images in working memory. For example, imagine these times on an analog clock face. Which of each pair has a bigger angle between the hands? 4:10 and 9:23 3:20 and 7:25 2:45 and 1:05 3:15 and 5:30

Imagery and Memory Images in working memory: Scanning tasks show that the farther apart two things are on an image, the longer it takes to mentally scan from one to the other (e.g., Kosslyn, Ball, & Reiser, 1978).

Imagery and Memory Kosslyn et al. (1978): Memorize this map… There’s a hut, a tree, a rock, a lake, a well, sand, and grass. Hear the name of an object, then another. Imagine a black dot zipping from one to the other on the shortest path. Push a button when it gets there. Kosslyn et al. (1978, p. 51)

Imagery and Memory Kosslyn et al. (1978): Kosslyn et al. (1978, p. 52)

Imagery and Memory Kosslyn et al. (1978): Kosslyn et al. (1978, p. 52)

Imagery and Memory Images in working memory: We’re eventually going to have to address some problems with knowing if people are really using images, or if there are task demands that make the results come out this way. Did Kosslyn et al.’s (1978) instructions cause the effect? Yes.

Imagery and Memory Kosslyn et al. (1978): Kosslyn et al. (1978, p. 54)

Imagery and Memory Kosslyn et al. (1978): Kosslyn et al. (1978, p. 54)

Imagery and Memory Kosslyn et al. (1978): Kosslyn et al. (1978, p. 54)

Imagery and Memory Images in working memory: Kosslyn et al. (1978) say it shows evidence for imagery, but that people don’t always use it. Which is right?

Imagery and Memory Learning pictures (long-term memory): Shepard (1967): Present 612 pictures or words with a recognition test. After 2 hours approximately 100% accuracy for pictures, 88% for words. After a week about 88% for both. Standing (1977): Learn 1,000 words, 1,000 simple pictures, or 1,000 bizarre pictures. After 2 days recognition memory was 61.5% for words, 77% for pictures, and 88% for bizarre pictures. Summation: Memory for pictures is better than memory for words, especially early on.

Imagery and Memory Learning with images: If pictures are better, will forming images help memory? Yes. Let’s start with a demonstration of the importance of the concrete-abstract dimension…

Imagery and Memory Learning with images: We’ll consider mnemonic devices (memory tricks) in the long-term memory unit, but let’s look at an imagery-based one here. Demonstration of the peg-word system: Learn your system by heart. Use the system to learn a list. For example, if “one is a horse” and you need to buy floor polish, you could imagine a horse polishing the floor. For recall, you say “one is a horse” and that’s your cue.

Imagery and Memory Learning with images: Paivio (1969) proposed a dual-code hypothesis to explain results like these. Concrete words and pictures and effective peg-word systems generate two codes in memory, a verbal code and an image code. Abstract words and ineffective peg-word systems only generate a verbal code. Having two codes gives you a greater variety of cues that you could use to recall, and more chances to recall.

Imagery and Memory Learning with images: More dual-code stuff. A symbolic distance effect occurs when it’s easier to make judgments about items that are farther apart on a dimension than it is to make judgments about items that are close on a dimension. For example: The fly is bigger than the flea. The mouse is bigger than the fly. The rabbit is bigger than the mouse. The dog is bigger than the rabbit. The horse is bigger than the dog.

Imagery and Memory Learning with images: Symbolic distance effect questions (true or false): The fly is bigger than the flea. The horse is bigger than the mouse. The rabbit is bigger than the dog. The mouse is bigger than the fly. The dog is bigger than the horse. Flea-Fly-Mouse-Rabbit-Dog-Horse

Imagery and Memory Learning with images: More dual-code stuff. Picture symbolic distance effect tasks are easier than verbal symbolic distance effect tasks (pictures access the system directly, verbal has to be recoded). It’s pretty odd that the exact sentence you saw before (The fly is bigger than the flea) takes longer to verify than a new sentence (The horse is bigger than the mouse) if you don’t have images.

Structure of Images Images could be: Propositions: It’s essentially a verbal/symbolic thing. The feeling that you have an image is epiphenomenal, there isn’t really an image. Pictures in the head: Images are what they feel like, a picture in the head whose properties are like the properties of the real thing. Perceptual processing: Images use the same perceptual hardware you use to see, just not so much. We’ll consider each.

Structure of Images Images could be: Propositions We’ll think about this later…

Structure of Images Images could be: Pictures in the head It’s obviously not a literal picture. For example, a picture of a tomato is red, if you have an image of one, it’s not actually red. Can we see if images and pictures are different? Yes.

Structure of Images Images could be: Pictures in the head Pictures support processing that images don’t. For example, you can reverse pictures, but most people report that reversing images is very hard (relative to with a picture).

Structure of Images Images could be: Pictures in the head Pictures support processing that images don’t. For example, it’s harder to decompose an image than it is to decompose a picture. Let’s try a couple of examples:

Structure of Images Images could be: Pictures in the head Get a clear mental image of the picture below:

Structure of Images Images could be: Pictures in the head Are the following shapes in the picture you just saw?

Structure of Images Images could be: Pictures in the head

Structure of Images Images could be: Pictures in the head Get a clear mental image of the picture below:

Structure of Images Images could be: Pictures in the head Are the following shapes in the picture you just saw?

Structure of Images Images could be: Pictures in the head

Structure of Images Images could be: Pictures in the head On the other hand, you can reinterpret images, but you have to be able to overcome low-level grouping processes (Finke, Pinker, & Farah, 1989). For example: Think of a lower-case k. Imagine a circle around it, just not touching it. Now, cut off the bottom half of the k. What do you have?

Structure of Images Images could be: Use of the perceptual apparatus If images use visual hardware and words don’t, verbal and image information should not interfere. If they all use the same hardware, you should see interference. Brooks (1968) looked at this. Let’s look at his task:

Structure of Images Images could be: Use of the perceptual apparatus Brooks’ (1968) task: Present block letter and answer sheet. Point to the Y or N depending on whether the corner is at an extreme part of the figure. Y N Y N Y N

Structure of Images Images could be: Use of the perceptual apparatus Brooks’ (1968) task: Participants memorized the figure and then answered yes and no by pointing. They could also just make verbal responses. Pointing made it really hard. On the other hand, when the task was to memorize a list of sentences and make judgments about the words, pointing was easier. This suggests that images are using visual/perceptual hardware.

Structure of Images Images could be: Use of the perceptual apparatus On the other hand, images don’t seem to give after-effects. Get a clear mental image of this flag. What should happen if you maintain the image for a minute?

Structure of Images Images could be: Use of the perceptual apparatus Now stare at the dot for a minute and see what happens.

Structure of Images Images could be: Use of the perceptual apparatus Images should probably produce similar effects if they’re perceptual.

Imagery Applications Estes, Verges, & Barsalou (2008) Reading words should lead to mental simulations of the words, using perceptual hardware. Part of this simulation is location. Trial: Prime: Cowboy Word: hat Target: Letter (top or bottom of the screen) If you simulate the location, then hat should interfere with letters at the top (you’re using that perceptual hardware). Boot would be opposite.

Bear

Snout

A

Imagery Applications Estes et al. (2008, p. 95)

Imagery Applications Estes, Verges, & Barsalou (2008) Sure enough, letters in objects’ typical locations took longer to identify.

Imagery Applications Zwaan & Yaxley (2003) Or Spatial iconicity effects also suggest that location is part of the representation of words and that location simulation is part of comprehension. Present a pair of words, are they related? Attic Basement Or

Imagery Applications Zwaan & Yaxley (2003) Zwaan & Yaxley (2003, p. 956)

Imagery Applications Zwaan & Yaxley (2003) When vertical arrangements were correct, participants were faster than when they were incorrect. Again, location seems to be part of the understanding of a word.

Aside We don’t address brains much in this class. But, in this case reaction times may be too crude to detect whether or not “typical arrangement” is part of your semantic representation. What follows is partially derived from Kutas and Federmeier (2000; doi:10.1016/S1364-6613(00)01560-6)

Aside With event-related potentials (ERPs) you can measure changes in EEG as a result of presenting an event. The figure on the next slide has lots of examples of N400: “a negative component peaking around 400 ms after stimulus-onset [that] has been shown to vary systematically with the processing of semantic information” (p. 463).

Kutas & Federmeier (2000, p. 465)

Aside Will you get an N400 when you present stimuli from Zwaan & Yaxley (2003)? Three kinds of comparisons: Iconic vs. Unrelated Attic Witch Basement Median

Hubbard (2012, p. 17)

Aside Will you get an N400 when you present stimuli from Zwaan & Yaxley (2003)? Three kinds of comparisons: Reverse Iconic vs. Unrelated Basement Witch Attic Median

Hubbard (2012, p. 18)

Aside Will you get an N400 when you present stimuli from Zwaan & Yaxley (2003)? Three kinds of comparisons: Iconic vs. Reverse Iconic Attic Basement Basement Attic

Hubbard (2012, p. 19)

Structure of Images Images could be: Which is it? We’re going to leave this issue unresolved. I think there is enough evidence to suggest that images are more than purely verbal/symbolic information (propositions). The extent to which they’re using perceptual hardware is something we won’t resolve here, but it’s an interesting cognitive question. Something to noodle on: Can embodiment resolve some of this? Something to noodle on: Can techniques like ERP help with this?

Reality Monitoring How do you know what you imagined vs. what you saw? Johnson & Raye (1981): Internally and externally generated memories differ along four dimensions: “externally generated memories…have more spatial and temporal contextual attributes” (p. 71) “[externally generated memories] should also have more sensory attributes, although imaginal processes presumably also generate some sensory information” (p. 71)

Reality Monitoring How do you know what you imagined vs. what you saw? Johnson & Raye (1981): Four dimensions: “externally generated representations are more semantically detailed—that is, contain more information or more specific information” (p. 71) “internally generated memories may typically have more operational attributes associated with them” (p. 71) (perception is automatic, but imagining takes effort and leaves behind evidence of that effort in the memory)

Reality Monitoring How do you know what you imagined vs. what you saw? Example: Imagine seeing a movie vs. seeing a movie (Finke, Johnson, & Shyi, 1988, p. 133) The real one will have details about the day and the place, etc. that you saw it The real one would have more information from perceptual processing The real one would have more details of the content of the movie The real one wouldn’t have much about effort to imagine it

Reality Monitoring How do you know what you imagined vs. what you saw? Johnson & Raye (1981): Deciding: Mostly one direction, choose that direction In the middle, go to an additional step (we’ll see more models like this as we go) Johnson & Raye, 1981, p. 71)

Reality Monitoring How do you know what you imagined vs. what you saw? Could we model this as a signal detection task? E.g., real vs. imagined? Sensitivity? Bias?

Reality Monitoring How do you know what you imagined vs. what you saw? Johnson & Raye (1981): Evidence: Internal and external memories differ (generation effect, Slamecka & Graf, 1978); generated items are remembered better Let’s try the demonstration…

Reality Monitoring How do you know what you imagined vs. what you saw? …Difficulty should matter: Finke, Johnson, and Shyi (1988) Show them harder to image halves, easier to image halves, or whole pictures Look at how difficulty affects their ability to remember what they were shown (was it whole or half when you saw it?)

Reality Monitoring How do you know what you imagined vs. what you saw? Finke, Johnson, and Shyi (1988): The vertical ones were easier (Finke, Johnson, and Shyi, 1988, p. 135)

Reality Monitoring How do you know what you imagined vs. what you saw? Finke, Johnson, and Shyi (1988): Results: Memory was good for all, and equal (top row) It was harder to know what you had seen vs. imagined for the easier to image vertical halves (bottom row) (Finke, Johnson, and Shyi, 1988, p. 135)

Reality Monitoring How do you know what you imagined vs. what you saw? Finke, Johnson, and Shyi (1988): Results: So: If you worked harder to make the image, you had more of the “operational attributes”, and that meant you could remember you imagined it When it was easy, the “operational attributes” were missing, and there was a better chance you thought you had seen it

Reality Monitoring How do you know what you imagined vs. what you saw? It might be profitable to think about situations where you have reality monitoring confusions and how they occur in those situations

End of Imagery Show