1. Brook's Image Scanning Experiment & Neuropsychological Evidence for Spatial Rehearsal
Psychology 355: Cognitive Psychology Instructor: John Miyamoto 04/26/2016: Lecture 05-2
Note: This Powerpoint presentation may contain macros that I wrote to help me create the slides. The macros aren’t needed to view the slides. You can disable or delete the macros without any change to the presentation.

2. Outline
Brandimonte's Mental Subtraction Experiment Mental Subtraction + Articulatory Suppression
Brook's Image Scanning Experiment: Interference between VSP & PL
Neuropsychological evidence for spatial rehearsal in the VSP
Similarities and differences between PL and VSP
Done!
Purpose of Brook's Image Scanning Experiment
Psych 355, Miyamoto, Spr ‘16

3. Next: Brook’s Image Scanning Experiment
Brooks, L. R. (1968). Spatial and verbal components of the act of recall. Canadian Journal of Psychology, 22,
Purpose #1: To show that there are 2 different short-term stores: VSP and PL
Purpose #2: To show that ...
Performing two different tasks in VSP interfere with each other.
Performing two different tasks in PL interfere with each other.
Performing one task in VSP and a different task in PL does not cause as much interference
Image Scanning Experiment
Psych 355, Miyamoto, Spr '16

4. Image Scanning (Brooks)
Experiment has 4 conditions.
2 types of stimulus: Diagrams or Sentences
2 types of responses: Pointing or Vocal Response*
Dependent variable = time to perform a task
Next: Explain the different stimulus types and response modes
Response Mode Pointing Vocal Diagrams Sentences
Stimulus Types
* There was actually a third response type, “tapping”, but it will not be discussed here – it turned out not to be very informative.
Point Out that Goldstein Textbook Omits the Sentence Stimuli
Psych 355, Miyamoto, Spr '16

5. Image Scanning (Brooks)
Experiment has 4 conditions.
2 types of stimulus: Diagrams or Sentences
2 types of responses: Pointing or Vocal Response*
Dependent variable = time to perform a task
Next: Explain the different stimulus types and response modes
Response Mode Pointing Vocal Diagrams Sentences
Stimulus Types
Same Slide with Remark that Goldstein Omits Sentence Stimuli
Psych 355, Miyamoto, Spr '16

6. Image Scanning (Brooks)
Experiment has 4 conditions.
2 types of stimulus: Diagrams or Sentences
2 types of responses: Pointing or Vocal Response*
Dependent variable = time to perform a task
Next: Explain the different stimulus types and response modes
Response Mode Pointing Vocal Diagrams Sentences
Stimulus Types
Goldstein's discussion of this study omits the Sentence stimuli. Only the diagram stimuli are discussed in the textbook.
Explain Diagram Stimulus x Vocal Response
Psych 355, Miyamoto, Spr '16

7. Image Scanning (Brooks)
Experiment has 4 conditions.
2 types of stimulus: Diagrams or Sentences
2 types of responses: Pointing or Vocal Response*
Dependent variable = time to perform a task
Next: Explain the different stimulus types and response modes
Response Mode Pointing Vocal Diagrams Sentences
Next Slide
Stimulus Types
Explain Diagram Stimulus x Vocal Response
Psych 355, Miyamoto, Spr '16

8. Image Scanning (Brooks, cont.)
Condition: Diagrammatic Stimulus & Vocal Response
Memorize the figure. Then it is removed.
Start your scan at the star and (mentally) move clockwise around the figure.
Say "OUT" when you reach an outside corner..
Say "IN" when you reach an inside corner.
Correct response: out, out, out, in, in, out, out, in, out, out, out
Dependent variable: Response time (time to complete the task)
Explain Diagram Stimulus with Pointing Response
Psych 355, Miyamoto, Spr '16

9. Image Scanning (cont.)
Next: Diagrammatic stimulus combined with pointing response:
Response Mode Pointing Vocal Diagrams Sentences
Stimulus Type
Previous Slide
Next Slide
Psych 355, Miyamoto, Spr '16

10. Image Scanning (Brooks, cont.)
Condition: Diagrammatic Stimulus & Pointing Response
Memorize the figure. Then it is removed. Same as before
Start your scan at the * and (mentally) move clockwise around the figure. Same as before.
Point to "Out" when you reach an outside corner.
Point to "In" when you reach an inside corner..
Correct response: (See diagram) out, out, in, in, out, out, in, out, out, out
Dependent variable: Response time (time to complete the task)
Stimulus
* Goldstein textbook shows the words "In" and "Out" lined up in straight columns, but the actual experimental task had irregular columns as shown on this slide.
* Irregular columns increase the load on spatial working memory; straight columns would not require as much attention to spatial layout.
Respond by pointing to letters on this sheet.
Transition to Next Condition: Sentence Stimulus & Vocal Response
Psych 355, Miyamoto, Spr '16

11. Image Scanning (cont.)
Explain sentence stimulus combined with vocal response:
Response Mode Pointing Vocal Diagrams Sentences
Stimulus Type
Next Slide
Condition: Sentence Stimulus & Vocal Response
Psych 355, Miyamoto, Spr '16

12. Image Scanning (Lee Brooks, cont.)
Sentence Stimulus
A bird in hand is worth two in the bush.
Condition: Sentence Stimulus & Vocal Response
Memorize the sentence. Then it is removed.
Now work from the beginning to the end of the sentence. Say "yes" each time you encounter a noun; say "no" when you encounter a word that is not a noun.
Correct response:
Dependent variable: Response time (time to complete the task)
no,
yes,
no,
yes,
no,
no,
no,
no,
no,
yes
Transition to Condition with Sentence Stimulus & Pointing Response
Psych 355, Miyamoto, Spr '16

13. Image Scanning (cont.)
Condition: Sentence Stimulus & Vocal Response Response Mode Pointing Vocal Diagrams Sentences
Stimulus Type
Next Slide
Sentence Stimulus x Pointing Response
Psych 355, Miyamoto, Spr '16

14. Image Scanning (Lee Brooks, cont.)
Condition: Sentence Stimulus & Vocal Response
Memorize the sentence. Then it is removed.
Work from the beginning to the end of the sentence. Point to "yes" each time you encounter a noun; say "no" when you encounter a word that is not a noun..
Correct response: no, yes, no, yes, no, no, no, no, no, yes
Sentence Stimulus: A bird in hand is worth two in the bush.
Respond by pointing to letters on this sheet.
Image Scanning Results
Psych 355, Miyamoto, Spr '16

15. Results: Image Scanning (Lee Brooks)
Stimulus Type
Response Mode Pointing Vocal
Diagrams Sentences
Slower
Faster
Faster
Slower
Same Slide: Why is Pointing Slower than Vocal For Diagrams & Opposite for Sentences?
Psych 355, Miyamoto, Spr '16

16. Results: Image Scanning (Lee Brooks)
Stimulus Type
Response Mode Pointing Vocal
Diagrams Sentences
Why is diagram/pointing slower than diagram/vocal? Why is sentence/vocal slower than sentence/pointing?
Slower
Faster
Faster
Slower
Discussion of Results
Psych 355, Miyamoto, Spr '16

17. Why is Diagram/Pointing Slower than Diagram/Vocal?
Stimulus Type
Response Mode Pointing Vocal
Diagrams Sentences
Diagram/Pointing: The stimulus and response compete for a common cognitive resource, representation in VSP. Information processing bottleneck causes slower response.
Diagram/Vocal: Stimulus & response use separate cognitive resources. No bottleneck.
Slower
Faster
Why is sentence/vocal slower than sentence/pointing?
Psych 355, Miyamoto, Spr '16

18. Why is Sentence/Vocal Slower than Sentence/Pointing?
Stimulus Type
Response Mode Pointing Vocal
Diagrams Sentences
Sentence/Vocal: Again, stimulus and response compete for a common cognitive resource, representation in PL. Information processing bottleneck causes slower response.
Sentence/Pointing: Stimulus & response use separate cognitive resources. No bottleneck.
Faster
Slower
xxx
Psych 355, Miyamoto, Spr '16

19. Brooks' experiment supports the existence of a visuospatial sketch pad (VSP) because ....
If there were only one STM store (not separate PL & VSP), then the effect of the pointing response would be the same in the visuospatial task and the verbal task. Similarly for the effect of the verbal response.
WM model assumes multiple memory stores – this lets WM predict conflicts between similar mental codes.
The preceding experiment illustrates a basic principle: A response in one modality (verbal or spatial) will interfere more with a memory representation in the same modality than with a memory representation in a different modality.
Spatial Rehearsal in Visual Working Memory
Psych 355, Miyamoto, Spr '16

20. Spatial Rehearsal in Working Memory – What Is It?
Spatial Rehearsal – maintaining information about a location in working memory
Delayed-Match-To-Sample (Monkey Study)
Psych 355, Miyamoto, Spr '16

21. Delayed-Match-To-Sample Paradigm
(a) Cue
(b) Delay
(c) Reward
Monkey sees where food is located.
10 second delay without visual input.
Monkey reaches for food (or makes a mistake and reaches in the wrong place).
Goldstein refers to this experimental task as the "delayed response task." See Figure 5.23, p. 143.
Psych 355, Miyamoto, Spr '16
Same Display – Single Cell Recordings Find Neural Correlates of (a), (b) & (c)

22. Delayed-Match-To-Sample Paradigm
(a) Cue
(b) Delay
(c) Reward
These results indicate importance of frontal lobes in maintaining spatial location information in VSP.
Single-cell recording in monkey PFC shows a correlation between neural activity & the three stages of this task.
Some neurons become active during the initial placement of the food.
Some neuron become active only during the delay period (spatial rehearsal).
Some neurons become active only when the response is made.
Same Display – Monkeys with PFC Lesions Cannot Do This Task
Psych 355, Miyamoto, Spr '16

23. Delayed-Match-To-Sample Paradigm
(a) Cue
(b) Delay
(c) Reward
Right
Wrong
These results indicate importance of frontal lobes in maintaining spatial location information in VSP.
Monkeys with frontal lesions cannot do this task. Human infants cannot do task until about 12 months old.
Multiple Location / Eye Movement Version of Delayed Response Task
Psych 355, Miyamoto, Spr '16

24. Monkey Performance on Delayed Eye-Movement Task
Same idea as the delayed-match-to-sample task (preceding slide), except that there are multiple locations to remember, and the response is an eye movement. See Goldstein, Figure 5.24 (p. 144).
The monkey's task is to see the target, wait until a response is permitted, and then move his eyes to the target position. He gets rewarded with a squirt of apple juice. +
Target Stimulus
Eye Movement Response
Delay Period + +
Psych 355, Miyamoto, Spr '16
Single-Cell Recordings When Perceiving the Cue; Maintaining Attention; Responding

25. These results show that scientist has identified a neurons that are specific to spatial rehearsal. Note that the neuron is only active while the monkey is rehearsing a specific spatial location, not for other spatial locations.
Funahashi, S., Bruce, C. J., & Goldman-Rakic, P. S. (1993). Dorsolateral prefrontal lesions and oculomotor delayed- response performance: Evidence for mnemonic "scotomas." Journal of Neuroscience, 13,
Neurons in DLPFC show sustained activity during delay period that is selective of the particular location to be remembered.
Errors occur when this sustained activity fails to initiate or decays before end of delay period.
Specific Lesions Cause Location-Specific Inability to Memory Loss
Psych 355, Miyamoto, Spr '16

26. error
Lesion
Perception for these locations is not impaired. Memory for other locations is not impaired. This shows that a specific brain injury can prevent a monkey from thinking about a location.
Lesions introduced into spatial memory areas cause location-specific inability to retain spatial information during the delay period. (Mnemonic scotoma).
Perception for these locations is not impaired, i.e., this is not a blind spot.
Memory for other locations is not impaired.
Neural Mind Reading
Psych 355, Miyamoto, Spr '16

27. Neural Mind Reading
Neural Mind Reading - guessing what someone was thinking from the neural response.
Neural mind reading of perceptual processing: fMRI used to guess which pattern a subject is viewing (Kamitani & Tong, 2005)
On each trial, the subject views the pattern on left or the pattern on the right:
Psychologists who are not told which stimulus was used on a given trial attempt to guess the stimulus based on fMRI image that was taken during that trial.
Psychologists achieve accuracy of 75% - 100% depending on the stimulus pair. Guessing rate would be 50%.
Perception study: Subject sees a striped disk. Experimenter who does not know which disk the subject was looking at has to predict the orientation of the stripes based on an fMRI image of the subject's visual cortex.
Visual working memory study: Subject sees both striped disks. The disks disappears. The subject is given a cue, 1 or 2, that tells him which disk to hold in memory. After an 11 second delay, the subject is shown a new striped disk Experimenter who does not know which disk the subject was looking at has to predict the orientation of the stripes based on an fMRI image of the subject's visual cortex. OR
Neural Mind Reading of Activity in VSP
Psych 355, Miyamoto, Spr '16

28. Neural Mind Reading of Activity in VSP
fMRI used to guess which pattern a subject is retaining in VSP (Harrison & Tong, 2009)
On each trial, the subject views a sequence of screens.
Psychologists who are not told which target was retained in VSP on a given trial attempt to guess the target based on fMRI image that was taken during that trial.
Psychologists achieve accuracy of 80% (average).
fMRI successfully detected VSP rehearsal of the target image. 2
Target 1
Target 2
Remember This Target
fMRI Recording
Test Stimulus
Clockwise or Counterclockwise?
Visual working memory study: Subject sees both striped disks. The disks disappears. The subject is given a cue, 1 or 2, that tells him which disk to hold in memory. After an 11 second delay, the subject is shown a new striped disk.
Note that every trial has the same stimuli up until the cue for the target (1 or 2). The fMRI differences are not caused by stimulus differences.
Experimenter who does not know which disk the subject was looking at has to predict the orientation of the stripes based on an fMRI image of the subject's visual cortex. 2
Summary: Neuropsych Evidence for WM Components
Psych 355, Miyamoto, Spr '16

29. Neuropsychological Evidence for WM Components
There is a short-term memory store (VSP) that is specific for visual imagery and spatial representations.
Evidence for “spatial rehearsal” in VSP
Single-cell recordings in monkey
Imaging studies of phonological loop:
No one brain locus shows increased activity during rehearsal.
Multiple brain areas that are associated with language show increased activity.
Evidence from cognitive impairments
Table Comparing PL to VSP - Overview of Layout
Psych 355, Miyamoto, Spr '16

30. Summary: Comparison Between PL & VSP
Phonological Loop
Common Characteristics
Differences
Visuospatial Sketchpad
Common Characteristics
Differences
Psych 355, Miyamoto, Win '13
Comparison of PL & VSP - END

31. Summary: Comparison Between PL & VSP
Phonological Loop
Limited capacity
Information is retained by an active process (verbal rehearsal)
Information is highly accessible
Multiple verbal tasks interfere with each other.
PL processes linguistic information;
Visuospatial Sketchpad
Limited capacity
Information is retained by an active process (manipulation of mental imagery)
Information is highly accessible
Multiple visual tasks interfere with each other.
VSP processes visual imagery and spatial information.
Verbal Task + Visual Task produces less interference than Verbal Task + Verbal Task or Visual Task + Visual Task
Different areas of the brain show enhanced activity during verbal and visual/spatial rehearsal.
END
Psych 355, Miyamoto, Spr '16