1. Evidence for a Visuospatial Sketch Pad (VSP) Psychology 355: Cognitive Psychology Instructor: John Miyamoto 4/28 /2015: Lecture 05-2 This Powerpoint presentation may contain macros that were used to create the slides. The macros aren’t needed to view the slides. If necessary, you can disable the macros without any change to the presentation.

2. Outline Brandimonte's Mental Subtraction Experiment Mental Subtraction + Articulatory Suppression Brook's Image Scanning Experiment Interference between VSP & PL Similarities and differences between PL and VSP Psych 355, Miyamoto, Spr '15 2 #

3. 3 Baddeley-Hitch Working Memory (WM) Model 1.Phonological Loop (PL) ♦ Short-term storage ♦ Rehearsal ♦ Manipulation of verbal information 2.Visuospatial Sketch Pad (VSP) ♦ Short-term storage of visual & spatial information ♦ Manipulation of visual images and spatial information. 3.Central Executive o Directs activity within the PL or VSP. o Coordinates activity between PL or VSP, and between these components and long-term memory (LTM). Brandimonte Mental Subtraction Task Next

4. Psych 355, Miyamoto, Spr '15 4 A A' Sample stimuli, A and A' ♦ First, the subject sees A. ♦ Next A disappears and the subject sees A'. Mental Subtraction Task: “Mentally subtract" the second stimulus from the first; then name the object that remains. Brandimonte: Mental "Subtraction" Task Brandimonte, M. A., Hitch, G. J., & Bishop, D. V. M. (1992). Influence of short-term mem­ory codes on visual image processing: Evidence from image transformation tasks. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 157-165. Same Slide with “Remainder” Image After Subtraction

5. Psych 355, Miyamoto, Spr '15 5 A A' Sample stimuli, A and A' ♦ First, the subject sees A. ♦ Next A disappears and the subject sees A'. Mental Subtraction Task: The subject must mentally "subtract" the second stimulus from the first, and name the object that remains. Remainder = ice cream cones. Brandimonte: Mental "Subtraction" Task Brandimonte, M. A., Hitch, G. J., & Bishop, D. V. M. (1992). Influence of short-term mem­ory codes on visual image processing: Evidence from image transformation tasks. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 157-165. Instructions for a Sample Trial in the Mental Subtraction Experiment Remainder after mental subtraction Subject is not shown this image.

6. Psych 355, Miyamoto, Spr '15 6 Subtraction Task: The subject must mentally "subtract" the second stimulus from the first, and name the object that remains. ♦ Possible answer for A and A' on Left: Ice cream cones. ♦ Possible answer for B and B' on Right: Fish Important to note that if the subject names the first image, then it is harder to access a name for the image that remains after subtracting the second image. Summary: Mental "Subtraction" Task Combined with Mental Subtraction with Articulatory Suppression A X B A' B' Y

7. Psych 355, Miyamoto, Spr '15 7 Combine Mental Subtraction with Articulatory Suppression Condition 1: Subject does the mental subtraction task. Condition 2: Subject says "la, la, la, la,...." while doing the mental subtraction task. Finding: Subjects perform BETTER in Condition 2 than in Condition 1. Why? Interpretation of Mental Subtraction Experiment Screen 1 Screen 2 Mental Image (not displayed to subject)

8. Psych 355, Miyamoto, Spr '15 8 Interpretation of Mental Subtraction Experiment The stimuli were designed so that subjects could name the object before "subtracting" the second image. ♦ E.g., with stimulus 1, a subject might think "candy". If a subject silently says "candy" while looking at the first image, it is harder to see "fish" after subtracting the second image. Saying "la, la, la, la,...." suppressed the tendency to name the left part of the stimulus, so the subject relies only on the visual image in VSP. This makes the subtraction task easier. Summary re Mental Subtraction

9. Psych 355, Miyamoto, Spr '15 9 Summary re Mental Subtraction Basic Assumption re VSP: People can maintain a visual/spatial representation of information by actively processing it in VSP. Hypothesis: Suppression of PL can improve processing on tasks if... 1)... people are in the habit of naming the image (recoding initial image into the PL), but..... 2).... the task is actually performed more easily in VSP. Result: Articulatory suppression does improve performance on the mental subtraction task. ♦ This result is hard to explain if STM is a single storage area without separate PL & VSP. Brook's Image Scanning Experiment

10. Psych 355, Miyamoto, Spr '15 10 Next: Brook’s Image Scanning Experiment Brooks, L. R. (1968). Spatial and verbal components of the act of recall. Canadian Journal of Psychology, 22, 349-368. Purpose #1: To show that there are 2 different stores: VSP and PL Purpose #2: To show that... ♦ Performing two different tasks in VSP interfere with each other. ♦ Performing two different tasks in PL interfere with each other. ♦ Performing one task in VSP and a different task in PL does not cause as much interference Image Scanning Experiment

11. Psych 355, Miyamoto, Spr '15 11 Image Scanning (Brooks) Experiment has 4 conditions. ♦ 2 types of stimulus: Diagrams or Sentences ♦ 2 types of responses: Pointing or Vocal Response* Dependent variable = time to perform a task Next: Explain the different stimulus types and response modes Response Mode PointingVocal Diagrams28.211.3 Sentences9.813.8 Stimulus Types * There was actually a third response type, “tapping”, but it will not be discussed here – it turned out not to be very informative. Point Out that Goldstein Textbook Omits the Sentence Stimuli

12. Psych 355, Miyamoto, Spr '15 12 Image Scanning (Brooks) Experiment has 4 conditions. ♦ 2 types of stimulus: Diagrams or Sentences ♦ 2 types of responses: Pointing or Vocal Response* Dependent variable = time to perform a task Next: Explain the different stimulus types and response modes Response Mode PointingVocal Diagrams28.211.3 Sentences9.813.8 Stimulus Types Explain Diagram Stimulus x Vocal Response Goldstein's discussion of this study omits the Sentence stimuli. Only the diagram stimuli are discussed in the textbook.

13. Psych 355, Miyamoto, Spr '15 13 Image Scanning (Brooks) Experiment has 4 conditions. ♦ 2 types of stimulus: Diagrams or Sentences ♦ 2 types of responses: Pointing or Vocal Response* Dependent variable = time to perform a task Next: Explain the different stimulus types and response modes Response Mode PointingVocal Diagrams28.211.3 Sentences9.813.8 Stimulus Types Next Slide Explain Diagram Stimulus x Vocal Response

14. Psych 355, Miyamoto, Spr '15 14 Image Scanning (Brooks, cont.) Condition: Diagrammatic Stimulus & Vocal Response Memorize the figure. Then it is removed. Start your scan at the star and (mentally) move clockwise around the figure. Say "OUT" when you reach an outside corner.. Say "IN" when you reach an inside corner. Correct response: out, out, in, in, out, out, in, out, out, out Dependent variable: Response time (time to complete the task) Explain Diagram Stimulus with Pointing Response

15. Psych 355, Miyamoto, Spr '15 15 Image Scanning (cont.) Next: Diagrammatic stimulus combined with pointing response: Response Mode PointingVocal Diagrams28.211.3 Sentences9.813.8 Stimulus Type Next Slide Previous Slide

16. Psych 355, Miyamoto, Spr '15 16 Condition: Diagrammatic Stimulus & Pointing Response Memorize the figure. Then it is removed. Same as before Start your scan at the * and (mentally) move clockwise around the figure. Same as before. Point to "Out" when you reach an outside corner. Point to "In" when you reach an inside corner.. Correct response: (See diagram) out, out, in, in, out, out, in, out, out, out Dependent variable: Response time (time to complete the task) Image Scanning (Brooks, cont.) Stimulus Respond by pointing to letters on this sheet. Transition to Next Condition: Sentence Stimulus & Vocal Response

17. Psych 355, Miyamoto, Spr '15 17 Image Scanning (cont.) Explain sentence stimulus combined with vocal response: Response Mode PointingVocal Diagrams28.211.3 Sentences9.813.8 Stimulus Type Next Slide Condition: Sentence Stimulus & Vocal Response

18. Psych 355, Miyamoto, Spr '15 18 Image Scanning (Lee Brooks, cont.) Condition: Sentence Stimulus & Vocal Response Memorize the sentence. Then it is removed. Now work from the beginning to the end of the sentence. Say "yes" each time you encounter a noun; say "no" when you encounter a word that is not a noun. Correct response: Dependent variable: Response time (time to complete the task) Sentence Stimulus A bird in hand is worth two in the bush. no, yes, no, yes, yes Transition to Condition with Sentence Stimulus & Pointing Response

19. Psych 355, Miyamoto, Spr '15 19 Image Scanning (cont.) Condition: Sentence Stimulus & Vocal Response Response Mode PointingVocal Diagrams28.211.3 Sentences9.813.8 Stimulus Type Next Slide

20. Psych 355, Miyamoto, Spr '15 20 Image Scanning (Lee Brooks, cont.) Condition: Sentence Stimulus & Vocal Response Memorize the sentence. Then it is removed. Work from the beginning to the end of the sentence. Point to "yes" each time you encounter a noun; say "no" when you encounter a word that is not a noun.. Correct response: no, yes, no, yes, no, no, no, no, no, yes Respond by pointing to letters on this sheet. Sentence Stimulus: A bird in hand is worth two in the bush. Image Scanning Results

21. Psych 355, Miyamoto, Spr '15 21 Response Mode PointingVocal Diagrams28.211.3 Sentences9.813.8 Results: Image Scanning (Lee Brooks) Stimulus Type Faster Slower Faster Same Slide: Why is Pointing Slower than Vocal For Diagrams & Opposite for Sentences?

22. Psych 355, Miyamoto, Spr '15 22 Response Mode PointingVocal Diagrams28.211.3 Sentences9.813.8 Why isdiagram/pointingslower thandiagram/vocal? Why issentence/vocalslower thansentence/pointing? Results: Image Scanning (Lee Brooks) Stimulus Type Faster Slower Faster Discussion of Results

23. Psych 355, Miyamoto, Spr '15 23 Response Mode PointingVocal Diagrams28.211.3 Sentences9.813.8 Diagram/Pointing: The stimulus and response compete for a common cognitive resource, representation in VSP. Information processing bottleneck causes slower response. Diagram/Vocal: Stimulus & response use separate cognitive resources. No bottleneck. Why is Diagram/Pointing Slower than Diagram/Vocal? Stimulus Type Slower Faster Why is sentence/vocal slower than sentence/pointing?

24. Psych 355, Miyamoto, Spr '15 24 Response Mode PointingVocal Diagrams28.211.3 Sentences9.813.8 Sentence/Vocal: Again, stimulus and response compete for a common cognitive resource, representation in PL. Information processing bottleneck causes slower response. Sentence/Pointing: Stimulus & response use separate cognitive resources. No bottleneck. Why is Sentence/Vocal Slower than Sentence/Pointing? Stimulus Type xxx Faster Slower

25. Psych 355, Miyamoto, Spr '15 25 Brooks' experiment supports the existence of a visuospatial sketch pad (VSP) because.... If there were only one STM store (not separate PL & VSP), then the effect of the pointing response would be the same in the visuospatial task and the verbal task. Similarly for the effect of the verbal response. WM model assumes multiple memory stores – this lets WM predict conflicts between similar mental codes. The preceding experiment illustrates a basic principle: A response in one modality (verbal or spatial) will interfere more with a memory representation in the same modality than with a memory representation in a different modality. Summary - END

26. Summary: Comparison Between PL & VSP Phonological Loop Common Characteristics Differences Visuospatial Sketchpad Common Characteristics Differences Psych 355, Miyamoto, Win '13 26 Comparison of PL & VSP - END

27. Summary: Comparison Between PL & VSP Phonological Loop Limited capacity Information is retained by an active process (verbal rehearsal) Information is highly accessible Multiple verbal inputs interfere with each other. PL processes linguistic information; Visuospatial Sketchpad Limited capacity Information is retained by an active process (manipulation of mental imagery) Information is highly accessible Multiple visual inputs interfere with each other. VSP processes visual imagery and spatial information. Psych 355, Miyamoto, Spr '15 27 Verbal processing does not interfere as much with visual processing Different areas of the brain show enhanced activity during verbal and visual/spatial rehearsal. END