1. Multisensory Perception and Biological Motion
Ayse Pinar Saygin, Margarita Sarri, Jon Driver
Institute of Cognitive Neuroscience
BUCNI Project Presentation Nov

2. Background
Recent interest in audio-visual multisensory interplay (e.g. Noesselt/Driver, 2007, J Neurosci; STS for matching temporal patterns with flashes and beeps)
Little known about multisensory processing for more complex or ecological stimuli (other than speech)

3. Background
Recent interest in audio-visual multisensory interplay (e.g. Noesselt/Driver, 2007, J Neurosci; STS for matching temporal patterns with flashes and beeps)
Little known about multisensory processing for more complex or ecological stimuli (other than speech)
Biological motion - activates STS
Seen movements often accompanied by sounds
We recently showed behavioural effects of heard ‘footsteps’ during dot-walker perception (Saygin, Driver & de Sa, Psychological Science, in press)
Single cells in the STS that are responsive to biological motion show increased activity when corresponding sounds are present

4. Design fMRI of audio-visual interplay in biological motion
Visual Stimuli: dot-jogger or same dots scrambled
Auditory Stimuli: “footstep” sounds in or out of synch with ‘foot’ dots
Visual noise: Principle of inverse effectiveness
Demo.

5. Proposal N=16 subjects, 6 runs of ~ 8 minutes
Event related, whole brain coverage, TR=2200 ms.
To be analyzed with SPM
Conditions
Biological/Scrambled (plus noise dots)
Matching/Mismatching sound
Task: Decide whether jogger is present
Unisensory (Biological, Scrambled, Just Sound, Fixation)
Predictions: Expected in the STS, possibly IFG, unisensory areas
AV_Congruent > AV_Incongruent
AV > A, V with Congruent sound
Possibly AV < A, V with Incongruent sound
Interaction with Visual Stimulus type: Multisensory effect greater for Biological compared with Scrambled