Visually-induced auditory spatial adaptation in monkeys and humans Norbert Kopčo, I-Fan Lin, Barbara Shinn-Cunningham, Jennifer Groh Center for Cognitive.

Slides:

Advertisements

Similar presentations

Chapter 5: Space and Form Form & Pattern Perception: Humans are second to none in processing visual form and pattern information. Our ability to see patterns.

Advertisements

Electrophysiology of Visual Attention. Does Visual Attention Modulate Visual Evoked Potentials? The theory is that Visual Attention modulates visual information.

Visual Attention Attention is the ability to select objects of interest from the surrounding environment A reliable measure of attention is eye movement.

Chapter 6: Visual Attention. Overview of Questions Why do we pay attention to some parts of a scene but not to others? Do we have to pay attention to.

Pre-frontal cortex and Executive Function Squire et al Ch 52.

Effects of Sound and Visual Congruency on Product Selection and Preference Brock Bass Felipe Fernandez Drew Link Andrew Schmitz.

Saccadic eye movements cause compression of time as well as space Concetta Morrone, Ross & Burr.

Visually-induced auditory spatial adaptation in monkeys and humans Norbert Kopčo, I-Fan Lin, Barbara Shinn-Cunningham, Jennifer Groh Center for Cognitive.

Visually-induced auditory spatial adaptation in monkeys and humans Norbert Kopčo Center for Cognitive Neuroscience, Duke University Hearing Research Center,

Electrophysiology of Visual Attention. Moran and Desimone (1985) “Classical” RF prediction: there should be no difference in responses in these two conditions.

Test Oct. 21 Review Session Oct 19 2pm in TH201 (that’s here)

Attention as Information Selection. Early Selection Early Selection model postulated that attention acted as a strict gate at the lowest levels of sensory.

ICA Madrid 9/7/ Simulating distance cues in virtual reverberant environments Norbert Kopčo 1, Scott Santarelli, Virginia Best, and Barbara Shinn-Cunningham.

What determines the reference framework for stereopsis? Andrew Glennerster, Martin Birch and Suzanne McKee* University Laboratory of Physiology, Oxford.

Visual Imagery One of the greatest problems confronting psychology is the nature of mental representation. Part of this debate is the nature of representations.

Effect of roving on spatial release from masking for amplitude-modulated noise stimuli Norbert Kopčo *, Jaclyn J. Jacobson, and Barbara Shinn-Cunningham.

Vienna 4/28/ Click vs. click-click vs. blink-click: Factors influencing human sound localization in the horizontal plane Norbert Kopčo TU Košice.

Active Vision Carol Colby Rebecca Berman Cathy Dunn Chris Genovese Laura Heiser Eli Merriam Kae Nakamura Department of Neuroscience Center for the Neural.

Visually-induced auditory spatial adaptation in monkeys and humans Norbert Kopčo, I-Fan Lin, Barbara Shinn-Cunningham, Jennifer Groh Center for Cognitive.

Spectral centroid 6 harmonics: f0 = 100Hz E.g. 1: Amplitudes: 6; 5.75; 4; 3.2; 2; 1 [(100*6)+(200*5.75)+(300*4)+(400*3.2)+(500*2 )+(600*1)] / = 265.6Hz.

The Positional Acuity of the Human Visual System Year 2 Practical Class Dr. Paul McGraw & Mr. Craig Stockdale.

Audiovisual Multisensory Facilitation: A Fresh Look at Neural Coactivation and Inverse Effectiveness. Lynnette Leone North Dakota State University.

The Effect of Object Size and Speed on Time to Collision Estimation in the Horizontal Plane Douna Montazer, Valdeep Saini, Nicole Simone, Danielle Thorpe.

Visual Adaptation And Spatial Auditory Processing Peter Lokša, Norbert Kopčo Collaborators: Jenni Groh, Barb Shinn-Cunningham, I-Fan Lin.

Attention Modulates Responses in the Human Lateral Geniculate Nucleus Nature Neuroscience, 2002, 5(11): Presented by Juan Mo.

THE VISUAL SYSTEM: EYE TO CORTEX Outline 1. The Eyes a. Structure b. Accommodation c. Binocular Disparity 2. The Retina a. Structure b. Completion c. Cone.

Sound Localization Strategies in Simple and Complex Environments Norbert Kopčo Department of Cybernetics and AI, TU Košice, Slovakia Hearing Research Center.

Week 6: (March 15, 2011) Auditory Maps and orienting: need for Coordinate Transformations.

Visual calibration, uncertainty, and attention in spatial auditory processing Norbert Kopčo Dept. of Cybernetics and AI, Technical University of Košice,

Active Vision: Memory, Attention and Spatial Representation in Parietal Cortex Carol Colby Rebecca Berman Cathy Dunn Chris Genovese Laura Heiser Eli Merriam.

Hearing Research Center

Zatorre et al. Nature Neuroscience, 2002 These animations illustrate the stimulus sequences used in Experiments 1 and 2. For each experiment, the animation.

A new neural framework for visuospatial processing Group #4 Alicia Iafonaro Alyona Koneva Barbara Kim Isaac Del Rio.

Neural Basis of the Ventriloquist Illusion Bonath, Noesselt, Martinez, Mishra, Schwiecker, Heinze, and Hillyard.

Experimental Psychology PSY 433

Evaluating Perceptual Cue Reliabilities Robert Jacobs Department of Brain and Cognitive Sciences University of Rochester.

Binding problems and feature integration theory. Feature detectors Neurons that fire to specific features of a stimulus Pathway away from retina shows.

Spatial coding of the Predicted Impact Location of a Looming* Object M. Neppi-Mòdona D. Auclair A.Sirigu J.-R. Duhamel.

Visually-induced auditory spatial adaptation in monkeys and humans Norbert Kopčo, I-Fan Lin, Barbara Shinn-Cunningham, Jennifer Groh Center for Cognitive.

Aligning Vision and Proprioception 2 Inactivation of PMv or PMd with small injections of muscimol did not disrupt reaching movements significantly in the.

 Example: seeing a bird that is singing in a tree or miss a road sign in plain sight  Cell phone use while driving reduces attention and memory for.

On the improvement of virtual localization in vertical directions using HRTF synthesis and additional filtering Wersényi György SZÉCHENYI ISTVÁN UNIVERSITY,

Outline Of Today’s Discussion 1.Some Disparities are Not Retinal: Pulfrich Effect 2.Random-Dot Stereograms 3.Binocular Rivalry 4.Motion Parallax.

Cortical Representations of Attention and Salience James A. Mazer Department of Neurobiology Yale School of Medicine Theoretical and Experimental Approaches.

Tonal Violations Interact with Lexical Processing: Evidence from Cross-modal Priming Meagan E. Curtis 1 and Jamshed J. Bharucha 2 1 Dept. of Psych. & Brain.

$ studying barn owls in the laboratory $ sound intensity cues $ sound timing cues $ neural pathways for sound location $ auditory space $ interaural time.

Fletcher’s band-widening experiment (1940)

The owl Ecology and behavior

Effective Communication Techniques. Interest Approach Give each student a copy of a relevant news article. Explain the importance of skimming and scanning.

Ten participants made bimanual reaching movements with a movement time goal of 500 ms. The trajectories of the hands were recorded with an Optotrak with.

Visual Attention (neural mechanisms) Arash Afraz.

Auditory Localization in Rooms: Acoustic Analysis and Behavior

Visual Influences on Echo Suppression

Backward Masking and Unmasking Across Saccadic Eye Movements

Active Vision Carol Colby Rebecca Berman Cathy Dunn Chris Genovese

Visually-induced auditory spatial adaptation in monkeys and humans

Minami Ito, Gerald Westheimer, Charles D Gilbert Neuron

Alteration of Visual Perception prior to Microsaccades

Vincent B. McGinty, Antonio Rangel, William T. Newsome Neuron

Attentional Modulations Related to Spatial Gating but Not to Allocation of Limited Resources in Primate V1 Yuzhi Chen, Eyal Seidemann Neuron Volume.

Learning to Link Visual Contours

Eye Movements Modulate Visual Receptive Fields of V4 Neurons

The Ventriloquist Effect Results from Near-Optimal Bimodal Integration

Ryo Sasaki, Takanori Uka Neuron Volume 62, Issue 1, Pages (April 2009)

Spatiotopic Visual Maps Revealed by Saccadic Adaptation in Humans

Jennifer K. Bizley, Ross K. Maddox, Adrian K.C. Lee

Neural Basis of the Ventriloquist Illusion

Stan Van Pelt and W. Pieter Medendorp

Social Information Signaling by Neurons in Primate Striatum

The Postsaccadic Unreliability of Gain Fields Renders It Unlikely that the Motor System Can Use Them to Calculate Target Position in Space Benjamin Y.

Presentation transcript:

Visually-induced auditory spatial adaptation in monkeys and humans Norbert Kopčo, I-Fan Lin, Barbara Shinn-Cunningham, Jennifer Groh Center for Cognitive Neuroscience, Duke University Hearing Research Center, Boston University Technical University, Košice, Slovakia

2Nov 6, 2007SFN 07 San Diego Introduction Vision affects auditory spatial perception (e.g. the ventriloquist effect) Visually-induced shifts in sound localization can persist in the absence of visual stimuli (e.g. barn owl prism adaptation studies)

3Nov 6, 2007SFN 07 San Diego QUESTION How does vision calibrate sound perception in primates? - monkeys and humans Unlike barn owls, monkeys and humans make eye movements. With every eye movement, the relationship between visual space and auditory space changes.

4Nov 6, 2007SFN 07 San Diego Visual and auditory spatial information are different! VISION: Retina provides “map” of object locations Locations shift when eyes move Frame of reference is “eye- centered”

5Nov 6, 2007SFN 07 San Diego Visual and auditory spatial information are different! AUDITORY: Sound location calculated from interaural timing and level differences Cue values do NOT shift when eyes move Frame of reference is “head-centered”

6Nov 6, 2007SFN 07 San Diego Goals Does visual-calibration of auditory space occur in eye-centered, head-centered, or a hybrid coordinate system? Are humans and monkeys similar?

7Nov 6, 2007SFN 07 San Diego Experimental Setup Audiovisual display as viewed by the subject Horizontal location (degrees) Vertical location (degrees) 9 speakers in front of listener (~1 m distance), separated by 7.5° (humans) or 6° (monkeys) Light-emitting diodes (LEDs) at three center speakers: - aligned with speakers, or - displaced to the left or to the right (by 5°-humans, 6°-monkeys) 2 LEDs below speaker array used as fixation points (FP) Stimuli: Auditory stimulus: 300-ms broadband noise burst Audio-Visual stimulus: Same noise with synchronously lid LED.

8Nov 6, 2007SFN 07 San Diego Experiment: Data Presentation Format Sample Stimulus-Response Raw Data Plot Actual Stimulus Location (°) Perceived / Responded Location (°) Sample Plot Showing Bias in Responses (Response – Actual Location) Actual Stimulus Location (°) Bias in Responses (°) Rightward bias Leftward bias Rightward bias Leftward bias

9Nov 6, 2007SFN 07 San Diego Experiment: Hypothesis 1. Induce Ventriloquist Effect for AV stimuli presented in central sub- region of space, keeping fixation point on the right ( +8°) 2. Test that adaptation results in the effect being present also for Auditory-only stimuli (same FP). 3. Induced shift will decrease outside the trained sub-region (for the same FP). 4. Move FP to the left. Test effect on Auditory-only stimuli: - No change  Head-centered. - Response pattern moves with eyes  Eye-centered. - Combined representations. Stimulus Location (°) Bias (°) Audiovisual display Expected Responses FP LEDs Speakers

10Nov 6, 2007SFN 07 San Diego Experiment: Procedure Audiovisual display as viewed by the subject Horizontal location (degrees) Vertical location (degrees) One trial consists of: 1. Fixation point (FP) appears. 2. Subject fixates FP. 3. Target stimulus is presented (Audio-Visual or Auditory-only). 4. Subject saccades to perceived location of stimulus (humans instructed to always saccade to sound). 5. Monkeys only: Reward for responding within a criterion window (+- 10° from speaker).

11Nov 6, 2007SFN 07 San Diego Experiment: Procedure Experiment divided into 1-hour blocks (12 for humans, 16 for monkeys; 7 humans, 2 monkeys). Within a block three types of trials, randomly interleaved: Three types of AV stimuli (AV stimulus type kept constant within a block. Data also collected with AV FP on left. These data mirror-flipped to simplify presentation. For monkeys, also AV- aligned stimuli at +-30° to enforce non-linearity): AV stimuli: 50 % FP LEDs Speakers A-only,trained FP: 25%A-only,shifted FP: 25% AV stimuli aligned Hypometric shift Hypermetric shift

12Nov 6, 2007SFN 07 San Diego Results: Humans Stimulus Location (°) Audiovisual display Expected Responses FP LEDs Speakers Human Behavior Data collapsed across direction of induced shift AV responses: - as expected Trained FP A-only responses: - Shift induced in trained sub-region - Generalization to untrained regions (asymmetrical) Shifted FP A-only responses: - Shift reduced in center region Head-centered representation, modulated by eye position Mean+SE

13Nov 6, 2007SFN 07 San Diego Results: Humans vs. Monkeys Audiovisual display FP LEDs Speakers Human Behavior Monkey data (only hypometric) AV responses: - as expected Trained FP A-only responses: - Shift in trained sub- region weaker - Generalization to untrained regions stronger (asymmetry opposite to humans) Shifted FP A-only responses: - Shifted with eyes Representation more eye-centered Mean+SE Monkey Behavior Mean Mean+individuals Mean+SE Mean+individuals

14Nov 6, 2007SFN 07 San Diego Summary Good news – the main results are consistent across species: Locally induced ventriloquist effect results in short-term adaptation, causing shifts in responses to A-only stimuli from trained sub-region. The pattern of induced shift is modified as the eyes move. Bad news – there is a lot of differences between species: HumansMonkeys Representationhead-centered, eye-modulatedeye-centered Generalization to untrained sub-regionsmore on the side away from FPopposite Difference between hyper- and hypometric shiftsnoyes Representation when shift induced on side (data not shown)head-centered, no eye modulationeye-centered

15Nov 6, 2007SFN 07 San Diego Discussion The differences between species can be caused by: 1. differences in neural representation and/or learned behavior E.g., monkeys often respond by double saccades 2. differences in the stage along the pathway at which the calibration occurs (Insert figure of ear->CN->SOC->IC->MGB->Aud. Ctx->Parietal Ctx->SC->oculomotor) E.g., in humans, the head-centered shift was induced fast, while the eye-centered modulation was much slower  different time scales may imply adaptation at different stages 3. procedural differences (e.g., monkeys work for water) Future work: Humans: examine temporal and spatial factors influencing the eye-centered modulation. Monkeys: attempt to induce stronger effect using larger A-V separation.