1. Vienna 4/28/2006 1 Click vs. click-click vs. blink-click: Factors influencing human sound localization in the horizontal plane Norbert Kopčo TU Košice Dept. of Cybernetics and AI Boston University Hearing Research Center Dartmouth College Center for Cognitive Neuroscience

2. Vienna 4/28/2006 2 Intro: Sound localization 3-dimmensional: azimuth, elevation, distance depends on: stimulus type: spectrum, temporal aspects environment: anechoic, reverberant source movement: static, dynamic presence of other stimuli (auditory or visual) a priori knowledge / expectations about the scene

3. Vienna 4/28/2006 3 Effect of additional stimuli The extra sound can act as a: Masker – localization worse Adaptor – localization biased (Attraction/Repulsion) Real sound (of which the target is a reflection) – localization worse/suppressed Perceptual stream of which the target is or is not a part Cue – localization better (doesn’t have to be auditory) Anchor – change localization strategy

4. Vienna 4/28/2006 4 Effect of additional sounds Temporal relations studied previously: Extra sound precedes target by: 10 secs to minsAdaptation/Repulsion 50 msecs to 1 secAdaptation/ reflections 4 – 40 msecs Precedence Concurrent soundsAdaptation/Repulsion Inverse orderBackward masking

5. Vienna 4/28/2006 5 Auditory Pathway and Spatial Hearing Cochlea – peripheral filtering and neural coding Olivary complex – processing of binaural information Thalamus (Inferior Colliculus) – integration, modulation detection Auditory Cortex – auditory object formation, figure/ground separation, ASA Posterior Parietal Cortex – supramodal spatial representation & attentional modulation

6. Vienna 4/28/2006 6 Current goal Begin to understand auditory localization in a more complex scene: -when target is preceded by another identical sound/s from a known location that the listener should ignore (Exp 1) -when target is preceded by visual or auditory cue that allows the listener to direct spatial attention (Exp 2) -when a concurrent visual stimulus induces a shift in auditory perception / ventriloquism (Exp 3)

7. Vienna 4/28/2006 7 Experiment 1 Perceptual and central effects in sound localization with a preceding distractor (aka Click vs. Click-click vs. Click-click-click-click...) Collaborators Barbara Shinn-Cunningham, Virginia Best Hearing Research Center Boston University

8. Vienna 4/28/2006 8 Exp 1 - Preceding distractor: Intro Several preceding studies indicated that preceding stimulus influences localization at SOAs of several hundreds milliseconds (Kopco et al., 2001, Perrott and Pacheco, 1989) Goal: -Characterize this influence (bias and std.dev. in responses) -Determine its cause. Candidates: -short-term adaptation in brainstem representations -reverberation suppression and acoustics -strategy -perceptual organization -attention: focused away from distractor location

9. Vienna 4/28/2006 9 Exp 1 - Preceding distractor: Hypotheses Peripheral factors will have short-term effects Central factors will influence results at longer separations Effect of reverberation can be separated by comparing performance in anechoic and echoic rooms Effect of perceptual organization can be addressed by modifying the stimuli

10. Vienna 4/28/2006 10 Exp 1 - Preceding distractor: Methods Anechoic room or a classroom Blocks of trials with fixed distractor location Trials with SOAs of 25,50,100,200 or 400 ms interleaved w/ no distractor trials Seven subjects

11. Vienna 4/28/2006 11 Exp 1 - Preceding distractor: Results Complex pattern of biases and standard deviation effects observed Four main effects in terms of bias discussed Bias 1: Lateral bias for frontal targets and lateral distractor in room

12. Vienna 4/28/2006 12 Exp 1 - Preceding distractor: Results – Bias 1 Largest effect Strongest at short SOAs No comparable effect of frontal distractor ROOM

13. Vienna 4/28/2006 13 Exp 1 - Preceding distractor: Results – Bias 1 Effect eliminated in anechoic room  has to do with reverberation. Acoustic or neural interaction? ANECHOIC ROOM

14. Vienna 4/28/2006 14 Exp 1 – Bias 1: Perceptual organization Effect not due to acoustics because correct representation is available ROOM: click-click- click-click … click ROOM: Click- click

15. Vienna 4/28/2006 15 Exp 1 – Bias 1: Standard deviation The largest increase in standard deviation corresponds with the largest bias  Neural suppression along with reflections BUT: Why only lateral distractor?

16. Vienna 4/28/2006 16 Exp 1 - Preceding distractor: Results – Bias 2 Targets in the middle of the range are attracted by the distractor, independent of: -Environment -Distractor location -Only at short SOAs  Interactions in low-level spatial maps (brainstem)

17. Vienna 4/28/2006 17 Exp 1 - Preceding distractor: Results – Bias 3 Lateral targets are repulsed by lateral distractors -Independent of SOA -Independent of environment Probably central effect: e.g., change in response strategy, using distractor as an anchor w/ known location Not in front because of higher resolution.

18. Vienna 4/28/2006 18 Exp 1 - Preceding distractor: Context There is bias also in the no-distractor responses The bias is always away from the non-present distractor Because the runs were interleaved, this bias had to build up anew during each run

19. Vienna 4/28/2006 19 Exp 1 - Preceding distractor: Context Difference in no-distractor responses in the frontal and lateral distractor context -Is independent of azimuth -Grows over time -Slightly stronger for the 8-click train context Contextual plasticity on time scale of minutes Similar to effects of long-term exposure Either due to bottom-up factors (distribution of stimuli) or top- down factors (focusing away from distractor) Contextual bias

20. Vienna 4/28/2006 20 Exp 1 - Preceding distractor: Summary A preceding distractor coming from a known location -Induces a complex pattern of biases -Over a range of time scales -Probably caused at different stages in the spatial auditory processing pathway

21. Vienna 4/28/2006 21 Experiment 2 Modality-dependant attentional control in human sound localization (aka Click vs. Beep-click vs. Blink-click) In collaboration w/ students Beáta Tomoriová, Rudolf Andoga, Martin Bernát Perception and Cognition Lab Technical University, Košice

22. Vienna 4/28/2006 22 Exp 2 – Uni-/Cross-modal attention: Intro Several studies explored the question whether directing automatic or strategic attention by an auditory cue can improve sound localization (Spence & Driver, 1994; Sach, 2000; Kopco & Shinn-Cunningham, 2003) Results: improvements in RTs (Spence&Driver), but small (Sach) or no (Kopco) improvements in performance Possible reason: the SOAs too short to orient attention Goal: -determine whether attentional effects occur at longer SOAs -compare the effect of a visual and auditory cue

23. Vienna 4/28/2006 23 Exp 2 – Uni-/Cross-modal attention: Hypotheses No effect of automatic attention (previous studies) Strategic attention will affect performance at long SOAs Effect modality-independent because spatial cuing very coarse (only left vs. right)

24. Vienna 4/28/2006 24 Exp 2 – Uni-/Cross-modal attention: Methods Virtual auditory environment Target – broadband click Cue indicates side of target: -visual (arrow on a computer screen) - auditory (monaural tone) -SOA: 400, 800, 1600 ms -Informative: 100%, 80%, 50% validity -analysis: mean and s.d. in responses

25. Vienna 4/28/2006 25 Exp 2 – Uni-/Cross-modal attn: Results - bias Mean effect of auditory cue (averaged across target azimuth): - Invalid cues cause medial bias, fairly independent of SOA - Valid cues cause similar medial bias

26. Vienna 4/28/2006 26 Exp 2 – Uni-/Cross-modal attn: Results - bias When cue modality is visual: - Invalid cues cause medial bias, similar to the auditory cues - Valid cues cause lateral bias that grows with SOA Modality through which expectation of the target location is controlled influences the perceived location

27. Vienna 4/28/2006 27 Exp 2 – Uni-/Cross-modal attn: Results – s.d. Effect in terms of standard deviation: No effect of auditory cue Visual cue never improves performance, but invalid cue at 1600 ms increases s.d. Summary: Cuing doesn’t improve performance Expectation of side of stimulus induces bias in a modality dependent way Might have something to do with the coordinate systems in which visual and auditory space are represented

28. Vienna 4/28/2006 28 Experiment 3 Behavioral examination of the auditory spatial coordinate system using the ventriloquism effect Collaboration Jennifer Groh Center for Cognitive Neuroscience, Dept of Psychological and Brian Sciences, Dartmouth College Barbara Shinn-Cunningham, I-Fan Lin Boston University

29. Vienna 4/28/2006 29 Exp 3 – Coordinate system of auditory space: Intro Mullette-Gillman et al. (2005): Does the visual and auditory spatial coding have the same reference frame in the monkey parietal cortex? Is the frame head-centered (as in auditory periphery) or eye- centered (as in visual periphery)? This is an issue only for primates and animals that can move their eyes (not barn owls) Result: some neurons in PPC A-only, some V-only, some AV, some head-centered, some eye-centered

30. Vienna 4/28/2006 30 Exp 3 – Coordinate system of auditory space: Intro Here, use the ventriloquism effect to address a similar question behaviorally in monkeys and in humans: Is the coordinate system at which the auditory behavioral responses are determined head- or eye-centered? Method: 1.Induce a local shift in the auditory spatial map for a fixed eye position. 2.Move eyes to a new position. 3.If the region of the shift doesn’t change  head-centric 4.Otherwise, eye-centric coordinate system

31. Vienna 4/28/2006 31 Exp 3 – Coordinate system of auditory space: Method Study performed in humans and in monkeys Monkey data here

32. Vienna 4/28/2006 32 Exp 3 – Coordinate system of auditory space: Results (preliminary) Difference between positive (rightward) and negative (leftward) shifts induced in the central region with left fixation point and generalization tested with right fixation point Result: Induced shift generalizes on the right side No shift in bias due to change in fixation point  Head-centric coordinate system

33. Vienna 4/28/2006 33 Overall summary Three experiments explored various aspects of horizontal sound localization Understanding is limited even in the simple auditory scenes studied Need follow-ups to clarify results

34. Vienna 4/28/2006 34 Acknowledgements US National Institutes of Health (PIs: Shinn- Cunningham and Groh) US National Academy of Sciences (Shinn- Cunningham, Kopčo) Slovak Scientific Grant Agency (Kopčo)

35. Vienna 4/28/2006 35 Distance perception in reverberant environments - is consistent experience necessary for accurate distance perception? - also, studies looking at other parameters (mono- vs. binaural, anechoic vs. reverberant, real vs. simulated environments) “Room learning” and calibration to its acoustic properties - is localization accuracy and “room learning” affected by changes in listener position in a room? - do speech perception mechanisms calibrate to different acoustic environments? Spatial release from masking - effect of signal and masker location on detectability/intelligibility of pure tones, broadband non-speech stimuli, and speech in anechoic and reverberant environments Overview of recent studies of binaural and spatial hearing