1. Review: The Biological Basis of Audition Recanzone and Sutter Presented by Joseph Schilz

2. Outline Introduction Organization of Audition Auditory Spatial Processing Interactions with visual stimuli Ventriloquism effect Ventriloquism after-effect Auditory Temporal Processing Temporal integration Forward masking Gap detection

3. Introduction Audition and vision major sensory systems Audition does not have the significant history of research that vision does Differences between audition and vision Tasks in decoding audition: Where a sound came from Spectral properties Temporal properties Identifying what sound represents Review's focus: location and temporal properties

4. Organization of Audition Fig 1. Recanzone and Sutter

5. Organization of Audition Fig 2. Recanzone & Sutter

6. Organization of Audition Fig 2. Kalatsky et al. Tonotopy

7. Organization of Audition Fig 2: Bitterman et al. Tonal Tuning

8. Organization of Audition Fig 2. Bizley & King Tonal/Spatial Tuning

9. Organization of Audition Fig 1: Design Ahveninen et al. “What” and “Where” Paths?

10. Organization of Audition Fig 2: Results Ahveninen et al. “What” and “Where” Paths?

11. Auditory Spatial Processing Cues Interactions with visual stimuli Ventriloquism effect Ventriloquism after-effect

12. Auditory Spatial Processing Fig 3. Recanzone and Sutter Cues

13. Auditory Spatial Processing Fig 1. Yu and Young Cues

14. Auditory Spatial Processing Ventriloquism effect Definition Cognitive factors “unity assumption” Non-cognitive factors Timing Compellingness Spatial discrepancy/agreement

15. Auditory Spatial Processing Ventriloquism effect Early studies assumed that the more precise modality would “capture” the less precise modality. Later studies showed a near optimal “mixing” of modality reports, respecting the measure- error variance of each modality.

16. Auditory Spatial Processing Ventriloquism effect Fig 2: Design and Results Kording et al.

17. Auditory Spatial Processing Ventriloquism after-effect If subject is presented with audio/visual stimuli of a consistent spatial disparity, subjects spatial perception of acoustic space shifted after Long lasting Does not transfer across frequencies Different from other adaption illusions: lasts tens of minutes, does not transfer across frequencies, in the same direction of adapting stimulus.

18. Auditory Spatial Processing Ventriloquism after-effect Fig 4: Design and Results Recanzone

19. Auditory Spatial Processing Ventriloquism after-effect Difficulty of single unit recording in illusions. Direct projection from auditory to visual (observed in primates)‏ Direct projection from visual to auditory (observed in ferrets) Several areas in brain with multisensory response Ghanzafar study

20. Auditory Temporal Processing Definition Temporal integration Forward masking Gap detection

21. Auditory Temporal Processing Definition Can mean processing of temporal aspects of stimuli or ability of neurons to encode stimulus by temporal aspects of firing We refer to former Temporal processing could be interpreted to include spectral processing We don't consider spectral processing here

22. Auditory Temporal Processing Temporal integration Our environments are noisy; audition might wait a bit and let noise average itself out before passing on a percept On the other hand, some decisions need to be made quickly; audition shouldn't hold onto information for too long How to assess? At what levels is this occurring?

23. Auditory Temporal Processing Temporal integration Fig 4. Dallos and Olsen

24. Auditory Temporal Processing Temporal integration Varies with loudness, frequency, duration Bloch's Law (Loudness x Duration = k)‏ Leaky integrator model Mean integration times of 30-40msec in humans

25. Auditory Temporal Processing Clock et al. found similar constants of integration in chinchilla cochlear nucleus neurons and chinchilla behavior Exponential leaky integrator fit model well But, auditory nerve neurons had time constants much larger Explanation Temporal integration

26. Auditory Temporal Processing Forward masking Two sounds presented sequentially, with some gap, sometimes subject will not perceive second sound. Depends on many factors Generally measured as a function of first sound's duration. Fig 4. Recanzone and Sutter

27. Auditory Temporal Processing Forward masking Likely a result of adaptation.

28. Auditory Temporal Processing Gap detection Temporal resolution vs. temporal integration One paradigm for temporal resolution: gap detection Humans able to detect gaps in noise as small as 1-2msec Auditory nerve firing shows gaping pattern Likely some role of cortex in detecting gaps, as shown by lesion, deactivation studies

29. Auditory Temporal Processing Gap detection Fig 2. Zhang et al.

30. Additional Works Referenced Ahveninen et al. Task-modulated “what” and “where” pathways in human auditory cortex PNAS 2006 103 (39) 14608-14613; published ahead of print September 18, 2006, doi:10.1073/pnas.0510480103 Y. Bitterman, R. Mukamel, R. Malach, I. Fried, & I. Nelken Ultra-fine frequency tuning revealed in single neurons of human auditory cortex Nature 451, 197-201 (10 January 2008)‏ Jennifer K. Bizley, Andrew J. King, Visual-auditory spatial processing in auditory cortical neurons, Brain Research, Volume 1242, Multisensory Integration, 25 November 2008, Pages 24-36, ISSN 0006-8993, DOI: 10.1016/j.brainres.2008.02.087. P. Dallos, W. Olsen, Integration of energy at threshold with gradual rise-fall tone pips. Journal of the Acoustical Soc. of America.Vol. 36, pp. 743-751, April 1964 V Kalatsky, D Polley, M Merzenich, C Schreiner, Mstryker, Fine functional organization of auditory cortex revealed by Fourier optical imaging PNAS 2005 102 (37) 13325-13330; published ahead of print September 1, 2005, doi:10.1073/pnas.0505592102

31. Additional Works Referenced Kording KP, Beierholm U, Ma WJ, Quartz S, Tenenbaum JB, et al (2007) Causal Inference in Multisensory Perception. PLoS ONE 2(9): e943. doi:10.1371/journal.pone.0000943 Recanzone G, Rapidly induced auditory plasticity: The ventriloquism aftereffect. Proc. Natl. Acad. Sci. USA Vol. 95, pp. 869–875, February 1998 J Yu, E Young, Linear and nonlinear pathways of spectral information transmission in the cochlear nucleus PNAS 2000 97 (22) 11780-11786 W. Zhang, R.J. Salvi, S.S. Saunders, Neural correlates of gap detection in auditory nerve fibers of the chinchilla, Hearing Research, Volume 46, Issue 3, July 1990, Pages 181-200, ISSN 0378-5955, DOI: 10.1016/0378-5955(90)90001-6.