Studies of Information Coding in the Auditory Nerve Laurel H. Carney Syracuse University Institute for Sensory Research Departments of Biomedical & Chemical Engineering and Electrical Engineering & Computer Science PhysiologyPsychophysicsModeling

Outline Background - Siebert’s Analytical Studies of Coding in the Auditory-Nerve –Rate-Place(frequency) Model –All-Information Model (Temporal & Rate cues) Extending the approach with Computation Examples : –Freq. Discrimination (tones) –“Formant” Freq. Discrimination –Level Discrimination (tones) From Ideal Observers to more ‘Realistic’ Models: – Coincidence Detectors for Level Decoding that combine Rate & Temporal Information

Coding of Sound in Auditory Nerve : Tuning Curves suggest a “Rate vs. Place” code But….. (After Kiang)

Saturation of rate is a problem for the rate-place encoding scheme Rate is not adequate to encode stimulus energy at the fiber’s CF. Note: as Rate  ’s, Variability  ’s

Additional information is present in the timing of AN responses

Siebert (‘68,‘70): Can the Limits of Human Perception for  Frequency and  Level be explained by basic properties of Auditory-Nerve responses? Log Frequency dB SPL - Analytical model - Simple tuning; Place map - Saturating rate-level functions - Steady-state responses - Phaselocking included (results limited to low freqs) - Random nature of AN responses described by Non- homogeneous Poisson process

(from Heinz et al., 2001, Neural Computation) Siebert’s Approach Applied to Frequency Discrimination

Use of Cramer-Rao Bound to estimate jnd [1/ variance] can be summed over population of fibers (assuming independence between fibers) Discrimination Threshold, or Just-Noticeable Difference (jnd), corresponds to difference in parameter of interest that equals standard deviation. Lower Bound on variance of frequency estimate [based on r(t)] depends on rate (Poisson assumption) and on partial derivative of rate w.r.t. parameter of interest

Comparison of Siebert’s Predictions to Human Performance: Frequency Discrimination Rate-Place All-Info Rate-Place All-Info (after Heinz et al., 2001, Neural Computation)

Siebert’s (‘68,’70) results suggest Rate-Place model for Human Frequency Discrimination at low frequencies. But Frequency discrimination gets Worse at High Freqs, and Rate-Place model doesn’t ! - Siebert’s analysis was limited by simple peripheral model. - Can extend the approach using a Computational Model for AN fibers (Heinz et al., 2001) : -Allows phase-locking to rolloff accurately vs. Freq. Does a more complete AN model change our conclusion? (from Heinz et al., 2001, Neural Computation) Rate-Place All-Info

Detailed AN response properties included in Computational AN model: - Phase-locking - Onset/offsets

Comparison of Siebert’s Predictions to Human Performance: Frequency Discrimination Rate-Place All-Info Rate-Place All-Info (after Heinz et al., 2001, Neural Computation)

Summary of Heinz et al.’s results: All-Info model matches trends in Human data, for Frequency (and Level) Discrimination. Rate-Place model can’t explain Freq Discrim at high freqs. But, Thresholds of Optimal model are too low. Optimal models help identify cues that are consistent with overall performance of listeners. More realistic (sub-optimal) processing mechanisms will have elevated thresholds that do a better job of predicting both the trends and absolute thresholds of human performance.

Extension of Siebert/Heinz approach to Complex Stimuli Modeling Discrimination of Center Frequency of Formant-like Harmonic Complexes (Tan & Carney, JASA, 2005)

Results for Human Listeners (Lyzenga & Horst, 1995) Center Frequency (Hz) Lowest thresholds are for Center Freqs between Harmonics Highest thresholds are for Center Freqs at Harmonic freqs Energy-based model predicts the opposite Center Freq Discrim JNDs for 3 spectral slopes

AN Models require Timing Info to Predict correct Threshold Trends AN Population Model Predictions AN Model based on Timing Info in Small # of Fibers Provides Best Predictions

For Harmonic Complexes Timing Information is required to predict trends in human performance But, Optimal Detector uses all timing information - What aspect of ‘timing’ is critical for these results? Can use Sub-Optimal Detectors to explore different aspects of timing: e.g. Across-fiber timing (spatio-temporal patterns) vs. Within-fiber timing patterns (intervals)

Level Coding in the Auditory Nerve based on Sub-Optimal Processing: Coincidence-Detection Level-dependent tuning of Basilar Membrane results in level-dependent timing of AN responses (Anderson et al., 1971). At low frequencies, this neural cue may contribute to level coding over a wide dynamic range. At high frequencies, level-dependent gain results in wide dynamic ranges of AN fibers. Cross-frequency Coincidence Detection can take advantage of both rate and timing cues.

Timing (phase) of AN spikes varies systematically with Level (Response Area from Anderson et al., 1971, J. Acoust.Soc.Am.)

Level-dependent BW, Gain, & Phase are included in computational AN model Phase Magnitude Low SPL Hi SPL (Zhang et al., 2001, J. Acoust. Soc. Am.)

Phase Magnitude Low SPL Hi SPL Nonlinear Auditory-Nerve model has: - Nonlinear timing low Frequencies) - Wide-dynamic ranges high Frequencies) (Heinz et al., ARLO, 2001)

Coincidence Detector CDs are sensitive to rate and/or timing!

Level Discrimination Predictions based on Coincidence Detection (CD) Model (Heinz et al., 2001, J. Acoust. Soc. Am.) 1 kHz10 kHz Inputs to CD from Nonlinear Computational AN model Decision variable based on Rate of CD ---Nonlinear Temporal cues important at low frequencies ---Wide-dynamic-range rate-level functions important at high frequencies

Conclusions : Can quantify info in computational Auditory-Nerve model response and compare to psychophysical performance. Combined Rate and Temporal info (“All-Info”) explains trends in listeners across a wide range of tone frequencies and levels, and for harmonic complex freq discrim task. Coincidence Detection (CD) is a simple mechanism for decoding Temporal and/or Rate info. CD is consistent with trends & absolute thresholds of Human Performance for Level Discrimination. CD does not explain performance in Harmonic Complex task. Prelim results suggest that an interval- based strategy to coding Instantaneous Frequency or a modulation-based strategy are more promising.

Collaborators: Michael Heinz - PhD 2000, HST-MIT; now at Hopkins Steve Colburn - Dept. of Biomedical Engr., Boston University Qing Tan - PhD, 2003 Boston University Supported by NIH-NIDCD, NSF, & The Gerber Fund NOTE: Code and papers are available at: