The influence of hearing loss and age on sensitivity to temporal fine structure Brian C.J. Moore Department of Experimental Psychology, University of Cambridge, Downing Street, Cambridge CB2 3EB, UK

Outline What is temporal fine structure (TFS)? The role of TFS in pitch perception Effect of hearing loss on sensitivity to TFS for pitch perception The role of TFS in speech perception Effect of hearing loss on sensitivity to TFS for speech perception Effect of age on sensitivity to TFS Possible implications for selection of hearing aids

Auditory representation of TFS and E: Normal hearing Each place on the basilar membrane (BM) behaves like a bandpass filter The response at each place can be considered as composed of TFS: carried in patterns of phase locking to individual cycles of the carrier E: carried in fluctuations in firing rate over time – but may also get phase locking to envelope Phase locking weakens at high frequencies: less TFS information above about 5 kHz (normal hearing)

E and TFS for bandpass filtered speech 'en' as in 'sense' CF: 4800 Hz ERBN: 540 Hz CF: 1500 Hz ERBN: 186 Hz CF: 370 Hz ERBN: 65 Hz

E and TFS for bandpass filtered speech 'en' as in 'sense' CF: 4800 Hz ERBN: 540 Hz CF: 1500 Hz ERBN: 186 Hz CF: 370 Hz ERBN: 65 Hz

Testing the role of TFS in pitch perception “Frequency-shifted” tones can be created from harmonic tones by shifting each component upwards by the same amount in Hz. Such tones have the same envelope repetition rate as the original harmonic tone, but different TFS. Level Frequency, Hz

Waveforms of unshifted and shifted tones

Pitch shift for high harmonics Pitch shifts with shift in component frequencies: Schouten, J. F. (1940). "The perception of pitch," Philips Tech. Rev. 5, 286-294. de Boer, E. (1956). "Pitch of inharmonic signals," Nature 178, 535-536. Patterson, R. D. (1973). "The effects of relative phase and the number of components on residue pitch," J. Acoust. Soc. Am. 53, 1565-1572. Sensitivity to TFS for high harmonics? But … the excitation pattern on the basilar membrane also shifts e.g. 1800, 2000, 2200  1830, 2030, 2230 Eliminating excitation-pattern shifts: Moore, G. A., and Moore, B. C. J. (2003). "Perception of the low pitch of frequency-shifted complexes," J. Acoust. Soc. Am. 113, 977-985. complex tones with many components passed through a fixed bandpass filter broadband noise added to mask components on edges of passband

SPECTRA

Excitation patterns: Harmonic (black) and shifted (red) complexes Nominal F0 = 400 Hz Filter centred on 11th harmonic

Simulation of waveforms on basilar membrane (CF = 1000 Hz)

Results of pitch-shift experiments Subjects were asked to match the pitch of a frequency-shifted complex tone with that of a harmonic tone with adjustable fundamental frequency. For tones with harmonics below the 14th, pitch shifted with increasing frequency shift Sensitivity to TFS For tones with all harmonics above the 14th, no pitch shift Sensitivity only to E

Assessing the role of TFS in pitch perception by hearing-impaired subjects HI and CI subjects generally are poor at fundamental frequency (F0) discrimination. Reasons: Reduced frequency selectivity (can’t resolve harmonics)? Reduced sensitivity to TFS? Problem in assessing the role of TFS: The task of Moore and Moore (2003) requires accurate pitch-matching. HI subjects cannot do this. Solution: Hopkins, K., and Moore, B. C. J. (2007). "Moderate cochlear hearing loss leads to a reduced ability to use temporal fine structure information," J. Acoust. Soc. Am. 122, 1055-1068. Measured the ability to discriminate sounds using TFS information: discriminate shifted from unshifted stimuli in a 3-alternative task.

Measuring the ability to use TFS Task: Listeners discriminated a harmonic complex (H) from a corresponding frequency-shifted complex (S): SHH or HSH or HHS Pick the odd one out Reducing excitation-pattern (spectral) cues: complex tones contained many components and were spectrally shaped (bandpass filtered) Broadband noise was added so that only components in the passband were audible Nominal F0 = 100, 200 or 400 Hz Center component with harmonic number =11 (all components unresolved)

Hopkins and Moore (2007): Method and Results NH subjects performed very well shift of 0.05F0 needed for d = 1 high sensitivity to TFS HI subjects performed very poorly, with four out of six subjects scoring no better than chance with a shift of 0.5F0 (the maximum possible). Conclusion: Listeners with moderate cochlear hearing loss show very little or no ability to use TFS information to discriminate harmonic and frequency-shifted tones for frequencies > 1100 Hz.

The TFS1 test (Moore and Sek, 2009, IJA) Moore B.C.J. & Sek A. 2009. Development of a fast method for determining sensitivity to temporal fine structure. Int. J. Audiol., 48, 161-171. Task: discriminate harmonic (H) and frequency-shifted inharmonic (I) tones Tones synthesised with many harmonics and passed through fixed bandpass filter centred on high harmonics minimises excitation-pattern cues Components added in random phase Noise added to mask distortion products

Task Two-interval forced choice Each interval has four 200-ms tone bursts with 100-ms intervals between bursts 300-ms silence between intervals One interval has H H H H Other interval has H I H I Task – pick the interval in which the tones alternate in pitch Frequency shift (F) varied adaptively to determine “threshold” If procedure calls for F > 0.5F0, switch to measuring % correct with F = 0.5F0 Software available from http://hearing.psychol.cam.ac.uk/TFS1/TFS1.htm

TFS and speech perception Common belief: envelope fluctuations in different frequency bands are the most important cues for speech intelligibility (Shannon et al., 1995). But …. when background sounds are present, TFS may play an important role. Example: Cochlear implants (CI) do not convey TFS and people with CI perform very poorly when background sounds are present.

(N-1) additional channels Channel carrier (tone/noise) N-channel vocoder processing to remove original TFS information but preserve envelope cues Coded audio OUT x (N-1) additional channels + tilt filter band-pass filter N Envelope extraction smoothing (low-pass filter) rectify Channel carrier (tone/noise) Audio IN In quiet (16 ch) In noise (16 ch) Can also be done using the Hilbert transform Two talkers (32 ch)

Speech sounds filtered into 32 1-ERBN wide bands. Assessing the importance of TFS for speech perception in the presence of a background talker Hopkins, K., Moore, B. C. J., Stone, M. A., 2008. Effects of moderate cochlear hearing loss on the ability to benefit from temporal fine structure information in speech. J. Acoust. Soc. Am. 123, 1140-1153. Speech sounds filtered into 32 1-ERBN wide bands. Bands up to the Nth recombined without processing (preserving TFS) (time delays produced by the filtering were compensated). Bands from N+1 to 32 noise vocoded (no original TFS). N varied from 0 to 32 N = 32 means intact speech (E + TFS) N = 0 means all bands noise vocoded SRT measured for a target talker (ASL sentences) in a background talker. NH and HI subjects with flat moderate loss tested. For HI subjects, Cambridge formula gain was applied.

Results

Main findings As N increased, the SRT decreased (improved) markedly (by about 15 dB) for the NH subjects, but improved by only about 5 dB for the HI subjects. HI subjects got only a small advantage from the TFS. When TFS was present in all channels, the SRT was about –17 dB for the NH subjects and +2 dB for the HI subjects! A few HI subjects showed near-normal benefit from adding TFS, but some showed no benefit. Similar findings using a tone vocoder.

Score on TFS1 task (mean for two frequency regions) Overall benefit from TFS: SRT for vocoded sounds minus SRT for unprocessed sounds. Correlation (HI only) = 0.67 p < 0.05

Effect of age on the use of TFS Hopkins, K., and Moore, B. C. J. (2011). "The effects of age and cochlear hearing loss on temporal fine structure sensitivity, frequency selectivity, and speech reception in noise," J. Acoust. Soc. Am. 130, 334-349. Measured: performance on the TFS1 task audiometric threshold at the test frequency sharpness of the auditory filter at the test frequency sensitivity to inter-aural phase (TFS-LF test) Three groups of subjects Young, normal hearing Old, normal hearing (up to 6 kHz) Hearing impaired, wide age range

Hopkins and Moore (2010): TFS1 results Young normal hearing Old normal hearing Hearing impaired (29-82 years) (mild loss) Scores above the dashed line are above chance Some NHO and HI subjects with mild losses cannot do the task, even when audiometric thresholds at the test frequency are 10 dB HL or less

Binaural sensitivity to TFS - ITDs

Ambiguity of ITDs for sinewaves For sinewaves an ITD is equivalent to an inter-aural phase difference (IPD). IPDs can only be detected for frequencies below about 1500 Hz.

Binaural sensitivity to TFS: the TFS-LF test Hopkins and Moore (2010, IJA): two-alternative forced-choice task each interval contains four tones with frequency f in one interval all tones are diotic in the other tones one and three are diotic while tones two and four have an interaural phase difference (IPD) task: pick the interval in which the tones appear to move within the head envelopes always synchronous across ears – TFS is needed to perform the task Software available from http://hearing.psychol.cam.ac.uk/TFS1/TFS1.htm

Results from Hopkins and Moore (2011) For 750 Hz, older NH ( ) perform more poorly than young NH ( )

Binaural TFS sensitivity for older subjects when absolute thresholds at the test frequency are “normal” Moore, B. C. J., Glasberg, B. R., Stoev, M., Füllgrabe, C., and Hopkins, K. (2012). "The influence of age and high-frequency hearing loss on sensitivity to temporal fine structure at low frequencies," J. Acoust. Soc. Am. 131, 1003-1006. Thirty-nine subjects Age range 61 to 83 yrs: mean = 69 yrs, SD = 6 yrs Selected to have normal or near-normal audiometric thresholds ( 30 dB HL) for frequencies up to 1000 Hz The audiometric thresholds at high frequencies covered a wide range 5 to 65 dB HL at 4000 Hz: mean = 36 dB HL, SD = 13 dB 10 to 85 dB HL at 8000 Hz: mean = 51 dB HL, SD = 20 dB Small or no asymmetry across ears

Results:binaural TFS sensitivity is correlated with age not absolute threshold

Binaural and monaural TFS sensitivity for the “middle aged” Moore, Vickers and Mehta, 2012, IJA (submitted) 35 subjects with absolute thresholds  20 dB HL up to 6 kHz Used the monaural TFS1 test and the binaural TFS-LF test Decline in TFS sensitivity occurs progressively with increasing age above about 20 years  = scores for musicians

Füllgrabe, Moore and Stone (unpublished) Measured sensitivity to TFS using the TFS1 test at 1 and 2 kHz and the TFS-LF test at 0.5 and 0.75 kHz Two groups: young (mean age = 23 yrs) older (mean age = 67 yrs) matched for verbal IQ all audiograms bilaterally normal (20 dB HL) for frequencies up to 6 kHz mean audiograms closely matched across groups TFS performance significantly poorer for the older group than for the young group

Füllgrabe, Moore and Stone (2) Measured the intelligibility of consonants in steady noise and noise that was sinusoidally amplitude modulated at 5 or 80 Hz Also measured intelligibility for sentences in a single talker background Same SBRs for the two groups Scores (averaged across all noise types and all SBRs) were correlated with scores on the TFS tasks, averaged across all centre frequencies and tasks r = 0.76, p < 0.01 r = 0.83, p < 0.01

Füllgrabe, Moore and Stone (3) The older subjects had poorer identification of speech in both the steady and modulated noises

Conclusions on the use of TFS TFS is important for pitch perception and for speech perception. People with cochlear hearing loss have reduced or no ability to use TFS. Age adversely affects monaural and binaural TFS processing

Possible reasons for reduced ability of hearing-impaired and older subjects to use TFS Reduced precision of phase locking controversial Change in relative phase of response at different points along the basilar membrane disrupts cross-correlation processes More complex and more rapidly varying TFS resulting from broader auditory filters disrupts central mechanisms for decoding TFS Mis-match between place information and TFS information Temporal jitter in transmission to central auditory system Disruption of central mechanisms involved in “decoding” TFS information Degeneration of auditory neurones, even when audiometric thresholds are normal

Possible clinical implications of reduced ability to use TFS Most (all?) hearing aids incorporate compression. An individual who has reduced ability to process TFS will rely on temporal envelope cues in different frequency channels to understand speech. Fast-acting compression can disrupt envelope cues. when the input signal is a mixture of voices, fast-acting compression introduces cross-modulation Stone, M. A., and Moore, B. C. J. (2004). "Side effects of fast-acting dynamic range compression that affect intelligibility in a competing speech task," J. Acoust. Soc. Am. 116, 2311-2323. May be better to use slow compression for such an individual. Fast compression may work well for someone with a good ability to use TFS.

Acknowledgements Supported by: The MRC Deafness Research UK Thanks to: Kathryn Hopkins Aleksander Sek Michael Stone Brian Glasberg Geoff Moore Christian Füllgrabe