1. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay Workshop “Radar and Sonar Signal Processing,” NSTL Visakhapatnam, 17-21 Aug 2015 Coordinator: Ms. M. Vijaya Session: 21 Aug 2015, 1100 to 1230 ============================================================================ Signal Processing for Hearing Aids: Challenges and Some Solutions P. C. Pandey IIT Bombay

2. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 2/15 Outline A.Speech & Hearing B.Sliding-band Dynamic Range Compression (Ref: N. Tiwari & P. C. Pandey, NCC 2014, Paper No.1569847357) C.Automated Modification of Consonant-Vowel Ratio of Stops (Ref: A. R. Jayan & P. C. Pandey, Int. J. Speech Technology, vol. 18, pp. 113–130, 2015) Outline A.Speech & Hearing B.Sliding-band Dynamic Range Compression (Ref: N. Tiwari & P. C. Pandey, NCC 2014, Paper No.1569847357) C.Automated Modification of Consonant-Vowel Ratio of Stops (Ref: A. R. Jayan & P. C. Pandey, Int. J. Speech Technology, vol. 18, pp. 113–130, 2015)

3. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 3/15 Part A Speech & Hearing P. C. Pandey, "Signal processing for hearing aids: Challenges and some solutions,” Invited talk, Workshop “Radar and Sonar Signal Processing,” NSTL Visakhapatnam, 17-21 Aug 2015 Workshop Coordinator: Ms. M. Vijaya Session: 21 Aug 2015, 1100 to 1230 ========================================================================================

4. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 4/15 Speech Production Excitation source & filter model Excitation: voiced/unvoiced glottal, frication Filtering: vocal tract filter

5. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 5/15 Speech segments Words Syllables Phonemes Sub-phonemic segments Phonemes: basic speech units Vowels: Pure vowels, Diphthongs Consonants: Semivowels, Stops, Fricatives, Affricates, Nasals /aba/ /apa/ /aga/ /ada/

6. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 6/15 Phonemic features Modes of excitation Glottal: Unvoiced (constriction at the glottis), Voiced (glottal vibration) Frication: Unvoiced (constriction in vocal tract), Voiced (constriction in v.t. & glottal vibration) Movement of articulators Continuant (steady-state v.t. configuration): vowels, nasal stops, fricatives Non-continuant (changing v.t.): diphthongs, semivowels, oral stops (plosives) Place of articulation (place of maximum constriction in v.t.) Bilabial, Labio-dental, Linguo-dental, Alveolar, Palatal, Velar, Gluttoral Changes in voicing frequency (Fo) Supra-segmental features: Intonation, Rhythm

7. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 7/15 Hearing Mechanism Peripheral auditory system External ear: sound collection ○ Pinna ○ Auditory canal Middle ear: impedance matching ○ Ear drum ○ Middle ear bones Inner ear (cochlea): analysis & transduction Auditory nerve: transmission of neural impulses Central auditory system Information processing & interpretation

8. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 8/15 Tonotopic map of cochlea Auditory system

9. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 9/15 Hearing Impairment Types of hearing losses Conductive Sensorineural Central Functional Sensorineural hearing loss Associated with abnormalities in the cochlear hair cells or the auditory nerve. Causes: aging, excessive noise exposure, infection, adverse effect of medicines, congenital. Hearing Impairment Types of hearing losses Conductive Sensorineural Central Functional Sensorineural hearing loss Associated with abnormalities in the cochlear hair cells or the auditory nerve. Causes: aging, excessive noise exposure, infection, adverse effect of medicines, congenital.

10. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 10/15 Effects of sensorineural hearing loss Elevated hearing thresholds: inaudibility of low-level sounds Reduced dynamic range & loudness recruitment (abnormal loudness growth): distortion of loudness relationship among speech components Increased temporal masking: poor detection of acoustic landmarks Increased spectral masking (widening of auditory filters): reduced ability to sense spectral shapes >> Poor intelligibility and degraded perception of speech, particularly in noisy environment. Effects of sensorineural hearing loss Elevated hearing thresholds: inaudibility of low-level sounds Reduced dynamic range & loudness recruitment (abnormal loudness growth): distortion of loudness relationship among speech components Increased temporal masking: poor detection of acoustic landmarks Increased spectral masking (widening of auditory filters): reduced ability to sense spectral shapes >> Poor intelligibility and degraded perception of speech, particularly in noisy environment.

11. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 11/15 Signal Processing in Hearing Aids Currently available techniques Frequency selective amplification: improves audibility but not necessarily intelligibility Automatic volume control: not effective in improving intelligibility Multichannel dynamic range compression (with settable attack & release times, compression ratios): effectiveness reduced due to processing artifacts Signal Processing in Hearing Aids Currently available techniques Frequency selective amplification: improves audibility but not necessarily intelligibility Automatic volume control: not effective in improving intelligibility Multichannel dynamic range compression (with settable attack & release times, compression ratios): effectiveness reduced due to processing artifacts

12. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12/15 Techniques under development Noise suppression Distortion-free dynamic range compression Techniques for reducing the effects of increased spectral masking o Binaural dichotic presentation o Spectral contrast enhancement o Multi-band frequency compression Improvement of consonant-to-vowel ratio (CVR): for reducing the effects of increased temporal masking Techniques under development Noise suppression Distortion-free dynamic range compression Techniques for reducing the effects of increased spectral masking o Binaural dichotic presentation o Spectral contrast enhancement o Multi-band frequency compression Improvement of consonant-to-vowel ratio (CVR): for reducing the effects of increased temporal masking

13. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 13/15 Analog Hearing Aids Pre-amp → AVC → Freq. Response → Amp. Digital Hearing Aids Pre-amp & AVC → ADC → Multi-band Amplitude Compr. & Freq. Resp. → DAC & Amp. Existing Problems Poor intelligibility in noisy environment & reverberation Distortions due to multiband amplitude compression Poor speech perception due to increased spectral & temporal masking Visit to audiologist for change of settings Analog Hearing Aids Pre-amp → AVC → Freq. Response → Amp. Digital Hearing Aids Pre-amp & AVC → ADC → Multi-band Amplitude Compr. & Freq. Resp. → DAC & Amp. Existing Problems Poor intelligibility in noisy environment & reverberation Distortions due to multiband amplitude compression Poor speech perception due to increased spectral & temporal masking Visit to audiologist for change of settings

14. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 14/15 Proposed Hearing Aids Distortion-free dynamic range compression & adjustable frequency response Noise suppression & de-reverberation Processing for reducing the effects of increased spectral masking Processing for reducing the effects of increased temporal masking Implementation of signal processing using a low-power DSP chip with acceptable signal delay (< 60 ms) User selectable settings Proposed Hearing Aids Distortion-free dynamic range compression & adjustable frequency response Noise suppression & de-reverberation Processing for reducing the effects of increased spectral masking Processing for reducing the effects of increased temporal masking Implementation of signal processing using a low-power DSP chip with acceptable signal delay (< 60 ms) User selectable settings

15. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 15/15 Some Solutions for Improving Speech Perception by Listeners with Moderate-to- Severe Sensorineural Loss Sliding-band dynamic range compression as a solution to the problem posed by loudness recruitment Automated modification of consonant-vowel ratio of stop consonants as a solution to the problem posed by increased intraspeech spectral and temporal masking. Implementation using a 16-bit fixed-point DSP processor & testing for satisfactory operation. Some Solutions for Improving Speech Perception by Listeners with Moderate-to- Severe Sensorineural Loss Sliding-band dynamic range compression as a solution to the problem posed by loudness recruitment Automated modification of consonant-vowel ratio of stop consonants as a solution to the problem posed by increased intraspeech spectral and temporal masking. Implementation using a 16-bit fixed-point DSP processor & testing for satisfactory operation.

16. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay To be continued in Part B