1. Laboratory for Experimental ORL K.U.Leuven, Belgium Dept. of Electrotechn. Eng. ESAT/SISTA K.U.Leuven, Belgium Combining noise reduction and binaural cue preservation for hearing aids: MWF-ITF Multichannel Wiener Filter with Interaural Transfer Function Van den Bogaert T., Doclo S., Moonen M. and Wouters J. ICASSP 2007 download at https://gilbert.med.kuleuven.be/~u0041407/2007ICASSP.pdf Contact: Tim.vandenbogaert@med.kuleuven.ac.be

2. 20-04-2007ICASSP 2007 Van den Bogaert et al.2 Overview Problem statement: binaural hearing aids, noise reduction and preservance of binaural cues Multichannel Wiener Filter approaches: –MWF: a standard N-microphone Multi-channel Wiener Filter approach –MWF-ITF: Extension of MWF to an MWF with integrated Interaural Transfer Function. –Experimental results: SRT and Localization Objective measures Perceptual measures (N=5)

3. 20-04-2007ICASSP 2007 Van den Bogaert et al.3 Problem statement Hearing impairment  reduction of speech intelligibility in background noise (even with amplification) –Signal processing to selectively enhance useful speech signal –Multiple microphones available: spectral + spatial processing –Most hearing impaired are fitted with hearing aids at both ears Binaural hearing: everything relating to hearing simultaneously (vs bilateral) with two ears –Binaural cues, in addition to spectral and temporal cues, play an important role in binaural noise reduction and sound localization. (important to preserve these cues) –It has been reported that current bilateral noise reduction systems have a negative impact on binaural hearing [Van den Bogaert et al. 2005, Van den Bogaert et al. 2006]

4. 20-04-2007ICASSP 2007 Van den Bogaert et al.4 Problem statement Bilateral system Binaural system + More microphones = more performance? - Need of binaural link - If adaptive typically no control on left/right proc. Hard to preserve interaural cues (mic mismatch, imperfections,...) - more control on left/right processing? +

5. 20-04-2007ICASSP 2007 Van den Bogaert et al.5 Problem statement Main binaural cues –Interaural phase or interaural time differences ITD range: from 10µs to 700µs Of the signal f<1300Hz and/or on the low frequency envelope for complex sounds –Interaural level differences ILD range: from 1dB up to more than 30dB Physically significant f>2000Hz IPD/ITD ILD Signal source

6. 20-04-2007ICASSP 2007 Van den Bogaert et al.6 Design criteria binaural noise reduction for HA’s: - Maximize noise reduction by using all available microphone signals (binaural link assumed), 2 outputs needed - Preserve the binaural cues - Limit the amount of speech distortion (HA constraints: Robustness of the system, low complexity, …) Problem statement

7. 20-04-2007ICASSP 2007 Van den Bogaert et al.7 Overview of binaural noise reduction techniques Different approaches –BSS methods (Robert Aichner ICASSP 3days ago) –Fixed beamforming [e.g. Desloge 1997] Low complexity Limited performance, only speech cues may be preserved (in ideal situations) –CASA based techniques [e.g. Wittkop 2003] Perfect preservation of speech/noise cues Mostly for 2 microphones, “spectral substraction” like problems –Adaptive beamforming, based on GSC structure passing the low freq part of the signal unproc. [e.g. Welker 1997] Preserves parts of the binaural cues Substantial drop in noise reduction –Binaural multi-channel wiener filter [e.g. Doclo 2002 Spriet 2004] Speech cues are preserved No assumptions about positions of sources and microphones Noise cues may be distorted Extension of MWF : preservation of binaural speech and noise cues without substantially compromising noise reduction performance

8. 20-04-2007ICASSP 2007 Van den Bogaert et al.8 Multichannel Wiener Filter Filtered output: Speech component Noise component 2M microphones: Goal: to estimate the speech component at the reference microphone of each hearing aid (r0, r1) typically the front omnidirectional one: Standard multichannel Wiener filter A hearing aid listening scenario

9. 20-04-2007ICASSP 2007 Van den Bogaert et al.9 Multichannel Wiener Filter Speech distortion Noise reduction Trade off parameter Introduce trade off parameter noise reduction/speech distortion To control or reduce speech distortion rewrite cost function: Speech distortion weighted multichannel Wiener filter In standard hearing aid beamforming, avoiding speech distortion is typically done by calibrating the speech reference path and removing the speech component in the noise ref path

10. 20-04-2007ICASSP 2007 Van den Bogaert et al.10 Multichannel Wiener Filter Depends on second-order statistics of speech and noise, no assumption of speech and noise source (can be integrated in VAD) Perfectly preserves the interaural cues of the speech component since in the left and right hearing aid an estimate is made of the speech component in the front microphone of this hearing aids. Shifts the interaural cues of the noise component to the cues of the speech component !!!! Estimate, f (VAD) Add term related to binaural cues of noise component to the MWF cost function Possible cues: ITD, ILD, Interaural Transfer Function (ITF) ITF-MWF

11. 20-04-2007ICASSP 2007 Van den Bogaert et al.11 Multichannel Wiener Filter – ITF extension Under assumption of a single noise source You can do this for the speech and noise component Goal: the ITF of the noise component at the output = ITF at the input Performance and influence of beta and alpha on Loc and SNR improvement performance?

12. 20-04-2007ICASSP 2007 Van den Bogaert et al.12 Experimental results Identification of HRTFs: –Binaural recordings on CORTEX MK2 artificial head –2 omni-directional microphones on each hearing aid (d=1cm) –Hrtfs measured = -90  :15  :90 , 90  :30  :270 , 1m from head –Conditions: T 60 =140 ms (T 60 =590 ms added) f s =16 kHz,  =1 Objective evaluation: –AI weighted SNR improvement –ITD and ILD error Perceptual evaluation: –Headphone exp with record. hrtfs –SRT measurements (50% Sp. Intell) –Localization using prerecorded hrtfs, S and N components are send seperately through the fixed filter, localize S and N in the room were the hrtfs were recorded

13. 20-04-2007ICASSP 2007 Van den Bogaert et al.13 Experimental results Left input signals Right input signals FFT Left outputRight output IFFT Frequency-domain filtering Off-line computation of statistics VAD Calculate filters for this specific sc. Stored filters are converged for a condition Sx Ny with Sx=speech weighted noise from angle x and Ny=babble noise from angle y

14. 20-04-2007ICASSP 2007 Van den Bogaert et al.14 Experimental results: objective evaluation Error measures correlated with design criteria: –Maximize speech intelligibility: Intelligibility weighted SNR improvement (left/right) –Minimize interaural cue distortion ILD of speech and noise component ITD of speech and noise component importance of i-th frequency bin for speech intelligibility low-pass filter 1500 Hz

15. 20-04-2007ICASSP 2007 Van den Bogaert et al.15 objective evaluation: localization S0N60

16. 20-04-2007ICASSP 2007 Van den Bogaert et al.16 objective evaluation: SRT β β α α S0N60 Additions: For T60=0.59s Left perf. S0N60 drops to 5dB SNR AI, right perf drops to 7dB SNR AI noise reduction Going from 2 to 4 microphones gives a gain of about 3 dB SNR AI to 9dB SNR AI compared to 2 microphone performance T60=0.14s importance of i-th frequency for speech intelligibility

17. 20-04-2007ICASSP 2007 Van den Bogaert et al.17 SRT: perceptual evaluation –Adaptive SRT procedure to find 50% Speech Recept Threshold –S0N60, dutch VU sentences, T60=140ms –average SRT without processing = -9.2 dB –SRT improvements in the range 11-13 dB Binaural speech intelligibility advantage because of spatial seperation speech and noise component does not seem to compensate for loss in SNR improvement Addition: performance drops to around 6dB SRT gain if T60=590ms (S0N90 tested for N=2)

18. 20-04-2007ICASSP 2007 Van den Bogaert et al.18 Localization: perceptual evaluation Condition SxN0: Speech arrives from angle x, with x from -90° till +90° in steps of 30°, noise arrives from 0 degrees. Perceptual procedure: calculate MWF filters offline trained on spatial condition SxNy. Now run a telephone ring arriving from angle x and angle y seperately through the filters and store the result. Play these wav files under headphones to the subject and ask to localize the telephone signal.

19. 20-04-2007ICASSP 2007 Van den Bogaert et al.19 Localization: perceptual evaluation SxN0 alfa=0 beta=0 Localization of Sx Localization of N0 alfa=0 beta=10

20. 20-04-2007ICASSP 2007 Van den Bogaert et al.20 Localization: perceptual evaluation SxN0

21. 20-04-2007ICASSP 2007 Van den Bogaert et al.21 Sum of localisation errors S x and N 0 Parameters can be tuned to achieve better overal localization performance -> at the cost of some noise reduction There is a correlation between physical and perceptual evaluation, even for localization. However error measures far from perfect. (do not include diffuseness, …) Localization: perceptual evaluation

22. 20-04-2007ICASSP 2007 Van den Bogaert et al.22 Conclusions (Speech distortion weighted) MWF preserves the speech cues, not the noise cues MWF-ITF enables, by constraining the filters W to an area where noise ITF is preserved, a trade off between preservance of speech and noise cues and noise reduction performance (a solution but not the perfect solution: multiple spectral overlapping noise sources,...) Preserving localization cues did not show a large benefit (due to the spatial seperation of speech and noise) / reduction (due to the extra constraints set on W) in SRT score.

23. 20-04-2007ICASSP 2007 Van den Bogaert et al.23 Acknowledgements download at https://gilbert.med.kuleuven.be/~u0041407/2007ICASSP.pdf Contact: Tim.vandenbogaert@med.kuleuven.ac.be

24. 20-04-2007ICASSP 2007 Van den Bogaert et al.24 objective evaluation: SRT (additions) The gain of going from 2 mics on one HA to 3 or 4 mics (low reverb): –Single Noise Scenario: 2 mic performance: 5 to 19 dB SNR AI improvement 2 to 3 mics: +2/+5 dB SNR AI improvement 3 to 4 mics: +1/+4 dB SNR AI improvement (max if noise source is at position of 2 mics -> adding a good SNR signal as 3rd or 4th microphone) –Multiple noise sources (3 noise sources) 2 mic performance: around 7 dB SNR AI improvement 2 to 3 mics: +2dB SNR AI improvement 3 to 4 mics: +2dB SNR AI improvement

25. 20-04-2007ICASSP 2007 Van den Bogaert et al.25 Localization: objective evaluation SxN0 large  changes direction of speech component to noise component  increase weight  (cf. physical and perceptual evaluation)