Design and low-cost implementation of a robust multichannel noise reduction scheme for cochlear implants Simon Doclo 1, Ann Spriet 1,2, Jean-Baptiste Maj 1,2, Marc Moonen 1, Jan Wouters 2, Bas Van Dijk 3, Jan Janssen 3 1 Dept. of Electrical Engineering (ESAT-SCD), KU Leuven, Belgium 2 Laboratory for Exp. ORL, KU Leuven, Belgium 3 Cochlear Technology Centre Europe, Belgium DARTS, 22 October 2003

2 Overview Problem statement : hearing in background noise Adaptive beamforming: GSC onot robust against model errors Design of robust noise reduction algorithm orobust fixed spatial preprocessor orobust adaptive stage Experimental results Low-cost implementation of adaptive stage ostochastic gradient algorithms ocomputational complexity + memory requirements Conclusions

3 Problem statement Hearing problems effect more than 12% of population Digital hearing instruments allow for advanced signal processing, resulting in improved speech understanding Major problem: (directional) hearing in background noise oreduction of noise wrt useful speech signal omultiple microphones + DSP in BTE ocurrent systems: simple fixed and adaptive beamforming orobustness important due to small inter-microphone distance hearing aids and cochlear implants design of robust multi-microphone noise reduction scheme  Introduction -Problem statement -State-of-the-art -GSC  Fixed beamforming  Adaptive stage  Implementation  Conclusions

4 Cochlear implants Working principle: sound is converted to electrical stimuli in speech processor, allowing deaf people to hear again cochlea brain middle ear sound external ear multi-channel electrode stimulator box speech processor  Introduction -Problem statement -State-of-the-art -GSC  Fixed beamforming  Adaptive stage  Implementation  Conclusions

5 Algorithmic requirements ‘Blind’ techniques: unknown noise sources and acoustic environment Adaptive: time-variant signals and acoustic environment Robustness: omicrophone characteristics (gain, phase, position) oother deviations from assumed signal model (e.g. VAD) Implementation issues: onumber of microphones olow computational complexity omemory  Introduction -Problem statement -State-of-the-art -GSC  Fixed beamforming  Adaptive stage  Implementation  Conclusions

6 State-of-the art noise reduction Single-microphone techniques: ospectral subtraction, Kalman filter, subspace-based oonly temporal and spectral information  limited performance Multi-microphone techniques: oexploit spatial information oFixed beamforming: fixed directivity pattern oAdaptive beamforming (e.g. GSC) : adapt to different acoustic environments  improved performance oMulti-channel Wiener filtering (MWF): MMSE estimate of speech component in microphones  improved robustness Sensitive to a-priori assumptions Robust scheme, encompassing both GSC and MWF  Introduction -Problem statement -State-of-the-art -GSC  Fixed beamforming  Adaptive stage  Implementation  Conclusions

7 Adaptive beamforming: GSC Fixed spatial preprocessor: o Fixed beamformer creates speech reference o Blocking matrix creates noise references Adaptive noise canceller: o Standard GSC minimises output noise power Spatial preprocessing Fixed beamformer A(z) Speech reference Blocking matrix B(z) Noise references Adaptive Noise Canceller   (adaptation during noise)  Introduction -Problem statement -State-of-the-art -GSC  Fixed beamforming  Adaptive stage  Implementation  Conclusions

8 Robustness against model errors Spatial preprocessor and adaptive stage rely on assumptions (e.g. no microphone mismatch, no reverberation,…) In practice, these assumptions are often not satisfied o Distortion of speech component in speech reference o Leakage of speech into noise references, i.e. Design of robust noise reduction algorithm: 1.Design of robust spatial preprocessor (fixed beamformer) using statistical knowledge about microphone characteristics 2.Design of robust adaptive stage by taking speech distortion into account in optimisation criterion  speech distortion weighted multichannel Wiener filter (SDW MWF) Speech component in output signal gets distorted Limit distortion both in and  Introduction -Problem statement -State-of-the-art -GSC  Fixed beamforming  Adaptive stage  Implementation  Conclusions

9 Design of fixed beamformer FIR filter-and-sum structure: arbitrary spatial directivity pattern for arbitrary microphone configuration Objective: calculate fixed FIR filters w n [k] such that beamformer performs desired spatial and spectral filtering Far-field: - planar waves - equal attenuation Spatial directivity pattern: Desired spatial directivity pattern:  Introduction  Fixed beamforming -Broadband design -Robustness  Adaptive stage  Implementation  Conclusions

10 Design procedures Design filter w such that spatial directivity pattern optimally fits  minimisation of cost function oBroadband problem: no design for separate frequencies  i  design over complete frequency-angle region Cost functions: oLeast-squares  quadratic function oNon-linear cost function  iterative optimisation = complex! oEigenfilter based on TLS-criterion  GEVD amplitude and phase only amplitude  Introduction  Fixed beamforming -Broadband design -Robustness  Adaptive stage  Implementation  Conclusions

11 Non-linear procedureTLS-Eigenfilter Simulations Angle (deg) Freq (Hz) dB Angle (deg) Freq (Hz) dB Parameters: -N=5, d=4cm -L=20, f s =8kHz -Pass: 40 o -80 o -Stop: 0 o -30 o + 90 o -180 o Delay-and-sum Angle (deg) Freq (Hz) dB

12 Small deviations from assumed microphone characteristics (gain, phase, position)  large deviations from desired directivity pattern, especially for small-size microphone arrays In practice microphone characteristics are never exactly known Consider all feasible microphone characteristics and optimise oaverage performance using probability as weight – requires statistical knowledge about probability density functions oworst-case performance  minimax optimisation problem – finite grid of microphone characteristics  high complexity Robust broadband beamforming Incorporate specific (random) deviations in design Measurement or calibration procedure  Introduction  Fixed beamforming -Broadband design -Robustness  Adaptive stage  Implementation  Conclusions

13 Simulations Non-linear design procedure N=3, positions: [ ] m, L=20, f s =8 kHz Passband = 0 o -60 o, Hz (endfire) Stopband = 80 o -180 o, Hz Robust design - average performance: Uniform pdf = gain ( ) and phase (-5 o -10 o ) Deviation = [ ] and [5 o -2 o 5 o ] Design JJ dev J mean J max Non-robust Average cost Maximum cost  Introduction  Fixed beamforming -Broadband design -Robustness  Adaptive stage  Implementation  Conclusions

14 Non-robust designRobust design No deviations Deviations (gain/phase) Simulations Angle (deg) Frequency (Hz) dB Angle (deg) Frequency (Hz) dB Angle (deg) Frequency (Hz) dB Angle (deg) Frequency (Hz) dB  Introduction  Fixed beamforming -Broadband design -Robustness  Adaptive stage  Implementation  Conclusions

Non-robust designRobust design Simulations 15

16 Design of robust adaptive stage Distorted speech in output signal: Robustness: limit by controlling adaptive filter oQuadratic inequality constraint (QIC): = conservative approach, constraint  f (amount of leakage) oTake speech distortion into account in optimisation criterion – 1/  trades off noise reduction and speech distortion 1/  = 0 or no speech leakage  GSC 1/  = 1  MMSE estimate of speech component in speech reference signal – Regularisation term ~ amount of speech leakage noise reductionspeech distortion Limit speech distortion, while not affecting noise reduction performance in case of no model errors  QIC  Introduction  Fixed beamforming  Adaptive stage -SP SDW MWF -Experimental validation  Implementation  Conclusions

17 Wiener solution Optimisation criterion: Problem: clean speech and hence speech correlation matrix are unknown! Approximation: VAD (voice activity detection) mechanism required! speech-and-noise periodsnoise-only periods  Introduction  Fixed beamforming  Adaptive stage -SP SDW MWF -Experimental validation  Implementation  Conclusions

18 Spatially-preprocessed SDW-MWF (1) In new optimisation criterion additional filter on speech reference signal may be added Spatial preprocessing Fixed beamformer A(z) Speech reference Blocking matrix B(z) Noise references Multi-channel Wiener Filter (SDW-MWF)   Speech Distortion Weighted Multichannel Wiener Filter (SDW-MWF)  Introduction  Fixed beamforming  Adaptive stage -SP SDW MWF -Experimental validation  Implementation  Conclusions

19 Spatially-preprocessed SDW-MWF (2) SP-SDW-MWF encompasses both GSC and SDW-MWF as special cases: oNo filter on speech reference   speech distortion regularised GSC (SDR-GSC) – regularisation term added to GSC: the larger the speech leakage, the larger the regularisation – special case: 1/  = 0 corresponds to traditional GSC – SDR-GSC outperforms GSC with quadratic inequality constraint oFilter on speech reference   SDW-MWF on pre-processed microphone signals – in absence of model errors = cascade of GSC + single-channel postfilter (SDW Wiener filter) – Model errors do not effect its performance! Outperforms QIC-GSC and SDR-GSC  Introduction  Fixed beamforming  Adaptive stage -SP SDW MWF -Experimental validation  Implementation  Conclusions

20 Experimental validation (1) Set-up: o3-mic BTE mounted on dummy head in office room (d = 1cm, 1.5cm) oSpeech source in front of dummy head (90  ) o5 stationary speech-like noise sources: 75 , 120 , 180 , 240 , 285  oMicrophone gain mismatch at 2 nd microphone Performance measures: oIntelligibility-weighted signal-to-noise ratio – I i = band importance of i th one-third octave band – SNR i = signal-to-noise ratio in i th one-third octave band oIntelligibility-weighted spectral distortion – SD i = average spectral distortion in i th one-third octave band (Power Transfer Function for speech component)  Introduction  Fixed beamforming  Adaptive stage -SP SDW MWF -Experimental validation  Implementation  Conclusions

21 Experimental validation (2) SDR-GSC: oGSC (1/  = 0) : degraded performance if significant leakage o1/  > 0 increases robustness (speech distortion  noise reduction) SP-SDW-MWF: oNo mismatch: same as SDR-GSC, larger due to SDW-WF post-filter oPerformance is not degraded by mismatch  Introduction  Fixed beamforming  Adaptive stage -SP SDW MWF -Experimental validation  Implementation  Conclusions

22 Experimental validation (3) GSC with QIC ( ) : QIC increases robustness GSC For large mismatch: less noise reduction than SP-SDW-MWF QIC  f (amount of speech leakage)  less noise reduction than SDR-GSC for small mismatch SP-SDW-MWF achieves better noise reduction than QIC-GSC, for a given maximum speech distortion level  Introduction  Fixed beamforming  Adaptive stage -SP SDW MWF -Experimental validation  Implementation  Conclusions

23 Low-cost implementation (1) Algorithms (in decreasing order of complexity): oGSVD-based – chic et très cher oQRD-based, fast QRD-based – chic et moins cher oStochastic gradient algorithms – chic et pas cher Stochastic gradient algorithm (time-domain) : oCost function results in LMS-based updating formula oAllows transition to classical LMS-based GSC by tuning some parameters (1/ , w 0 ) regularisation termClassical GSC  Introduction  Fixed beamforming  Adaptive stage  Implementation -Stochastic gradient -Complexity and memory  Conclusions

24 Low-cost implementation (2) Stochastic gradient algorithm (time-domain): oRegularisation term is unknown – Store samples in memory buffer during speech-and-noise periods and approximate regularisation term – Better estimate of regularisation term can be obtained by smoothing (low-pass filtering) Stochastic gradient algorithm (frequency-domain): oBlock-based implementation: improve gradient estimate by averaging over K samples oFrequency-domain: fast convolution and fast correlation Complexity reduction Tuning of  and 1/  per frequency Still large memory requirement due to data buffers oApproximations allow to replace data buffers by correlation matrices in frequency-domain  memory reduction Large buffer required  Introduction  Fixed beamforming  Adaptive stage  Implementation -Stochastic gradient -Complexity and memory  Conclusions

25 Complexity + memory Parameters: N = M = 2 (  mics,  adaptive filters), L = 64, f s = 16kHz, L buf = Computational complexity: Memory requirement: AlgorithmComplexityMIPS QIC-GSC(3N-1)FFT + 14N – SDW-MWF (no approximation) (3M+5)FFT + 28M SDW-MWF (approximations)(3M+2)FFT + 8M M AlgorithmMemorykWords QIC-GSC4(N-1)L + 6L0.64 SDW-MWF (no approximation) 2ML buf + 6LM + 7L41.22 SDW-MWF (approximations) 4LM 2 + 6LM + 7L2.24 Complexity comparable to FD implementation of QIC-GSC Substantial memory reduction through approximations  Introduction  Fixed beamforming  Adaptive stage  Implementation -Stochastic gradient -Complexity and memory  Conclusions

26 Conclusions Design of robust multimicrophone noise reduction algorithm: oDesign of robust fixed spatial preprocessor  need for statistical information about microphones oDesign of robust adaptive stage  take speech distortion into account in cost function SP-SDW-MWF encompasses GSC and MWF as special cases Experimental results: oSP-SDW-MWF achieves better noise reduction than QIC-GSC, for a given maximum speech distortion level oFilter w 0 improves performance in presence of model errors Implementations: Stochastic gradient algorithms available at affordable complexity and memory Further research: robustness against VAD-errors oe.g. parameters dependent on input SNR Spatially pre-processed SDW Multichannel Wiener Filter  Introduction  Fixed beamforming  Adaptive stage  Implementation  Conclusions

27 Relevant publications Doclo S., Moonen M., “GSVD-Based Optimal Filtering for Single and Multi-Microphone Speech Enhancement”, IEEE Transactions on Signal Processing, vol. 50, no. 9, Sep. 2002, pp Spriet A., Moonen M., Wouters J., “A multichannel subband GSVD approach to speech enhancement”, European Transactions on Telecommunications, vol. 13, no. 2, Mar. 2002, pp Spriet A., Moonen M., Wouters J., “Robustness analysis of GSVD based optimal filtering and generalized sidelobe canceller for hearing aid applications”, in Proc. of the IEEE Workshop on Applications on Signal Processing to Audio and Acoustics, New Paltz, New York, Oct. 2001, pp Maj J.B., Royackers L., Moonen M., Wouters J., “SVD-Based Optimal Filtering Technique For Noise Reduction In Dual Microphone Hearing Aids: A Real Time Evaluation”, submitted to IEEE Transactions on Biomedical Engineering. Doclo S., Moonen M., “Design of far-field and near-field broadband beamformers using eigenfilters”, Signal Processing, vol. 83, no. 12, pp , Dec Doclo S., Moonen M., “Design of broadband beamformers robust against gain and phase errors in the microphone array characteristics”, IEEE Transactions on Signal Processing, vol. 51, no. 10, Oct. 2003, pp Doclo S., Moonen M., “Design of broadband beamformers robust against microphone position errors”, in Proc. of the International Workshop on Acoustic Echo and Noise Control, Kyoto, Japan, Sep. 2003, pp Spriet A., Moonen M., Wouters J., “Spatially pre-processed speech distortion weighted multi-channel Wiener filtering for noise reduction”, Internal Report 03-46, ESAT- SISTA, K.U.Leuven (Leuven, Belgium), Spriet A., Moonen M., Wouters J., “Stochastic Gradient based implementation of spatially pre-processed multi-channel Wiener filtering for noise reduction in hearing aids”, Internal Report 03-47, ESAT-SISTA, K.U.Leuven (Leuven, Belgium), Available at SISTA publication engine:  Introduction  Fixed beamforming  Adaptive stage  Implementation  Conclusions