Stefan Bleeck, Institute of Sound and Vibration Research, Hearing and Balance Centre University of Southampton

 Can sparse coding help to overcome problems caused by hearing loss? ◦ overview of the hearing process ◦ Examples of sparse algorithms for hearing aids and cochlear implants ◦ Preliminary results 5/11/2012

5/11/2012

5/11/2012 Important for sound localization, linear => boring

5/11/2012 Important to explain limits of hearing, linear => boring

5/11/2012  Contained within bony labyrinth in temporal bone  Cochlea does hearing  Semicircular canals+utricle does balance  Same mechanism, nerve, evolution, similar problems

5/11/ frequency mapping

5/11/  Stereocilia detect vibrations within cochlea.  Introduce half-wave rectification  Nonlinear

13 orders of magnitude 10 Power (watts/m 2 )

5/11/ ABSOLUTE THRESHOLD CURVE membrane moves m Threshold as function of Frequency

5/11/2012  Amplitude (nonlinear amplification)  Frequencies (combination tones)  compression Demo: sweeps

12 Distance along BM (mm) BM displacement (nm) damaged “passive” healthy “active” OHCs inject energy in this region OHCs provide up to 40 dB amplification (= factor of 100) Travelling Wave Envelope on Basilar Membrane due to Pure Tone Stimulus:

 50% of 60 year old, 90% of 80 year old  Hearing aids are not good enough  ‘damage’ €2.4 Billion per year in EU  Lack of research funding today 5/11/2012

5/11/2012 Electron micrographs of cochlear hair cells. Left: healthy, right: damaged by noise exposure.

5/11/2012 Hearing impairment loosing audibility, Also widening of filter both results in difficulties to understand language, especially in noise

Listening in noise 0 dB 40 dB-15 dB ASR Normal Hearing Impaired SNR Word recognition 100% 0% Aided Un-aided 50%

5/11/2012

5/11/2012

5/11/2012 Problem: Hearing loss constitutes a bottle neck: not all information can get through Solution: extract less, but important information -Extract content based on Information not on Energy -Specifically speech related information

2 Neural representation: (Transformation) 3 Denoising (sparsification) 1 periphery model

5/11/2012 ‘Sparse’ algorithms developed in our group noise

5/11/2012 Filter bank

 Non-negative matrix factorization ◦ Matrix Z is factorised into two non-negative matrices W and H (basis vectors (5) and activity over time) ◦ (motivated by the processing in CI and auditory neurons) ◦ Z here is the ‘envelopegram’ (22 channels, 128 pt) ◦ Factorization using Euclidean cost function: ◦ Sparseness constrained: 5/11/2012 g(H)= regularity function λ= sparsity factor

Iterative algorithm to minimize the cost function by gradient decent: λ depends on SNR because of trade-off intelligibility - quality low noise: no sparsification high noise: lots Task: fine out how! Online experiment (restricted by speed of hardware) Offline experiment (unrestricted) 5/11/2012

5/11/2012 For ‘bin’, ‘pin’, ‘din’, ‘tin’ Z W H

5/11/2012

5/11/2012 On-line experimental set up:

 22 channel filter bank  16 ms frames  Gaussian noise SNR=5 dB clean noisy denoised time frequency

5/11/2012 Results from CI listeners in online experiment (problems with iteration!)

5/11/2012 Results from CI listeners in offline experiment results for all participants Averaged Best sparsification as function of snr:

Conclusions: Sparse coding can help reduce acoustic information in a useful way Development still in its infancy, hardware restrictions still relevant High impact research field with lots of potential funding Strength of our group: clinical evaluation, weakness at the moment: lack of signal processing experts

5/11/2012 Hu, H., Li, G., Chen, L., Sang, J., Wang, S., Lutman, M. E., & Bleeck, S. (2011). Enhanced sparse speech coding strategy for cochlear implants. European Signal Processing Conference (EUSIPCO). Hu, H., Taghia, J., Sang, J., Taghia, J., Mohammadiha, N., Azarpour, M., Dokku, R., et al. (2011). Speech Enhancement via Combination of Wiener Filter and Blind Source Separation. International Conference on Intelligent Systems and Knowledge Engineering. Sang, J., Hu, H., Li, G., Lutman, M. E., & Bleeck, S. (2011a). Application of a sparse coding strategy to enhance speech perception for hearing aid users. British Society of Audiology Short Papers Meeting. Sang, J., Hu, H., Li, G., Lutman, M. E., & Bleeck, S. (2011b). Enhanced Sparse Speech Processing Strategy in Cochlear Implants. Conference on implantable Auditory Prostheses (CIAP). Sang, J., Li, G., Hu, H., Lutman, M. E., & Bleeck, S. (2011a). Supervised Sparse Coding in Cochlear Implants. Conference on implantable Auditory Prostheses (CIAP). Sang, J., Li, G., Hu, H., Lutman, M. E., & Bleeck, S. (2011b). Supervised Sparse Coding Strategy in Hearing Aids. Annual Conference of the International Speech Communication Association (INTERSPEECH). Bleeck, S., Wright, M. C. M., & Winter, I. M. (2012). Speech enhancement inspired by auditory modelling. International Symposium on Hearing. Hu, H., Mohammadiha, N., Taghia, J., Leijon, A., Lutman, M. E., Bleeck, S., & Wang, S. (2012). Sparsity Level in a Non- negative Matrix Factorization Based Speech Strategy in Cochlear Implants. EUSIPCO. Li, G, Lutman, M. E., Wang, S., & Bleeck, S. (2012). Relationship between speech recognition in noise and sparseness. International Journal of Audiology, 51(2), 75–82. doi: / Sang, J., Hu, H., Zheng, C., Li, G., Lutman, M. E., & Bleeck, S. (2012). Evaluation of a Sparse Coding Shrinkage Algorithm in Normal Hearing and Hearing Impaired Listeners. EUSIPCO (pp. 1–5).