2. Stefan Bleeck, Institute of Sound and Vibration Research, Hearing and Balance Centre University of Southampton

3.  Can sparse coding help to overcome problems caused by hearing loss? ◦ overview of the hearing process ◦ Examples of sparse algorithms for hearing aids and cochlear implants ◦ Preliminary results 5/11/2012 bleeck@gmail.com2

4. 5/11/2012 bleeck@gmail.com3

5. 5/11/2012 bleeck@gmail.com4 Important for sound localization, linear => boring

6. 5/11/2012 bleeck@gmail.com5 Important to explain limits of hearing, linear => boring

7. 5/11/2012 bleeck@gmail.com6  Contained within bony labyrinth in temporal bone  Cochlea does hearing  Semicircular canals+utricle does balance  Same mechanism, nerve, evolution, similar problems

8. 5/11/2012 bleeck@gmail.com7 7 frequency mapping

9. 5/11/2012 bleeck@gmail.com8 8  Stereocilia detect vibrations within cochlea.  Introduce half-wave rectification  Nonlinear

10. bleeck@gmail.com 13 orders of magnitude 10 Power (watts/m 2 ) 10 -12 10 -2 10 -4 10 -6 10 -8 10 -10 1

11. 5/11/2012 10 ABSOLUTE THRESHOLD CURVE membrane moves 10 -13 m Threshold as function of Frequency bleeck@gmail.com

12. 5/11/2012 bleeck@gmail.com11  Amplitude (nonlinear amplification)  Frequencies (combination tones)  compression Demo: sweeps

13. 12 Distance along BM (mm) BM displacement (nm) damaged “passive” healthy “active” OHCs inject energy in this region OHCs provide up to 40 dB amplification (= factor of 100) Travelling Wave Envelope on Basilar Membrane due to Pure Tone Stimulus:

14.  50% of 60 year old, 90% of 80 year old  Hearing aids are not good enough  ‘damage’ €2.4 Billion per year in EU  Lack of research funding today 5/11/2012 bleeck@gmail.com13

15. 5/11/2012 bleeck@gmail.com14 Electron micrographs of cochlear hair cells. Left: healthy, right: damaged by noise exposure.

16. 5/11/2012 bleeck@gmail.com15 Hearing impairment loosing audibility, Also widening of filter both results in difficulties to understand language, especially in noise

18. Listening in noise 0 dB 40 dB-15 dB ASR Normal Hearing Impaired SNR Word recognition 100% 0% Aided Un-aided 50%

19. 5/11/2012 bleeck@gmail.com18

20. 5/11/2012 bleeck@gmail.com19

21. 5/11/2012 bleeck@gmail.com20 Problem: Hearing loss constitutes a bottle neck: not all information can get through Solution: extract less, but important information -Extract content based on Information not on Energy -Specifically speech related information

22. 2 Neural representation: (Transformation) 3 Denoising (sparsification) 1 periphery model

23. 5/11/2012 bleeck@gmail.com22 ‘Sparse’ algorithms developed in our group noise

24. 5/11/2012 bleeck@gmail.com23 Filter bank

25.  Non-negative matrix factorization ◦ Matrix Z is factorised into two non-negative matrices W and H (basis vectors (5) and activity over time) ◦ (motivated by the processing in CI and auditory neurons) ◦ Z here is the ‘envelopegram’ (22 channels, 128 pt) ◦ Factorization using Euclidean cost function: ◦ Sparseness constrained: 5/11/2012 bleeck@gmail.com24 g(H)= regularity function λ= sparsity factor

26. Iterative algorithm to minimize the cost function by gradient decent: λ depends on SNR because of trade-off intelligibility - quality low noise: no sparsification high noise: lots Task: fine out how! Online experiment (restricted by speed of hardware) Offline experiment (unrestricted) 5/11/2012 bleeck@gmail.com25

27. 5/11/2012 bleeck@gmail.com26 For ‘bin’, ‘pin’, ‘din’, ‘tin’ Z W H

28. 5/11/2012 bleeck@gmail.com27

29. 5/11/2012 bleeck@gmail.com28 On-line experimental set up:

30.  22 channel filter bank  16 ms frames  Gaussian noise SNR=5 dB clean noisy denoised time frequency

31. 5/11/2012 bleeck@gmail.com30 Results from CI listeners in online experiment (problems with iteration!)

32. 5/11/2012 bleeck@gmail.com31 Results from CI listeners in offline experiment results for all participants Averaged Best sparsification as function of snr:

33. Conclusions: Sparse coding can help reduce acoustic information in a useful way Development still in its infancy, hardware restrictions still relevant High impact research field with lots of potential funding Strength of our group: clinical evaluation, weakness at the moment: lack of signal processing experts

34. 5/11/2012 bleeck@gmail.com33 Hu, H., Li, G., Chen, L., Sang, J., Wang, S., Lutman, M. E., & Bleeck, S. (2011). Enhanced sparse speech coding strategy for cochlear implants. European Signal Processing Conference (EUSIPCO). Hu, H., Taghia, J., Sang, J., Taghia, J., Mohammadiha, N., Azarpour, M., Dokku, R., et al. (2011). Speech Enhancement via Combination of Wiener Filter and Blind Source Separation. International Conference on Intelligent Systems and Knowledge Engineering. Sang, J., Hu, H., Li, G., Lutman, M. E., & Bleeck, S. (2011a). Application of a sparse coding strategy to enhance speech perception for hearing aid users. British Society of Audiology Short Papers Meeting. Sang, J., Hu, H., Li, G., Lutman, M. E., & Bleeck, S. (2011b). Enhanced Sparse Speech Processing Strategy in Cochlear Implants. Conference on implantable Auditory Prostheses (CIAP). Sang, J., Li, G., Hu, H., Lutman, M. E., & Bleeck, S. (2011a). Supervised Sparse Coding in Cochlear Implants. Conference on implantable Auditory Prostheses (CIAP). Sang, J., Li, G., Hu, H., Lutman, M. E., & Bleeck, S. (2011b). Supervised Sparse Coding Strategy in Hearing Aids. Annual Conference of the International Speech Communication Association (INTERSPEECH). Bleeck, S., Wright, M. C. M., & Winter, I. M. (2012). Speech enhancement inspired by auditory modelling. International Symposium on Hearing. Hu, H., Mohammadiha, N., Taghia, J., Leijon, A., Lutman, M. E., Bleeck, S., & Wang, S. (2012). Sparsity Level in a Non- negative Matrix Factorization Based Speech Strategy in Cochlear Implants. EUSIPCO. Li, G, Lutman, M. E., Wang, S., & Bleeck, S. (2012). Relationship between speech recognition in noise and sparseness. International Journal of Audiology, 51(2), 75–82. doi:10.3109/14992027.2011.625984 Sang, J., Hu, H., Zheng, C., Li, G., Lutman, M. E., & Bleeck, S. (2012). Evaluation of a Sparse Coding Shrinkage Algorithm in Normal Hearing and Hearing Impaired Listeners. EUSIPCO (pp. 1–5).