1. Speech & Audio Processing - Part–II Digital Audio Signal Processing
Marc Moonen
Dept. E.E./ESAT-STADIUS, KU Leuven
homes.esat.kuleuven.be/~moonen/

2. Speech & Audio Processing
Part-I (H. Van hamme)
speech recognition
speech coding (+audio coding)
speech synthesis (TTS)
Part-II (M. Moonen): Digital Audio Signal Processing
microphone array processing
noise cancellation
acoustic echo cancellation
acoustic feedback- cancellation
active noise control
3D audio
PS: selection of topics

3. Digital Audio Signal Processing
Aims/scope
Case study: Hearing instruments
Overview
Prerequisites
Lectures/course material/literature
Exercise sessions/project
Exam

4. Aims/Scope Aim is 2-fold : Speech & audio per se
S & A industry in Belgium/Europe/…
Basic signal processing theory/principles :
Optimal filters
Adaptive filter algorithms (APA, Filtered-X LMS,..)
Kalman filters (linear/nonlinear)
etc...

5. Case Study: Hearing Instruments 1/14
Outer ear/middle ear/inner ear
Tonotopy of inner ear: spatial arrangement of where sounds of different frequency are processed
Low-freq tone ©
= Cochlea
Neural activity
for low-freq tone
High-freq tone
Neural activitity
for high-freq tone 5

6. Case Study: Hearing Instruments 2/14
Hearing loss types:
conductive
sensorineural
mixed
One in six adults (Europe)
…and still increasing
Typical causes:
aging
exposure to loud sounds …
[Source: Lapperre]

7. Case Study: Hearing Instruments 3/14
Hearing impairment : Dynamic range & audibility
Normal hearing Hearing impaired
subjects subjects
Level
100dB
0dB

8. Case Study: Hearing Instruments 4/14
Hearing impairment : Dynamic range & audibility
Dynamic range compression (DRC) (…rather than `amplification’)
Level
100dB
100dB
Output Level (dB)
0dB
0dB
100dB
0dB
Input Level (dB)
Design: multiband DRC, attack time, release time, …

9. Case Study: Hearing Instruments 5/14
Hearing impairment : Audibility vs speech intelligibility
Audibility does not imply
intelligibility
Hearing impaired subjects need 5..10dB larger signal-to-noise ratio (SNR) for speech understanding in noisy environments
Need for noise reduction (=speech enhancement) algorithms:
State-of-the-art: monaural 2-microphone adaptive noise reduction
Near future: binaural noise reduction (see below)
Not-so-near future: multi-node noise reduction (see below)
SNR
20dB
0dB
Hearing loss (dB, 3-freq-average)

10. Case Study: Hearing Instruments 6/14
Hearing Aids (HAs)
Audio input/audio output
(`microphone-processing-loudspeaker’)
‘Amplifier’, but so much more than an amplifier!!
History:
Horns/trumpets/…
`Desktop’ HAs (1900)
Wearable HAs (1930)
Digital HAs (1980)
State-of-the-art:
MHz’s clock speed
Millions of arithmetic operations/sec, …
Multiple microphones
1921
2007 (Oticon)

11. Case Study: Hearing Instruments 7/14
Cochlear Implants (CIs)
Audio input/electrode stimulation output
Stimulation strategy + preprocessing similar to HAs
History:
Volta’s experiment…
First implants (1960)
Commercial CIs ( )
Digital CIs (1980)
State-of-the-art:
MHz’s clock speed, Mops/sec, …
Multiple microphones
Other: Bone anchored HAs, middle ear implants, …
Alessandro Volta
Intra-cochlear electrode
© Cochlear Ltd
Electrical stimulation
for low frequency
Electrical stimulation
for high frequency

12. Case Study: Hearing Instruments 8/14
External Processor
Digital/analog-conversion (cfr infra)
Digital processing & filterbank
Etc..
Coil
Inductive/magnetic coupling
Implant
Electrode array
PS: number of CI-implantees worldwide approx
PS: 1 CI is approx. 25kEURO, plus surgery, revalidation,..
PS: 3 companies (Cochlear LtD, Med-El, Advanced Bionics)
© Cochlear Ltd
12/32 12

13. Case Study: Hearing Instruments 9/14
HA technology requirements
Small form factor (cfr. user acceptance)
Low power: 1…5mW (cfr. battery lifetime ≈ 1 week)
Low processing delay: 10msec (cfr. synchronization with lip reading)
DSP challenges in hearing instruments
Dynamic range compression (cfr supra)
Dereverberation: undo filtering (`echo-ing’) by room acoustics
Feedback cancellation
Noise reduction

14. Case Study: Hearing Instruments 10/14
DSP Challenges: Feedback Cancellation
Problem statement: Loudspeaker signal is fed back into microphone, then amplified and played back again
Closed loop system may become unstable (howling)
Similar to feedback problem in public address systems (for the musicians amongst you)
Model F -
Similar to echo cancellation in GSM handsets, Skype,…
but more difficult due to signal correlation

15. Case Study: Hearing Instruments 11/14
DSP Challenges: Noise reduction Multimicrophone ‘beamforming’, typically with 2 microphones, e.g. ‘directional’ front microphone and ‘omnidirectional’ back microphone
“filter-and-sum” the
microphone signals

16. Case Study: Hearing Instruments 12/14
Binaural hearing: Binaural auditory cues
ITD (interaural time difference)
ILD (interaural level difference)
Binaural cues (ITD: f < 1500Hz, ILD: f > 2000Hz) used for
Sound localization
Noise reduction
=`Binaural unmasking’ (‘cocktail party’ effect)
0-5dB
signal
ILD
ITD

17. Case Study: Hearing Instruments 13/14
Binaural hearing aids
Two hearing aids (L&R) with wireless link & cooperation
Opportunities:
More signals (e.g. 2*2 microphones)
Better sensor spacing (17cm i.o. 1cm)
Constraints: power/bandwith/delay of wireless link
..10kBit/s: coordinate program settings, parameters,…
..300kBits/s: exchange 1 or more (compressed) audio signals
Challenges:
Improved localization through cue preservation
Improved noise reduction + benefit from binaural unmasking
Signal selection/filtering, audio coding, synchronisation, …

18. Case Study: Hearing Instruments 14/14
Future: Multi-node noise reduction – sensor networks

19. Overview General speech communication set-up :
- background ‘noise’  noise suppression, source separation
- far-end echoes  acoustic echo cancellation
- reverberation  de-reverberation/deconvolution
Applications :
teleconferencing/teleclassing
hands-free telephony
hearing aids, etc..

20. Overview : Lecture-2&3 Microphone Array Processing
Spatial filtering - Beamforming
Fixed (Lecture 2) vs. adaptive (Lecture3)
Example filter-and-sum beamformer :
Application: hearing aids

21. Overview : Lecture-4 Noise Reduction
`microphone_signal[k] = speech[k] + noise[k]’
Single-microphone noise reduction
Spectral Subtraction Methods (spectral filtering)
Iterative methods based on speech modeling
(Wiener & Kalman Filters)
Multi-microphone noise reduction
Beamforming revisited
Optimal filtering approach : spectral+spatial filtering

22. Overview : Lecture-5 Acoustic Echo Cancellation
Adaptive filtering problem:
non-stationary/wideband/… speech signals
non-stationary/long/… acoustic channels
Adaptive filtering algorithms
AEC Control
AEC Post-processing
Stereo AEC

23. Overview : Lecture-6 Acoustic Feedback Cancellation amplifier
Ex: Hearing aids
Ex: PA systems
correlation between filter input (`x ’) and near-end signal (‘ n ’)
fixes : noise injection, pitch shifting, notch filtering, ...
amplifier

24. Overview : Lecture-7 Active Noise Control
Solution based on `filtered-X LMS’
Application : active headsets/ear defenders

25. Overview : Lecture-7bis & 8
3D Audio & Loudspeaker Arrays
Binaural synthesis
…with headphones
head related transfer functions (HRTF)
…with 2+ loudspeakers (`sweet spot’)
crosstalk cancellation

26. Aims/Scope (revisited)
Aim is 2-fold :
Speech & audio per se
Basic signal processing theory/principles :
Optimal filtering / Kalman filters (linear/nonlinear)
here : speech enhancement
other : automatic control, spectral estimation, ...
Advanced adaptive filter algorithms
here : acoustic echo cancellation
other : digital communications, ...
Filtered-X LMS
here : 3D audio
other : active noise/vibration control

27. Lectures Lectures: 7*2hrs (Lectures 1-7) + 1*1hr (Lectures 8)
PS: Time budget = (15hrs)*4 = 60 hrs
Course Material: Slides
Use version !
Download from DASP webpage

28. Prerequisites H197 Signals & Systems (JVDW)
HJ09 Digital Signal Processing (I) (PW)
signal transforms, sampling, multi-rate, DFT, …
HC63 DSP-CIS (MM)
filter design, filter banks, optimal & adaptive filters

29. Literature Literature (General) (available in DSP-CIS library)
Simon Haykin
`Adaptive Filter Theory’ (Prentice Hall 1996)
P.P. Vaidyanathan
`Multirate Systems and Filter Banks’ (Prentice Hall 1993)
Literature (specialized) (some available in DSP-CIS library)
S.L. Gay & J. Benesty
`Acoustic Signal Processing for Telecommunication’ (Kluwer 2000)
M. Kahrs & K. Brandenburg (Eds)
`Applications of Digital Signal Processing to Audio and Acoustics’
(Kluwer1998)
B. Gold & N. Morgan
`Speech and Audio Signal Processing’ (Wiley 2000)

30. Exercise Sessions/Project
Acoustic source localization
Direction-of-arrival estimation
Noise reduction
Echo cancellation
Simulated set-up
Direction-of-arrival θ

31. Acoustic Source Localization Project
Runs over 4 weeks (non-consecutive)
Each week
1 PC/Matlab session (supervised, 2.5hrs)
2 ‘Homework’ sesions (unsupervised, 2*2.5hrs)
PS: Time budget = 4*(2.5hrs+5hrs) = 30 hrs
‘Deliverables’ after week 2 & 4
Grading: based on deliverables, evaluated during sessions
TAs: (English+Italian)
PS: groups of 2

32. Acoustic Source Localization Project
Work Plan
Week 1: Design Matlab simulation set-up
Week 2: Direction-of-arrival (DoA) estimation
*deliverable*
Week 3: DoA estimation + noise reduction
Week 4: DoA estimation + echo cancellation
..be there !

33. Exam Oral exam, with preparation time Open book Grading
7 for question-1
7 for question-2
+6 for project
___
= 20

34. September Retake Exam Oral exam, with preparation time Open book
Grading
7 for question-1
7 for question-2
+6 for question (related to project work)
___
= 20

35. Website TOLEDO http://homes.esat.kuleuven.be/~dspuser/dasp/
Contact:
Slides (use `version ’ !!)
Schedule
DSP-library
FAQs (send questions to

36. Questions? Ask teaching assistant (during exercises sessions)
questions to
teaching assistant or
3) Make appointment
ESAT Room 01.69