Dept. Elec. Engineering, K.U.Leuven DSP Everywhere… Applications of DSP in Audio and Digital Communications Simon Doclo Dept. Elec. Engineering, K.U.Leuven simon.doclo@esat.kuleuven.ac.be www.esat.kuleuven.ac.be/~doclo/

Introduction DSP in Digital Communications Wireless systems: GSM, WLAN 3G-systems: UMTS, CDMA Wireless systems: GSM, WLAN Modems: Cable, ADSL, VDSL Line echo cancellation Satellite communications Optical communication Simon Doclo DSP Everywhere…

Introduction DSP in Audio Applications Audio and speech coding Audio effects Audio and speech coding Tele-conferencing Voice-controlled systems Hands-free telephony Hearing aids / cochlear implants Simon Doclo DSP Everywhere…

Introduction Other applications Anywhere (digital) signals are present, DSP-techniques are required! Medical applications Cryptography Process control in chemical, pharmaceutical, energy plants Image and video processing … Simon Doclo DSP Everywhere…

Overview Introduction DSP in digital communications systems: xDSL-modems: modulation, equalisation DSP in audio applications: Hands-free communication: echo, noise and reverberation Basic techniques: Acoustic echo cancellation (AEC) Multi-microphone beamforming Application: hearing aids Conclusion Simon Doclo DSP Everywhere…

x 1000 Telephone Line modems High-speed data communication: optical, cable, wireless, telephone line Telephone Line Modems voice-band modems : up to 56kbits/sec in 0…4kHz band ADSL modems : up to 6Mbits/sec in 30kHz…1MHz band VDSL modems : up to 52Mbits/sec in …10MHz band Time to download 10 Mbyte-file: x 1000 Modem Time 56 Kbps voice-band modem 24 minutes 128 Kbps ISDN 10 minutes 6 Mbps ADSL 13 seconds 52 Mbps VDSL 1.5 seconds Simon Doclo DSP Everywhere…

xDSL Modems ADSL : ‘Asymmetric Digital Subscriber Line’ HDSL : ‘High Speed Digital Subscriber Line’ VDSL : ‘Very High Speed Digital Subscriber Line’ …-1993: ADSL spurred by interest in video-on-demand (VOD) 1995 : ADSL/VOD interest decline 1996 : ADSL technology trials prove viability. 1997-... : ADSL deployment, reoriented to data applications, as telco’s reaction to cable operators offering high- speed internet access with cable modems 2000-… : VDSL Simon Doclo DSP Everywhere…

xDSL Modems Analog/digital telephone network: BW  3 kHz, SNR  35 dB  Shannon capacity ADSL/VDSL: higher bandwidth, lower SNR + impairments Bitrate depends on length of copper line Upstream Downstream Subscriber Central office Copper wire 300 m 6.4 Mbps 52 Mbps VDSL 3 km 640 Kbps 6 Mbps ADSL Length Up Down 12 MHz 1.1 MHz Bandwidth vb. Simon Doclo DSP Everywhere…

DMT Principles: IFFT/FFT-based modulation Modulation-technique : DMT (Discrete Multitone) Basic idea: Decompose frequency into ‘tones’ (FFT/IFFT) Assign bits according to SNR per tone ADSL spec (=ANSI standard): 256 tones, 512-point (I)FFTs carrier spacing fo= 4.3215 kHz, basic sampling rate 2.21 MHz (=512*4.3215 kHz) VDSL (=proposal): up to 4096 tones, same carrier spacing frequentie SNR bits/toon Simon Doclo DSP Everywhere…

ADSL Spectrum ADSL spectrum : Simon Doclo DSP Everywhere…

Communication impairments (1) Frequency-dependent channel attenuation introduces inter-symbol interference (ISI)  equalization Coupling between wires in same or adjacent binders introduces crosstalk (XT) Near-end Xtalk (NEXT) Far-end Xtalk (FEXT) Other systems (e.g. HPNA) Radio Frequency Interference (RFI): e.g. AM broadcast, amateur radio Noise: e.g. impulsive noise (=high bursts of short duration) Echo: due to hybrid impedance mismatch  echo cancellation Conclusion: Need advanced modulation, DSP,etc. ! useful signal FEXT NEXT Simon Doclo DSP Everywhere…

Communication impairments (2) ADSL channel attenuation, crosstalk, noise Simon Doclo DSP Everywhere…

Modulation - Demodulation (1) DMT-transmission block scheme: S/P FFT IFFT P/S Discrete equivalent channel 4-QAM 8-QAM Bitstream (freq domain) FEQ Modulation (IFFT) Time domain signal Demodulation (FFT) Equalisation Simon Doclo DSP Everywhere…

Modulation - Demodulation (2) Transmission: modulation is realized by means of 2N-point Inverse Discrete Fourier Transform (IFFT) (example N=4 ) Receiver: demodulation with inverse operation, i.e. FFT real real Simon Doclo DSP Everywhere…

The Magic Prefix Trick (1) Additional feature : before transmission, a ‘prefix’ is added to each time-domain symbol, i.e. the last samples are copied and put up front : Simon Doclo DSP Everywhere…

The Magic Prefix Trick (2) Prefix insertion : in the receiver, the samples corresponding to the prefix are removed (=unused) : IFFT P/S S/P Discrete equivalent channel FFT FEQ Simon Doclo DSP Everywhere…

The Magic Prefix Trick (3) if channel impulse response has length L (= L non-zero taps) and ( is prefix length), then all ‘transient effects’ between symbols are confined to the prefix period : Tx-side Rx-side Channel Tone 3 Tone 2 Tone 1 Tone 0 Tone 3 Tone 2 Tone 1 Tone 0 ch(t) * s(t) r(t) Prefix From IFFT Guardband To FFT Simon Doclo DSP Everywhere…

The Magic Prefix Trick (4) Magic trick fails if , resulting in inter-symbol-interference (ISI) = interference from previous symbol(s) (same carrier) inter-carrier interference (ICI) = interference from other carriers In the receiver, after removing the samples corresponding to the prefix, the i-th tone is observed, multiplied by a factor H(i.fo), i.e. the channel response for frequency f=i.fo ‘Prefix trick’ is based on a linear convolution (filtering by channel impulse response) being turned into a circular convolution, which corresponds to component-wise multiplication in frequency domain  easy equalization ! Simon Doclo DSP Everywhere…

Overview Introduction DSP in digital communications systems: xDSL-modems: modulation, equalisation DSP in audio applications: Hands-free communication: echo, noise and reverberation Basic techniques: Acoustic echo cancellation (AEC) Multi-microphone beamforming Application: hearing aids Conclusion Simon Doclo DSP Everywhere…

Hands-free communication Recorded microphone signals are corrupted by: Far-end echoes  acoustic echo cancellation Acoustic background noise  noise suppression Room reverberation  dereverberation Application: hands-free telephony, hearing aids, voice control Simon Doclo DSP Everywhere…

Signal model: some maths… Multi-microphone signal enhancement algorithms: Extract clean speech/audio signal from microphone recordings Exploit spatial and frequency diversity between speech and noise Microphone signals (m=1…M): Output signal: compute filters g[k] : echo cancellation: noise reduction/dereverberation: gm[k] cancels noise components gm[k] focuses on speech s[k] unknown known (=loud-speaker signal) Simon Doclo DSP Everywhere…

Overview Introduction DSP in digital communications systems: xDSL-modems: modulation, equalisation DSP in audio applications: Hands-free communication: echo, noise and reverberation Basic techniques: Acoustic echo cancellation (AEC) Multi-microphone beamforming Application: hearing aids Conclusion Simon Doclo DSP Everywhere…

Acoustic echo cancellation (AEC) Suppress acoustic and line echo: to guarantee normal conversation conditions : users do not like to hear a delayed and filtered version of their own voice to prevent the closed-loop system from becoming unstable if amplification is too high Simon Doclo DSP Everywhere…

Room Acoustics Propagation of sound waves in an acoustic environment results in signal attenuation spectral distortion The attenuation and distortion can be modeled quite well as a linear filtering operation Non-linear distortion mainly stems from the loudspeakers. Its effect is typically of second order, therefore (often) not taken into account The linear filter h[k] modeling the acoustic path between loudspeaker and microphone is represented by the acoustic impulse response Simon Doclo DSP Everywhere…

Acoustic Impulse Response (1) Different parts: dead time direct path impulse and early reflections, which depend on the geometry of the room an exponentially decaying tail called reverberation, coming from multiple reflections For typical applications the impulse reponse is between 100 and 400 ms long  several 100 to 1000 taps @ 8-16 kHz memory requirement for circular buffers in DSP Because people move around in the recording room, the acoustic impulse response is highly time-varying Simon Doclo DSP Everywhere…

Acoustic Impulse Response (2) ESAT speech laboratory : T60  120 ms Paleis voor Schone Kunsten : T60  1500 ms Original speech signal : Simon Doclo DSP Everywhere…

Acoustic Impulse Response : FIR or IIR ? If the acoustic impulse response is modeled as an FIR filter  many hundreds to several thousands of filter taps are required an IIR filter  filter order can be reduced, but still several hundreds of filter coefficients are required (=‘bad’ model for acoustic impulse response) Remark: IIR-filters good model for classical filters (LP,HP,BP,BS) hence FIR models are typically used in practice as they are guaranteed to be stable as adaptive filtering techniques are called for: FIR adaptive filters are easier than IIR adaptive filters Simon Doclo DSP Everywhere…

AEC based on Adaptive Filtering Goal: Identify acoustic impulse response h[k] and subtract filtered loudspeaker signal from microphone signal Thanks to the adaptivity time-varying acoustics can be tracked AEC is ‘self-learning’ performance superior to performance of conventional techniques Simon Doclo DSP Everywhere…

Adaptive Filtering Algorithms Algorithm: 2 steps Filter loudspeaker signal  error signal indicates how close this signal is to recorded microphone signal Update filter: update depends on error signal filtered signal desired signal error signal Simon Doclo DSP Everywhere…

Normalized Least Mean Square (NLMS) with L is the adaptive filter length,  is the adaptation stepsize,  is a regularization parameter and k is the discrete-time index Data filtering Filter update Circular data buffer Filter coefficients Simon Doclo DSP Everywhere…

Control Algorithm ‘AEC is more than just an adaptive filter’ : adaptive filter is supplemented with control software, which mainly controls the adaptation speed (e.g. no adaptation during double-talk) In practice echo suppression is limited to 30 dB due to time-variance, non-linearities, finite filterlength  postprocessing (e.g. center-clipping) Simon Doclo DSP Everywhere…

Real-time DSP Implementation (1) AEC-implementation on DSP (lab equipment): TMS320C44 @ 50 MHz : data acquisition (ADC/DAC) TMS320C40 @ 50 MHz : acoustic echo cancellation (AEC) AEC ADC/DAC Simon Doclo DSP Everywhere…

Real-time DSP Implementation (2) Adaptive filtering part : several algorithms can be selected NLMS : time-domain algorithm PB-FDAF : frequency-domain algorithm (better performance) Control software double-talk detection non-linear postprocessing algorithm Variable sampling rate Common sampling rates for speech applications: 8 kHz, 16 kHz for audio applications: 22.05 kHz, 44.1 kHz, 48 kHz Echo paths up to 325 ms can be modeled and tracked with the FDAF based on LMS at 8 kHz sampling frequency and 16 ms delay Simon Doclo DSP Everywhere…

Real-time DSP Implementation (3) Execution times for the most important blocks of the DSP code were measured : N=768 FFT-size=128 fs=8000 Hz block 64 samples = 8 ms Simon Doclo DSP Everywhere…

Demo Local speaker Output AEC Output AEC Far-end signal without Double-talk without Detection Far-end signal Near-end signal Simon Doclo DSP Everywhere…

Overview Introduction DSP in digital communications systems: xDSL-modems: modulation, equalisation DSP in audio applications: Hands-free communication: echo, noise and reverberation Basic techniques: Acoustic echo cancellation (AEC) Multi-microphone beamforming Application: hearing aids Conclusion Simon Doclo DSP Everywhere…

Beamforming basics Background/history: antenna array design for RADAR Array elements are combined electronically such that: array can be steered towards specific direction  higher directivity beam shaping is possible Beamforming for hands-free communication : focus beam on speech source(s)  speech enhancement and dereverberation put spatial nulls in direction of noise sources noise reduction Classification: fixed beamforming: data-independent  fixed filters gm[k] e.g. delay-and-sum, weighted-sum, filter-and-sum adaptive beamforming: data-dependent  adaptive filters gm[k] e.g. LCMV-beamformer, Generalized Sidelobe Canceller Simon Doclo DSP Everywhere…

Delay-and-sum beamforming (1) Microphone signals are delayed and summed together  array can be virtually steered to angle  Angular selectivity is obtained, based on constructive ( =) and destructive ( ) interference Uniform delay-and-sum beamforming implies Uniform array  equal inter-microphone distance Uniformly distributed delays Simon Doclo DSP Everywhere…

Delay-and-sum beamforming (2) Spatial directivity pattern H(,) for uniform DS-beamformer H(,) has sinc-like shape and is frequency-dependent M=5 microphones d=3 cm inter-microphone distance =60 steering angle fs=16 kHz sampling frequency Simon Doclo DSP Everywhere…

Delay-and-sum beamforming (3) For an ambiguity, called spatial aliasing, occurs. This is analogous to time-domain aliasing where now the spatial sampling (=d) is too large. M=5, =60, fs=16 kHz, d=8 cm Spatial aliasing Simon Doclo DSP Everywhere…

Filter-and-sum beamformer Better directivity patterns than DS-beamformer are obtained with weighted-sum and filter-and-sum beamformers e.g. Frequency-independent directivity pattern M=8 Logarithmic array L=50 =90 fs=8 kHz Simon Doclo DSP Everywhere…

Adaptive beamforming Adaptive filter-and-sum structure: Minimize noise output power, while maintaining a chosen frequency response in look direction (and/or other linear constraints) LCMV = Linearly Constrained Minimum Variance minimize variance of output z[k] in order to avoid desired signal to be distorted or cancelled out, J linear constraints are added Simon Doclo DSP Everywhere…

Generalized Sidelobe Canceller (1) GSC consists of three parts: Fixed (delay-and-sum) beamformer, in order to achieve spatial alignment of speech source speech reference Blocking matrix, placing spatial nulls in the direction of the speech source noise references Multi-channel adaptive filter with delay Postproc Simon Doclo DSP Everywhere…

Generalized Sidelobe Canceller (2) Blocking matrix Ca : creating maximum M-1 independent noise references by placing spatial nulls in look-direction different possibilities: e.g. Griffiths-Jim, Walsh broadside Problems of GSC: impossible to reduce noise from look-direction reverberation effects cause signal leakage in noise reference adaptive filter is only updated when no speech is present ! Simon Doclo DSP Everywhere…

Overview Introduction DSP in digital communications systems: xDSL-modems: modulation, equalisation DSP in audio applications: Hands-free communication: echo, noise and reverberation Basic techniques: Acoustic echo cancellation (AEC) Multi-microphone beamforming Application: hearing aids Conclusion Simon Doclo DSP Everywhere…

Application: Hearing Aids (1) Hearing problems are very common nowadays Most of the users are dissatisfied with the performance of their hearing aid in noisy environments (cocktail party effect) increase speech intelligibility by reducing background noise Traditional hearing aids: one microphone, analog, limited signal processing amplification of all incoming sound without distinction between different sound sources Enabling technologies: microphone miniaturisation  integrate multiple microphones into one hearing aid micro-electronics: size ASIC < 10 mm2, low power consumption advanced DSP techniques (noise reduction, feedback suppression) Simon Doclo DSP Everywhere…

Application: Hearing Aids (2) Improvement of speech intelligibility by reduction of background noise BTE hearing aid with 2 (or more) closely-spaced microphones GSC in switched mode: Beamfomer : weights can be adapted during speech Noise suppression (ANC) : only adaptation during noise Speech detection : determine when speech is present Simon Doclo DSP Everywhere…

Conclusion DSP-techniques can be found in many every-day products: audio applications: CD, MiniDisc, hands-free telephony communications: GSM, modems, WLAN medical applications: hearing aids, cochlear implants Implementation differences: sampling rate, memory requirements, complexity Basic techniques: filters, filterbanks, FFT/IFFT  frequency filtering adaptive filters  track changing systems multi-sensor systems  spatial filtering Simon Doclo DSP Everywhere…