2. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay CEP-cum-TEQUIP-KITE Course “Digital Signal Processing”, IIT Bombay, 2–6 November 2015, Course Coordinator: Prof. V. M. Gadre =============================================================== Audio Signals and Dynamic Range Compression P. C. Pandey EE Dept, IIT Bombay pcpandey @ ee.iitb.ac.in www.ee.iitb.ac.in/~pcpandey Day, Date, & Time: Tuesday 3/11/2015, 1115 – 1245 Venue: Maths Dept, A1A2 Classroom, IIT Bombay

3. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 2/27 Overview 1. Introduction 2.Sliding-band Dynamic Range Compression 3.Offline & Real-time Implementations 4.Test Results 5.Summary & Conclusion References U. Zölzer, “Dynamic range control,” in Digital Audio Signal Processing, 2nd ed., Chichester, West Sussex, U.K.: Wiley, 2008, pp. 225-240. N. Tiwari and P. C. Pandey, “A sliding-band dynamic range compression for use in hearing aids,” in Proc. National Conference on Communications 2014, Kanpur, paper no. 1569847357.

4. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 3/27 1. Introduction Dynamic Range Ratio of maximum to minimum signal amplitude, expressed in dB. Dynamic Range of Audio Signals: 40−120 dB. Dynamic Range Control Measuring the input level and adaptively adjusting the signal level in accordance with the requirements of Signal acquisition Processing; Storage; Transmission Reproduction Listening environment; Listener characteristics

5. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 4/27 Some Applications A/D Converter & Recording System: Optimal use of the amplitude range without causing saturation or overload. Sound Reproduction: Lowest level above the ambient noise and highest within the linear range of the speaker. Hearing aids for Persons with Moderate-to-Severe Sensorineural Hearing Loss: To present sounds comfortably within the severely limited dynamic range of the listener by amplifying the low level sounds without making the high level sounds uncomfortably loud.

6. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 5/27 Sensorineural Hearing Loss Causes Loss of the sensory mechanism in the inner ear or the abnormalities in the auditory nerve Problems Reduced dynamic range of hearing: Frequency-dependent elevation of hearing threshold levels (HTL) without corresponding increase in uncomfortable listening level (UCL), with a narrow gap between HTL and UCL (as low as 10 DB) Loudness recruitment: Abnormally rapid growth in loudness with sound level. Different growth function for different frequencies. No easy established tests.

7. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 6/27 Processing Steps in Dynamic Range Control Level estimation (input or output) Gain calculation Gain application Classification of Dynamic Range Controllers On the basis of signal level calculation: single-band or multiband On the basis of gain control method: feedback or feed- forward

8. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 7/27 Processing Gain dependent on the dynamically varying signal level. Parameters: Compression threshold (T H ) Compression ratio (CR) Attack & release time in level calculation Problems Single-Band Dynamic Range Compression Compensation for frequency-dependent loudness growth not feasible. Power mostly contributed by low-frequency components → level of of high-frequency components controlled by low-frequency components → Inaudibility of high frequency components, distortions in temporal envelope

9. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 8/27 Multiband Dynamic Range Compression General Scheme of Processing Spectral components of the input signal divided in multiple bands and the gain for each band calculated on the basis of signal power in that band. Parameters (band specific): compression threshold T H, compression ratio CR, attack & release time for detection.

10. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 9/27 Some Earlier Investigations Lippmann et al. (1980): 16-channel compression. 9% improvement in recognition score over linear amplification. Asano et al.(1991): Multiband dynamic range compression realized as a single time-varying FIR filter & implemented on a 32-bit DSP fixed-point processor. Less spectral distortion due to smoothened frequency response of FIR filter. Stone et al. (1999): Comparison of single and four-channel compression schemes & effect of varying CR, T H, and attack & release times. Intelligibility & quality tests showed no specific preference for schemes. Li et al. (2000): Wavelet-based compression (7 octave sub-band analysis using wavelet filter bank & resynthesis after applying a logarithmic compression on the wavelet coefficients). Increase in intelligibility without introducing noticeable distortions. Magotra et al. (2000): Multiband dynamic range compression using a 16-bit fixed-point processor. Taylor's series approximation used for the compression function to reduce computations in gain calculation.

11. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 10/27 Disadvantages of Multiband Compression Spurious spectral distortions Reduction in spectral contrasts and modulation depth Distortion in spectral shape of spectral peaks (speech formants) lying across the band boundaries Distortion of transitions of spectral peaks across the adjacent bands Time-varying magnitude response without corresponding variation in the phase response leading to quality degradation → Audible distortions, perceptible discontinuities, adverse effect on the perception of certain speech cues.

12. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 11/27 Example of distortion due to multiband dynamic range compression during spectral transition Processed output: multiband compression with 18 auditory critical bands, CR = 30, T a = 6.4 ms, T r = 192 ms Swept sinusoidal input: constant amplitude, 125 –250 Hz linearly swept frequency, 200 ms sweep duration Time (s)

13. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 12/27 Investigation for a Solution Real-time dynamic range compression to compensate for frequency-dependent loudness recruitment associated with sensorineural hearing loss for use in hearing aids with a low- power processor. Low distortions Low computational complexity & memory requirement Low signal delay (algorithmic + computational)

14. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 13/27 Proposed Scheme: Sliding-Band Dynamic Range Compression Proposed for significantly reducing the temporal and spectral distortions associated with the currently used single-band and multiband compressions in hearing aids. Realized with computational complexity acceptable for implementation on a 16-bit fixed-point DSP processor and signal delay acceptable for real-time application. Investigations Using offline & Real-Time Implementations S election of processing parameters Evaluation of the Implementations Informal listening, PESQ measure

15. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 14/27 2. Sliding-Band Dynamic Range Compression Short-time spectral analysis: windowing, zero-padding, DFT calculation Spectral modification: gain calculation, output spectrum calculation Resynthesis: IDFT calculation, windowing, overlap-add Processing Applying a frequency-dependent gain function, with the gain for each spectral sample determined by the short-time power in auditory critical bandwidth centered at it & in accordance with the specified hearing thresholds, compression ratios, and attack and release times.

16. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 15/27 Spectral modification: P mc (k): Power at upper comfortable listening level CR(k): Compression ratio Short-time spectral analysis: windowing (length L, shift S ), zero- padding, N -point DFT Resynthesis: N -point IDFT, overlap-add

17. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 16/27 Gain Calculation Auditory critical bandwidth BW(k) = 25 + 75(1 + 1.4f 2 ) 0.69, freq. sample = k, freq. = f Target gain calculation Power at upper comfortable listening level: P mc (k) Compression ratio: CR(k) Input power: P ic (k), Output power: P oc (k) Target gain: G t (k) = P oc (k) / P ic (k) Compression relation dB scale: [P oc (k) / P mc (k)] dB = [P ic (k) / P mc (k)] dB / CR(k) linear scale: P oc (k) / P mc (k) = [P ic (k) / P mc (k)] 1/ CR(k) Target gain for k th spectral sample [G t (k)] dB = [1 − 1 / CR(k)] [P mc (k) / P ic (k)] dB

18. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 17/27 Gain changed in steps from the previous value towards the target value with settable attack and release times Fast attack: to avoid the output level from exceeding UCL during transients Slow release: to avoid the pumping effect or amplification of breathing Number of steps during attack phase = s a Number of steps during release phase = s r Target gain corresponding to min. input level = G max Target gain corresponding to max. input level = G min Gain ratio for attack phase γ a = (G max / G min ) 1/sa Gain ratio for release phase γ r = (G max / G min ) 1/sr Gain for i th window & k th spectral sample G(i,k) = max[G(i − 1,k) / γ a, G t (i,k)] for G t (i,k) < G(i − 1,k) min[G(i − 1,k) γ r, G t (i,k)] for G t (i,k) > G(i − 1,k) Attack time T a = s a S / f s, Release time T r = s r S / f s [f s = sampling freq., S = window shift]

19. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 18/27 Implementation Related Challenges Audible distortions due to modifications in the short-time mag. spectrum without associated modification in the phase spectrum. High computational complexity: log or series approximation based gain calculation at each spectral sample for use in sliding- band compression. Solutions Analysis-synthesis using least-square error based signal estimation from modified STFT (Griffin & Lim, 1984): Processing artifacts reduced by masking the effect of phase discontinuities in the modified short-time complex spectrum. Look-up table based gain calculation: Two-dimensional look-up table relating the input power with gain as a function of frequency. Permits compression function most suited to compensate for the abnormal loudness growth.

20. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 19/27 3. Offline & Real-Time Implementations Implementation for Offline Processing Implementation using Matlab 7.10 for evaluating the proposed technique and the effect of processing parameters. Processing parameters ◦ f s = 10 kHz ◦ Frame length = 25.6 ms ( L = 256 ) ◦ Overlap = 75% ( S = 64 ) ◦ FFT size N = 512 2D look-up table for frequency-dependent compression based on a linear relation between input-dB and output-dB, with settable CR(k) and P mc (k). ◦ Input range: 20 log intervals (trade-off: small gain increments, look-up table size). ◦ Look-up table with 256×20 entries Attack and release times ◦ s a =1, T a = 6.4 ms : Fast attack to avoid uncomfortable level during transients ◦ s r =30, T r = 192 ms : Slow release to avoid pumping & amplification of breathing

21. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 20/27 Implementation for Real-Time Processing Implementation on a 16-bit fixed-point DSP board to examine suitability of the technique for use in hearing aids. DSP chip: TI/TMS320C5515 ◦16 MB memory space ( 320 KB on-chip RAM with 64 KB dual access data memory) ◦ Three 32 -bit programmable timers ◦4 DMA controllers each with 4 channels ◦ FFT hardware accelerator ( up to 1024 -point FFT) ◦ Max. clock speed: 120 MHz DSP Board: eZdsp ◦ 4 MB on-board NOR flash for user program ◦ Stereo codec TLV320AIC3204: 16/20/24/32-bit ADC & DAC, 8 – 192 kHz sampling Software development: C using TI's 'CCStudio ver. 4. 0

22. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 21/27 Input-output operations: DMA based I/O with cyclic buffers ADC and DAC: one codec (left channel) with 16 -bit quantization Processing parameters (same as for offline processing): f s = 10 kHz, L = 256, S = 64, N = 512 Data representation (input samples, spectral values, processed samples): 16 -bit real & 16 -bit imaginary Implementation details

23. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 22/27 Data transfers & buffering operations ( S = L/4 ) DMA cyclic buffers 5 -block S - sample input buffer 2 -block S - sample output buffer Pointers Current input block Just-filled input block Current output block Write-to output block (incremented cyclically on DMA interrupt) Signal delay: Algorithmic: 1 frame ( 25.6 ms), Computational ≤ frame shift ( 6.4 ms)

24. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 23/27 4. Test Results Tests for Verification & Evaluation Offline processing Verification of the compression technique for speech input with a large level variation and examination of the effect of different set of processing parameters. Assessment of output speech quality (using informal listening) for different input speech materials and time varying levels. Comparison of distortions introduced by different compression techniques during spectral transitions. Real-time processing Comparison of the processed outputs from offline & real-time implementation: informal listening, PESQ measure (0 – 4.5). Signal delay & computational requirement.

25. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 24/27 Example: "you will mark ut please" concatenated with scaling factors for variation in the input level. CR = 2, T a = 6.4 ms, T r = 6.4 & 192 ms. Input waveform Scaling factor Unprocessed waveform Processed T r = 6.4 ms, low P mc Processed T r = 192 ms, low P mc Processed T r = 6.4 ms, high P mc Processed T r = 192 ms, high P mc Time (s) Results from Offline Processing Processing of different speech materials with varying levels: No audible roughness or distortion during informal listening.

26. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 25/27 Time (s) Distortions during spectral transitions: Example of swept sinusoidal input. Sliding band compression output Multiband compression (18 auditory critical bands) output Single-band compression output Input: constant amplitude, 125 –250 Hz linearly swept frequency, 200 ms sweep duration CR = 30, T a = 6.4 ms, T r = 192 ms.

27. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 12345 26/27 Results from Real-Time Processing Informal listening: real-time output perceptually similar to the offline output PESQ for real-time w.r.t. offline : 3.5 Signal delay = 36 ms Use of processing capacity: 41% (lowest acceptable clock: 50 MHz, max = 120 MHz) Unprocessed Offline processed Real-time processed Example: "you will mark ut please" concatenated with scaling factors for variation in the input level. CR = 2, T a = 6.4 ms, T r = 192 ms, low P mc. Time (s)

28. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay 1234 5 27/27 5. Summary & Conclusions Summary: Development & investigation of sliding band compression scheme Realized using modified fixed-frame analysis-synthesis for low computational complexity & without distortions associated with phase discontinuities. Suitable for speech & non-speech audio & provision for settable attack time, release time, & compression ratios. Implemented using 16-bit fixed-point DSP chip & tested for satisfactory operation: 36 ms signal delay, 41% use of processing capacity, indicating scope for combination with other processing techniques. Conclusion: Sliding-band compression can be used to compensate for frequency-dependent loudness recruitment without introducing the distortions associated with single-band & multiband compression.

29. pcpandey@ee.iitb.ac.in EE Dept., IIT Bombay Thank you