Suppression of Musical Noise Artifacts in Audio Noise Reduction by Adaptive 2D filtering Alexey Lukin AES Member Moscow State University, Moscow, Russia Jeremy Todd AES Member iZotope Inc., Cambridge, MA Convention paper 7168

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 2/18 Spectral subtraction Reduction of additive stationary noise: Reduction of additive stationary noise: ► Magnitude spectra are subtracted ► Phase spectrum is left intact STFT Noise spectrum estimation Inverse STFT x[t]X[f,t] – W[f,t] S[f,t]s[t] spectral subtraction workflow

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 3/18 Musical noise artifact Variance of noise spectrum leads to spurious bursts of energy after spectral subtraction Variance of noise spectrum leads to spurious bursts of energy after spectral subtraction STFT of white noise, close-up and overall image

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 4/18 Musical noise artifact Variance of noise spectrum leads to spurious bursts of energy after spectral subtraction Variance of noise spectrum leads to spurious bursts of energy after spectral subtraction STFT of white noise before and after a simple spectral subtraction

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 5/18 Musical noise artifact Simple approaches for reduction: Simple approaches for reduction: ► “Oversubtraction” Loss of signal details Loss of signal details ► Mixing in the original noise Only moderate noise reduction amount Only moderate noise reduction amount ► Time smoothing of gates’ gain Smearing of transients, “noise echoes” Smearing of transients, “noise echoes” – – –

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 6/18 Existing approaches Use of time-smoothed energy estimates: Use of time-smoothed energy estimates: ► Introduction of “attack” and “release” time for sub-band gates ► Ephraim-Malah method: combination of “a- priori” (time-smoothed) and “a-posteriori” (instantaneous) energy estimates

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 7/18 Existing approaches Use of 2D filtering of a spectrogram: Use of 2D filtering of a spectrogram: ► [Whipple 1994] Explicit detection and elimination of energy bursts ► [Goh et al 1998] Detection of musical noise by analysis of local variance and use of median filter for its elimination ► [Lin/Gourban 2003] 2D smoothing for signal detection, 1D time smoothing for processing ► [Soon/Koh 2003] 1D DFT applied to rows of a complex spectrogram, coefficient shrinkage

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 8/18 Non-Local Means algorithm Recently proposed NLM algorithm [Buades 2005] Recently proposed NLM algorithm [Buades 2005] ► Image denoising by comparison of patches

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 9/18 Non-Local Means algorithm Non-Local Means for image denoising Non-Local Means for image denoising ► Capable of preserving and enhancing the 2D structure Illustration from Buades et al 2005 Weights are high for q1, q2, but not for q3

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 10/18 Application to audio Use NLM algorithm to smooth SNR[f,t] Use NLM algorithm to smooth SNR[f,t] ► Spectrograms have a prominent 2-dimensional structure: harmonic peaks, frequency-correlated transients ► NLM, unlike 1-dimensional time smoothing methods, is able to account for frequency correlations in a spectrogram

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 11/18 DFT thresholding DFT thresholding applied to spectrogram patches (again, similarly to image denoising): DFT thresholding applied to spectrogram patches (again, similarly to image denoising): ► Sparse representation of harmonics → better signal/noise separation ► Different type of artifacts compared to NLM → a hybrid algorithm will have less artifacts of each type

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 12/18 Results Spectrogram of a noisy signal SNR = 15 dB

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 13/18 Results After simple spectral subtraction SNR = 21.7 dB

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 14/18 Results Result of Ephraim-Malah suppression SNR = 21.0 dB

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 15/18 Results Result of the proposed method SNR = 21.8 dB

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 16/18 Results Features of the proposed method: Features of the proposed method: ► Adjustable amount of suppression of musical noise ► Adaptive 2D smoothing minimizes loss of signal details ► Computationally expensive (but allows effective parallel implementation on multi-core processors) + –

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 17/18 Results Improvement of SNR after different methods Improvement of SNR after different methods Method \ SNR 25 dB 15 dB 5 dB Simple spectral subtraction Ephraim-Malah method The proposed method

A. Lukin, J. Todd “Suppression of Musical Noise Artifacts” 18/18 Your questions Demo web page: