2. Spatial Covariance Models For Under- Determined Reverberant Audio Source Separation N. Duong, E. Vincent and R. Gribonval METISS project team, IRISA/INRIA, France Oct. 2009 1

3. Content 2  Under-determined source separation  Spatial covariance models  Model parameter estimation  Experimental evaluation  Conclusion

4. Under-determined source separation 3  Use recorded mixture signals to separate sources, where  Convolutive mixing model: Denote the vector of mixing filters from source to microphone array, the contribution of to all microphones and the vector of mixture signals are computed as :

5. BSS approaches Sparsity assumption: only FEW sources are active at each time-frequency point Binary masking: only ONE source is active at each time-frequency point L1-norm minimization: Short-term Fourier transform 4

6. Beamforming model is denoted as and approximated by the distance between each source to microphones [T. Gustafsson et.al.], i.e. in stereo mixture: Covariance matrix of source images Spatial covariance matrix (rank 1) modeling the mixing process 5 Source variance

7. Spatial covariance models 6  Purpose of the paper: explore the extension of Gaussian framework, i.e. and, that better account for reverberation  We evaluate potential separation performance by estimating the spatial model parameter from training data  Source separation by Wiener filtering  Models for spatial covariance matrix:  Rank-1 convolutive model  Rank-1 anechoic model  Full-rank direct+diffuse model  Full-rank unconstrained model.

8. Rank-1 models 7  Rank-1 anechoic model Where is steering vector specified in the beamforming approach  Rank-1 convolutive model Where is the Fourier transform of the mixing filters

9. Full-rank direct+diffuse model 8 Assuming that the direct part and the reverberant part are uncorrelated and the reverberant part is diffuse where and can be specified from statistical room acoustic, i.e. depends on the microphone distance, wall area, and wall reflection coefficient - In the rectangular room:

10. Full-rank unconstrained model 9 - A more general model than the previous models where the coefficients of are not related a priori - Allows more flexible modeling of the mixing process since the reverberation part is rarely diffuse and is correlated with the direct part in practice - Expected to improve separation performance of real-world convolutive mixtures.

11. Model parameter estimation We investigate the potential separation performance achievable via each model in:  Semi-blind context: Spatial covariance matrices are estimated from true source images but source variances are blindly estimated from the mixture in the ML sense Where is the Kullback-Leibler (KL) divergence between the empirical covariance matrices and the model-based matrices.  Oracle context: Both and are estimated from the true source images.

12. Experiment  Purpose: - Compare the source separation performance of the model-based algorithms - Criteria: SDR, SIR, SAR r s1s1 s2s2 s3s3 m1m1 m2m2 1.8m 1.5m Room dimensions: 4.45 x 3.35 x 2.5 m Source and microphone height: 1.4 m Microphone distance: d = 20 cm or 5 cm Source-to-microphone distance: 120 cm or 50 cm  Experimental setup: - Speech length: 5 seconds - Sampling rate: 16 kHz - Sine window for STFT with length of 1024 taps 11

13. Experimental result ContextCovariance model Number of spatial parameters SDRSIRSAR Semi-blind Rank-1 anechoic6 0.91.74.9 Rank-1 convolutive3078 4.06.46.5 Full-rank direct+diffuse8 3.16.15.6 Full-rank unconstrained6156 5.810.37.9 Binary masking3078 3.310.32.9 L1-norm minimization3078 2.48.13.8 Oracle Rank-1 anechoic6 0.44.47.0 Rank-1 convolutive3078 4.210.25.3 Full-rank direct+diffuse8 10.217.311.4 Full-rank unconstrained6156 10.917.912.1 12

14. Conclusion 13 - Proposed to model the convolutive mixing process by full-rank spatial covariance matrices - Experimental results confirm that full-rank spatial covariance matrices better account for reverberation and potentially improve separation performance compared to rank-1 matrices.  Work in progress - Validated the power of the proposed algorithms over real-world recordings with small source movement (demo session) - Blind context: learning the model parameters from the recorded mixture (submitted to ICASSP 2010 ).  Future work: - Consider separation of diffuse and semi-diffuse sources

15. Thanks for your attention! See you again in the demo session tonight & Your comments…? 14