Full-rank Gaussian modeling of convolutive audio mixtures applied to source separation Ngoc Q. K. Duong, Supervisor: R. Gribonval and E. Vincent METISS project team, INRIA, Center de Rennes - Bretagne Atlantique, France Nov

Table of content 2  Problem introduction and motivation  Considered framework and contributions  Estimation of model parameters  Conclusion and perspective

Under-determined source separation 3  Use recorded mixture signals to separate sources, where  Convolutive mixing model: Denotes the source images, i.e. the contribution of a source to all microphones, and the vector of mixture signals where the vector of mixing filters from source to microphone array

Baseline approaches Sparsity assumption: only FEW sources are active at each time-frequency point Binary masking (DUET): only ONE source is active at each time-frequency point L1-norm minimization: 4 STFT with narrowband approximation These techniques remain limited in the realistic reverberant environments since the narrowband approximation does not hold

Considered framework Models the STFT coefficients of the source images as zero-mean multivariate Gaussian random variables, i.e. Spatial covariance models Rank-1 model (given by the narrowband assumption): Full-rank unconstrained model: The coefficients of are unrelated a priori 5 Most general possible model which allows more flexible modeling the mixing process Scalar source variances encoding spectro-temporal power of sources I x I spatial covariance matrices encoding spatial position and spatial spread of sources

Considered framework 6 Source separation can be achieved in two steps: 1. Model parameters are estimated in the ML sense - Expectation Maximization (EM) algorithm is well-known as an appropriate choice for this ML estimation of the Gaussian mixing model 2. Source separation by multichannel Wiener filtering Raised issues: - Parameter initialization for EM - Permutation alignment (well-known in frequency-domain BSS)

Proposed algorithm 7 Flow of the BSS algorithms ISTFTSTFT Initialization by Hierarchical Clustering Model parameter estimation by EM Permutation alignment Wiener filtering In each step, we adapt the existing methods for the rank-1 model to our proposed full-rank unconstrained model

Parameter initialization [S. Winter et al. EURASIP vol.2007] 8 Principle: perform the hierarchical clustering of the mixture STFT coefficients in each frequency bin after a proper phase and amplitude normalization Adaptations to our algorithms: 1. and are computed from the phase normalized STFT coefficients instead of from both phase and amplitude normalized coefficients 2.We defines the distance between clusters as the average distance between samples instead of the minimum distance between them. Source variance initialization:

EM algorithm 9 EM for rank-1 model [ C. Fevotte and J-F Cardoso, WASPAA2005 ] - Mixing model: must consider noise component Adaptations to the full-rank model - Apply EM directly to the noiseless mixing model, i.e. - Derive alternating parameter update rule (M-step) by maximizing the likelihood of the complete data

Permutation alignment [H. Sawada et al. ICASSP2006 ] 10 Phase of before and after permutation alignment with Principle: permute the source orders base on the estimated source DoAs and the clustered phase-normalized mixing vectors. Adaptation to the full-rank model: Computing the first principal component of by PCA and then applying the algorithm to the “equivalent” mixing vector The order of is permuted identically to that of

Experiment setup r=0.5m s1s1 s2s2 s3s3 m1m1 m2m2 1.8m 1.5m Source and microphone height: 1.4 m Room dimensions: 4.45 x 3.35 x 2.5 m Microphone distance: d = 0.05 m Reverberation time: 50, 130, 250, 500ms Number of stereo mixtures3 Speech length8 s Sampling rate16 kHz STFT window typeSine Window length1024 Number of EM iterations10 Number of clusters K30 Geometry setting Parameter and program settings 11

Experimental result Full-rank model outperforms both the rank-1 model and baseline approaches in a realistic reverberant environments mixture 12

Conclusion & future work 13 Contributions - Proposed to model the convolutive mixing process by full-rank unconstrained spatial covariance matrices - Designed the model parameter estimation algorithms for the full-rank model by adapting the estimation for rank-1 model - We showed that the proposed algorithm using the full-rank unconstrained spatial covariance model outperforms state-of-the-art approaches. Current result (in collaboration with S. Arberet and A. Ozerov) Combined the proposed full-rank unconstrained covariance model with NMF model for source spectra (to appear in ISSPA, May 2010). Future work Consider the full-rank unconstrained model in the context of source localization.

Thanks for your attention! & Your comments…? 14