1. PART IV: EXTENSIONS AND DISCUSSION

2. Linear System Parameterization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 2 Linear System Parameterization Linear System Parameterization Any musical instrument is controlled by several independent parameters For example: – A flute is controlled by covering holes – A violin is controlled by the length of the strings – The speech system is controlled by the position of the tongue, lips, teeth, etc.

3. Linear System Parameterization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 3 Linear System Parameterization Linear System Parameterization Each system can be seen as a “black box” configured by controlling parameters – We observe the output of the system – The sound depends on the system configuration – Our goal: recover the controlling parameters

4. Linear System Parameterization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 4 Parameter Recovery Parameter Recovery parameters are conveyed by the eigenvectors We show that the parameters are conveyed by the eigenvectors obtained via the diffusion framework [Talmon et al., TSP 12’] Example: [Rabin, 10’]

5. Linear System Parameterization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 5 Temporal Evolution Temporal Evolution Example – violin recordings: – Several samples of the same tone – Each sample is slightly different due to different finger placements on the string Temporal evolution of the controlling parameters – Perceptual system variation (large scale) – Small fluctuations regime (small scale) Formulating the stochastic dynamic as Ito processes [Talmon et al., TSP 12’]

6. Linear System Parameterization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 6 Main Components Main Components Feature Selection Local Metric Improved and Extendable Kernel

7. Linear System Parameterization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 7 Feature Selection Feature Selection covariance function Use the covariance function of the observed signal – Circumvents the dependency on the realization of the input signal – Robust and generic Formulated as a nonlinear function of the parameters [Talmon et al., TSP 12’] – Enables to present the stochastic dynamics using the Ito lemma Example – AR process: – Observe the AR system of order 1: with zero mean white excitation and a controlling parameter (pole) – The covariance of the observable signal nonlinearly depends on the controlling parameter: – Viewed as a function:

8. Linear System Parameterization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 8 Local Metric Local Metric The corresponding Gaussian kernel converges to a diffusion operator No natural extension is available Let be two points in the parametric space and let be their mapping to the feature space The Euclidean distance in the parameter space can be approx. by where and are the local covariance matrices of the features

9. Linear System Parameterization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 9 Local Metric Local Metric

10. Linear System Parameterization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 10 Improved and Extendable Kernel Improved and Extendable Kernel Use the following metric Define affinity The original kernel Extended kernel

11. Linear System Parameterization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 11 Improved and Extendable Kernel Improved and Extendable Kernel

12. Linear System Parameterization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 12 Improved and Extendable Kernel Improved and Extendable Kernel

13. Linear System Parameterization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 13 Improved and Extendable Kernel Improved and Extendable Kernel Let be the SVD of We have In addition: – are the eigenvectors of The eigenvectors represent the independent parameters

14. Linear System Parameterization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 14 Experimental Results Experimental Results Example – AR process – Accurate recovery of the poles of AR processes – Enables to capture the actual degrees of freedom when the poles are dependent – May be viewed as natural parameterization of speech signals

15. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 15 Overview Overview single Consider a reverberant room with a single source and a single microphone – Whether localization can be performed based on the measurement? Prior recordings from various known locations – It becomes a problem of matching – How to incorporate the prior recordings Source

16. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 16 Concept Concept View the acoustic channel as a “black box” controlled by the environmental factors – Room dimensions, reflection coefficients, position of microphone and source Assume the environmental factors are fixed (except the position of the source) – – The prior recordings may be relevant – The source location is the single controlling parameter “Localization” “Localization” – recover the controlling parameter by observing the outputs of the black box

17. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 17 Notation Notation Let denote an acoustic impulse response between the microphone and a source, at relative position – and are the azimuth and elevation direction of arrival (DOA) angles – is the distance between the source and the microphone.

18. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 18 Training Data Training Data We pick predefined positions of the source From each position, we play an arbitrary input signal, and record the signal picked up with the microphone where and are the input and output signals of the room impulse response corresponding to the source location

19. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 19 Training Data Training Data We repeat the training from each source location times – However, the position of the source is slightly perturbed. Let denote the small perturbations of. Let be the input and output signals corresponding to the repeated measurements.

20. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 20 Algorithm Input Algorithm Input The input of the algorithm is a new measurement of an arbitrary unknown input signal from an unknown source position Recover the source position given the measured signal – Capitalize the prior training information

21. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 21 Feature Extraction Feature Extraction The measured signal heavily depends on the unknown input signal – Consequently, the information on the position of the source is weakly disclosed by the “raw” time domain measurements To overcome this challenge, we want to compute features – better convey the position – less dependent on the particular input signal The covariance function of the output signal is given by where and denote the covariance functions of and

22. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 22 Feature Extraction Feature Extraction For each measurement, we compute a feature vector consisting of elements of the covariance Let,, and denote the covariance elements of,, and – Geometrically the vectors are viewed as a “cloud” of points around in Used to estimate the local covariance matrix Training – Input –

23. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 23 Training Stage Training Stage We compute an affinity matrix between the training vectors where is the kernel scaleRecall: Key point: Key point: The proposed kernel enables to capture the actual variability in terms of the source position

24. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 24 Training Stage Training Stage Let and be the eigenvalues and eigenvectors of the kernel. There exist eigenvectors that represent the data in terms of its independent parameters We assume these eigenvectors correspond to the largest eigenvalues – these independent parameters represent the desired position coordinates of the source The spectral representation of the kernel recovers the source position

25. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 25 Test Stage Test Stage Extend the spectral representation Extend the spectral representation to the new measurement – Compute weights – Calculate new spectral coordinate embedding Let be the embedding of the measurements onto the space spanned by the extended vectors corresponding to the source position

26. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 26 Test Stage Test Stage We show that the map recovers the position of the source up to a monotonic distortion – The monotonic character enables the map to organize the measurements according to the values of the source position coordinates Estimate the position: Estimate the position: interpolate the training positions according to distances in the embedded space where consists of the -nearest training measurements in the embedded space, and

27. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 27 Algorithm Summary Algorithm Summary Training Stage: – Obtain training measurements, and calculate the corresponding feature vectors – Given “clouds” of additional measurements, estimate local covariances – Compute the affinity kernel and apply EVD – Detect the relevant eigenvectors Test Stage: – Obtain a new measurement, and calculate the corresponding feature vector – Extend the spectral representation to the new measurement – Construct the embedding – Recover the new position by interpolating the training positions according to the embedding “organization”

28. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 28 Recording Setup Recording Setup acoustic room An acoustic room in Bar-Ilan University of dimensions – Reverberations controlled by double-sided panels (either absorbing or reflecting) – The panels arranged to yield reverberation time of

29. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 29 Recording Setup Recording Setup Inside the room – An omnidirectional microphone in a fixed location – A long “arm” was connected to the base of the microphone, and attached to a turntable that controls the horizontal angle. – A mouth-simulator was located on the far-end of the arm

30. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 30 Recording Setup Recording Setup Recordings – DOA angles of spacing – of a zero-mean and unit-variance white Gaussian noise sampled at was played from the mouth-simulator – The movement of the arm along angles was repeated timesOrganization – Due to small perturbations of the long arm, the exact location is not maintained during the entire period – Each measurement was divided into segments – Segments from the same DOA angle are viewed as a “cloud” of points

31. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 31 Source Localization Source Localization Results – The eigenvector recovers the DOA – Accurate estimation of the DOA

32. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 32 Summary Summary Presented a supervised algorithm for source localization using a diffusion kernel Based solely on single-channel recordings Applied a manifold learning approach to incorporate prior measurements Experimental results conducted in a real reverberant environment showed accurate estimation of the source direction of arrival

33. Supervised Source Localization PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 33 Future Work Future Work Extend the algorithm for noisy environments Investigate the influence of environmental changes following the training stage – When furniture are added or when people are moving around the speaker – How relevant the prior recordings are? Incorporate the prior information in traditional (multichannel) methods – Can we achieve improved results?

34. ConclusionsConclusions PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 34 Conclusions Conclusions Combining geometric information is beneficial – Better performance – Efficient and simple – Insight and analysis The presented methods enable capturing of – Transient interferences – Acoustic parameters AR processes – Artificial signals (AR processes)

35. ConclusionsConclusions PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon\35 35 Future Work Future Work Further improve the methods speech and music – Support more complicated signals (e.g. speech and music) Develop filtering/processing framework – Supervised – Nonlinear – Incorporate temporal information – Handle measurement noise

36. PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon 36 References – Part I References – Part I M. Berouti, R. Schwartz, and J. Makhoul, Enhancement of speech corrupted by acoustic noise, in Proc. of ICASSP-79, pp. 208–211. S. F. Boll, Suppression of acoustic noise in speech using spectral subtraction, IEEE Trans. Acoust., Speech, Signal Process., vol. 27, no. 2, pp. 113–120, Apr. 1979. D. Burshtein and S. Gannot, Speech enhancement using a mixture-maximum model, IEEE Trans. Speech Audio Processing, vol. 10, pp. 341–351, Sept. 2002. I. Cohen, Noise spectrum estimation in adverse environments: Improved minima controlled recursive averaging, IEEE Trans. Speech Audio Process., vol. 11, no. 5, pp. 466–475, Sep. 2003. I. Cohen, Speech spectral modeling and enhancement based on autoregressive conditional heteroscedasticity models,” Signal Process., vol. 68, no. 4, pp. 698–709, Apr. 2006. I. Cohen and B. Berdugo, Speech enhancement for non-stationary environments, Signal Process., vol. 81, pp. 2403–2418, Nov. 2001. I. Cohen and B. Berdugo, Noise estimation by minima controlled recursive averaging for robust speech enhancement, IEEE Signal Processing Lett., vol. 9, pp. 12–15, Jan. 2002. Y. Ephraim and D. Malah, Speech enhancement using a minimum mean-square error short-time spectral amplitude estimator, IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-32, no. 6, pp. 1109–1121, Dec. 1984. Y. Ephraim and D. Malah, Speech enhancement using a minimum mean-square error log-spectral amplitude estimator,” IEEE Trans. Acoust., Speech, Signal Process., vol. 33, no. 2, pp. 443–445, Apr. 1985.

37. PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon 37 References – Part I References – Part I Y. Ephraim and H. L. Van Trees, A signal subspace approach for speech enhancement, IEEE Trans. Speech Audio Process., vol. 3, no. 4, pp. 251–266, Jul. 1995. Y. Hu and P. C. Loizou, A generalized subspace approach for enhancing speech corrupted by colored noise,” IEEE Trans. Speech Audio Process., vol. 11, no. 4, pp. 334–341, Jul. 2003. S. Gannot, D. Burshtein, and E. Weinstein, Iterative and sequential Kalman filter-based speech enhancement algorithms, IEEE Trans. Speech Audio Processing, vol. 6, pp. 373-385, Aug. 1998. J. S. Lim and A.V. Oppenheim, Enhancement and bandwidth compression of noisy speech, Proc. of the IEEE, vol. 67, pp. 1586- 1604, Dec. 1979. R. Martin, Noise power spectral density estimation based on optimal smoothing and minimum statistics, IEEE Trans. Speech Audio Process., vol. 9, no. 5, pp. 504–512, Jul. 2001. [Online] Code available: http://webee.technion.ac.il/people/IsraelCohen/http://webee.technion.ac.il/people/IsraelCohen/

38. PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon 38 References – Part II References – Part II S. Godsill, Digital Audio Restoration - a statistical model based approach. Springer-Verlag London, 1998. E. A. P. Habets, S. Gannot, and I. Cohen, Late reverberant spectral variance estimation based on a statistical model, IEEE Signal Processing Letters, vol. 16, no. 9, pp. 770–773, Sep. 2009. R. Talmon, I. Cohen, and S. Gannot, Speech enhancement in transient noise environment using diffusion filtering, in Proc. ICASSP- 2010. R. Talmon, I. Cohen, and S. Gannot, Transient noise reduction using nonlocal diffusion filters, IEEE Trans. Audio, Speech Lang. Process., vol 19, no.6, pp.1584-1599, Aug. 2011. R. Talmon, I. Cohen, and S. Gannot, Clustering and suppression of transient noise in speech signals using diffusion maps, in Proc. ICASSP-2011. R. Talmon, I. Cohen, and S. Gannot, Single-channel transient interference suppression with Diffusion Maps, submitted, 2011. S. Vaseghi, Advanced Digital Signal Processing and Noise Reduction, 3rd ed. John Wiley & Sons Ltd., 2006.

39. PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon 39 References – Part III References – Part III D. Barash, A fundamental relationship between bilateral filtering, adaptive smoothing, and the nonlinear diffusion equation, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, pp. 844–847, 2002. M. Belkin and P. Niyogi, Laplacian eigenmaps for dimensionality reduction and data representation, Neural Comput., vol. 15, pp. 1373–1396, 2003. A. Buades, B. Coll, and J. M. Morel, A review of image denoising algorithms, with a new one, Multiscale Model. Simul., vol. 4, pp. 490–530, 2005. R. Coifman and S. Lafon, Diffusion maps, Appl. Comput. Harmon. Anal., vol. 21, pp. 5–30, Jul. 2006. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, and S. W. Zucker, Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps, Proc. Nat. Acad. Sci., vol. 102, no. 21, pp. 7426–7431, May 2005. D. L. Donoho and C. Grimes, Hessian eigenmaps: New locally linear embedding techniques for high-dimensional data, Proc. Nat. Acad. Sci., vol. 100, pp. 5591–5596, 2003. S. Lafon, Y. Keller, and R. R. Coifman, Data fusion and multicue data matching by diffusion maps, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 11, pp. 1784–1797, Nov. 2006. B. J. Matkowsky and Z. Schuss, Eigenvalues of the Fokker-Planck operator and the approach to equilibrium for diffusions in potential fields, SIAM Journal of Applied Math., vol. 40, pp. 242–254, 1981.

40. PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon 40 References – Part III References – Part III B. Nadler and M. Galun, Fundamental limitations of spectral clustering, Neural Information Process. Systems (NIPS), vol. 19, 2006. B. Nadler, S. Lafon, R. Coifman, and I. G. Kevrekidis, Diffusion maps, spectral clustering and reaction coordinates of dynamical systems, Appl. Comput. Harmon. Anal., pp. 113–127, 2006. S. T. Roweis and L. K. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science, vol. 260, pp. 2323–2326, 2000. B. Scholkopf, A. Smola, and K. Muller, Nonlinear component analysis as a kernel eigenvalue problem, Neural Comput., vol. 10, pp. 1299–1319, 1996. J. Shi and J. Malik, Normalized cuts and image segmentation, IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888–905, Aug. 2000. A. Singer, Y. Shkolnisky, and B. Nadler, Diffusion interpretation of non local neighborhood filters for signal denoising, SIAM Journal Imaging Sciences, vol. 2, no. 1, pp. 118–139, 2009. R. Talmon, I. Cohen, S. Gannot, and R.R. Coifman, Supervised graph-based processing for sequential transient interference suppression, submitted, 2012. J. B. Tenenbaum, V. de Silva, and J. C. Langford, A global geometric framework for nonlinear dimensionality reduction, Science, vol. 260, pp. 2319–2323, 2000.

41. PART IV: EXTENSIONS AND DISCUSSION Cohen, Gannot and Talmon 41 References – Part IV References – Part IV J. B. Allen and D. A. Berkley, Image method for efficiently simulating small room acoustics, Journal of the Acoustical Society of America, vol. 65, no. 4, p. 943-950, 1979. D. Kushnir, A. Haddad, and R. Coifman, Anisotropic diffusion on sub-manifolds with application to earth structure classification, vol. 32, no. 2, pp. 280-294, 2012. A. Singer, Spectral independent component analysis, Appl. Comput. Harmon. Anal., vol. 21, pp. 128–134, 2006. A. Singer and R. Coifman, Non-linear independent component analysis with diffusion maps, Appl. Comput. Harmon. Anal., vol. 25, pp. 226–239, 2008. A. Singer, R. Erban, I. G. Kevrekidis, and R. Coifman, Detecting intrinsic slow variables in stochastic dynamical systems by anisotropic diffusion maps, Proc. Nat. Acad. Sci., vol. 106, no. 38, pp. 16 090–1605, 2009. R. Talmon, I. Cohen, and S. Gannot, Supervised source localization using diffusion kernels, in Proc. WASPAA-2011. R. Talmon, D. Kushnir, R. R. Coifman, I. Cohen, and S. Gannot, Parametrization of linear systems using diffusion kernels, IEEE Trans. Signal Process., vol. 60, no. 3, pp. 1159-1173, Mar. 2012. [online] E. A. P. Habets, Room impulse response (RIR) generator, Available: http://home.tiscali.nl/ehabets/rir generator.html. [online] Code available: http://users.math.yale.edu/rt294/.