2. PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS

3. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 2 Modern Data Analysis Modern Data Analysis Data points in a high-dimensional space – Images, songs, finance and neuroscience data The data cannot fill up the high-dimensional space “uniformly” – The feature-space dimension is chosen by the user or the acquisition system – Often, the data lies on a low-dimensional structure, conveying its intrinsic degrees of freedom

4. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 3 Principal Component Analysis Principal Component Analysis Given a high-dimensional dataset We look for a global and linear projection, s.t. with and where The solution consists of the principal eigenvectors of (the “empirical correlation matrix”)

5. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 4 Nonlinear Dimensionality Reduction Nonlinear Dimensionality Reduction Swiss roll benchmark example: – A 2D structure lies in 3D space – A linear mapping is not applicable

6. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 5 Manifold Learning Manifold Learning Popular methods – Kernel PCA [Schoelkopf et al., 96’] – ISOMAP [Tenenbaum et al., 00’] – Locally linear embedding (LLE) [Roweis & Saul, 00’] – Laplacian eigenmaps [Belkin & Niyogi, 03’] – Hessian eigenmaps [Donoho & Grimes, 03’] – Diffusion mapsdiffusion geometry – Diffusion maps [Coifman & Lafon, 05’] and diffusion geometry Common outline – Build a graph from the data using a specially-tailored metric – Find a parametric representation of the graph

7. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 6 Graph Construction Graph Construction Step I: Let be a high-dimensional data set of samples Define a pair-wise affinity, e.g. Step II: We view as nodes of an undirected symmetric graph – Two nodes and are connected by an edge with weight Construct a Markovian process on the graph by normalizing the kernel with – represents the probability of transition from to in a single step

8. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 7 Markov Process Markov Process Notation – the kernel matrix corresponding to the affinity function – the corresponding transition matrix where is a diagonal matrix with – the Markovian process defined by the transition matrix Advancing the Markovian process a single step forward on the dataset The probabilistic interpretation of a single step: Running the chain forward yields the expected values of the Markovian process starting at after a single step

9. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 8 Diffusion Interpretation Diffusion Interpretation Consider the probability of transition in a few steps – Raising the transition matrix to the power of – The probability of transition is usually larger in few steps than in a single step (brings the samples “closer”) – Integrates more samples along the transition path For large data set and small kernel scale ( ) the discrete random- walk on the graph converges to a continuous diffusion process on the underlying manifold [Singer, 06’] – Utilize the analysis of diffusion processes

10. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 9 Graph Laplacian and Eigen-decomposition Graph Laplacian and Eigen-decomposition Define the graph-Laplacian [Chang, 97’] An arbitrary embedding satisfies Preserve locality – High affinity – small distance in the embedding – Connection to the eigen-decomposition of the Laplacian – Can be calculated via the transition matrix

11. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 The transition matrix of the random-walk has – A complete sequence of left and right singular vectors – Positive singular eigenvalues: 10 Embedding Embedding Let be the diffusion mappings of into a new Euclidean space

12. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 11 Dimensionality Reduction Dimensionality Reduction dimensionality reduction A fast decay of the eigenvalues enables dimensionality reduction [Coifman & Lafon, 06’]

13. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 12 Toy Example Toy Example Notion of feature extraction Ability to capture the natural parameters of the data [Rabin, 10’]

14. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 13 Toy Example Toy Example Notion of feature extraction Ability to capture the natural parameters of the data Construct a transition matrix from a Gaussian kernel The first non-trivial eigenvector, sorted Spectral decomposition

15. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 14 Toy Example Toy Example Notion of feature extraction Ability to capture the natural parameters of the data Images sorted according to their value in the first eigenvector

16. Diffusion Maps PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 15 Diffusion Distance Diffusion Distance Describes the affinity in terms of graph connectivity Local structures and rules of transition are integrated into a global metric Define a new affinity metric between any two vectors [Coifman & Lafon, 06’]

17. Transient Clustering PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 16 Clustering Using Diffusion Maps Clustering Using Diffusion Maps Graph construction: – Collect the spectral features of the abrupt part estimate into vectors: – View the vectors as graph nodes – Connect nodes and by an edge with weight

18. Transient Clustering PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 17 Clustering Using Diffusion Maps Clustering Using Diffusion Maps Diffusion mapping Diffusion mapping: Let be the diffusion mappings of into a new Euclidean space diffusion distance Define a new Gaussian kernel based on the diffusion distance transients non transients

19. Transient Clustering PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 18 Experimental Results Experimental Results A scatter plot of the 1st, 3rd, and 5th coordinates of the diffusion map of speech contaminated by kitchen sounds A scatter plot of the 1st, 2nd, and 3rd coordinates of the diffusion map of speech contaminated by door knocks The frame content (either consists of transient interference or not) appears in different color (red and blue)

20. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 19 How to incorporate the geometric info? How to incorporate the geometric info? Statistical estimation Statistical estimation Exploits the rate of change of signals Manifold learning Manifold learning Captures the unique temporal and spectral features of transients Nonlocal filtering Nonlocal filtering Exploits transient repetitions

21. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 20 Overview Overview Transient Enhancement Using OM-LSA Abrupt and Decaying Parts Estimation Nonlocal Filtering OM-LSAOM-LSA

22. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 21 Nonlocal Filtering Nonlocal Filtering Notation: – – is a normalized affinity function between the spectra of frames Each step can be interpreted as averaging over similar time frames according to Let denote the # of NL steps

23. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 22 Nonlocal Filtering Nonlocal Filtering The initial estimate of the abrupt part spectrum Nonlocal filtering: – Temporal averaging which exploits subband dependencies – Assume that the kernel separates between transient and non-transient frames A “clean” version of the abrupt spectrum is obtained by: – Spectral subtraction suppresses the additive speech residual – The speech does not typically span the entire spectrum

24. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 23 Statistical Model Statistical Model Assume a simple case: – No divergence between transient events – “Stationary” speech The spectral variance in a single band has a two Gaussian mixture distribution : Creating a “two-wells” potential : – Each well represents the transient presence/absence. Full model described in [Talmon, Cohen & Gannot, TASSP 11’]

25. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 24 Diffusion Interpretation Diffusion Interpretation Using diffusion interpretation of NL filters [Singer et al., 09’] The diffusion operator (Fokker-Planck): The filtering depends on the characteristic of the diffusion process in two- wells potential: [Matkowsky & Schuss, 81’], [Gardiner, 04’] – Relaxation time – Relaxation time in each well – to properly attenuate the residual speech – Mean first passage time (MFPT) – Mean first passage time (MFPT) to exit a well – to approximate wrong identification of transients The # of NL steps:

26. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 25 Relaxation Time Relaxation Time The characteristic relaxation time inside the well centered at is given by the curvature of the bottom of the well The corresponding # of NL steps

27. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 26 Mean First Passage Time Mean First Passage Time The mean first passage time from well to via the barrier

28. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 27 Results Results The wells should be well separated in order to distinguish the presence/absence of transients The proper # of NL steps should satisfy

29. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 28 Trends Trends As the transients become more diverse, the Gaussian distribution is “smeared” As illustrated, the right well becomes wider and shallower, and the barrier is lower

30. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 29 Trends Trends As the speech spectral envelope becomes more diverse, we obtain similar trends:

31. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 30 Results (cont.) Results (cont.) longer relaxation time shorter mean first passage time Both cases result in longer relaxation time and shorter mean first passage time: More difficult to distinct the transients More steps should be employed

32. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 31 Misidentification Misidentification In the simple (two Gaussians) case: – The probability of misidentifying a transient – The probability of falsely identifying a transient Mis-identification occurs in case a sample from one hypothesis exists in the opposite well

33. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 32 High-dimensional Processing High-dimensional Processing Intuitively, by comparing the spectral features of all freq. bins, we exploit the unique structure of the transient In terms of “diffusion”: – The bottoms of the high-dimensional potential wells are more separated – The barrier between the wells becomes higher As a result: – The misidentification probability decreases – # of NL steps is less restricted (the MFTP to exit a well increases)

34. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 33 Experimental Results Experimental Results Setup: – Recorded speech and transient signals sampled at 16 KHz – Speech signals are taken from the TIMIT database, and recorded transient interferences are taken from an online free corpus – The additive stationary noise part is a computer generated white Gaussian noise with SNR of 20 dB – The length of each speech utterance and the corresponding transient interference is 20 sec – Such transient-interference signal typically consists of 25 to 30 events

35. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 34 Experimental Results Experimental Results Speech contaminated by metronome Enhanced speech

36. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 35 Experimental Results Experimental Results Using Euclidean distance: Using diffusion distance: Audio examples:

37. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 36 Extensions Extensions Improved nonlocal operator: Improved nonlocal operator: [Talmon, Cohen & Gannot, ICASSP’10] – Use the operator : Similar eigenvectors and slower decay of spectrum – Enables better and more robust enhancement Simultaneous suppression of transients: Simultaneous suppression of transients: [Talmon, Cohen & Gannot, ICASSP’11] – Utilize the ability of diffusion maps to distinguish between different transient types

38. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 37 Extensions Extensions Supervised processing: – Use training to capture the structure of the transients and acquire an intrinsic basis – Employ nonlinear “graph-based” filtering to enhance the transients by projecting the signal onto the space spanned by the acquired basis

39. Transient Interference Suppression PART III: TRANSIENT INTERFERENCE SUPPRESSION USING DIFFUSION MAPS Cohen, Gannot and Talmon \38 38 Discussion Discussion Combining the classical approach along with a modern data-driven approach Capturing the geometric structure of the signals enriches the a-priori assumed statistical model and enables good performance – Diffusion maps successfully captures the geometry of transients Up next: – Develop approaches to handle more “complicated” signals (e.g. acoustic, speech and music)