0. Signal Subspace Speech Enhancement
Page 0 of 43

1. Presentation Outline Introduction Principals
Orthogonal Transforms (KLT Overview)
Papers Review
Page 1 of 47

2. Introduction Two major classes of speech enhancement
By modeling of noise/speech: like HMM
Highly dependent on speech signal syntax and noise characteristics
Based on transformation: Spectral Subtraction
Musical noise
Signal Subspace belongs to the second class (nonparametric)
Page 2 of 47

3. Schematic Diagram Orthogonal Inverse Noisy signal Transform Transform
Modifying
Coefficients
Orthogonal
Transform
Inverse
Transform
Noisy signal
(time domain)
Estimated
Clean
Signal
Page 3 of 47

4. Schematic Diagram Noisy signal (time domain) Framing Orthogonal
Signal+Noise subspace
Estimating
Clean signal
from
Signal+Noise subspace
Framing
overlapping
Orthogonal
Transform
Estimating
Dimensions of
Subspaces Gs
Inverse
Transform
Clean
Signal Gn
Producing two orthogonal subspaces
Noise subspace
Page 4 of 47

5. Principals
Procedure
Estimate the dimension of the signal+noise subspace in each frame
Estimate clean signal from (S+N) subspace by considering some criteria (main part)
energy of the residual noise
energy of the signal distortion
Nulling the coefficients related to the noise subspace
Page 5 of 47

6. Principals Assumptions Noise & speech are uncorrelated
Noise is additive & white (whitened)
Covariance matrix of the noise in each frame is positive definite and close to a Toeplitz matrix
Signal is more statistically structured than noise process
Page 6 of 47

7. Principals Key Factor in Signal Subspace method
Covariance matrices of the clean signal have some zero eigenvalues.
The improvement in SNR is proportional to the number of those zeros.
Nullifying the coefficients of the noise subspace corresponds to that of weak spectral components in spectral subtraction.
Page 7 of 47

8. Orthogonal Transforms
Signal Subspace decomposition can be achieved by applying:
KLT
via Eigenvalue Decomposition (ED) of signal covariance matrix
via Singular Value Decomposition (SVD) of data matrix
SVD approximation by recursive methods
DCT as a good approximation to the KLT
Walsh, Haar, Sine, Fourier,…
Page 8 of 47

9. Orthogonal Transforms: Karhunen-Loeve Transform (KLT)
Also known as “Hotelling”, “Principal Component” or “Eigenvector" Transform
Decorrelates the input vector perfectly
Processing of one component has no effect on the others
Applications
Compression, Pattern Recognition, Classification, Image Restoration, Speech Recognition, Speaker Recognition,…
Page 9 of 47

10. KLT Overview Let R be the correlation matrix of a random
complex sequence
then
Where E is the expectation operator and R is Hermitian matrix.
Page 10 of 47

11. KLT Overview is called the KLT matrix.
Let be unitary matrix which diagonalizes R
are the eigenvalues of R.
is called the KLT matrix.
Page 11 of 47

12. KLT Overview Property of : Consider the following transform:
sequence y is uncorrelated because :
y has no cross-correlation
Page 12 of 47

13. KLT Overview What is ? where and `s are ith column of
Thus are eigenvectors corresponding to
Page 13 of 47

14. KLT Overview
Comments
The arrangement of y auto-correlations is the same as that of
KLT can be based on Covariance matrix
Using largest eigenvalues to reconstruct sequence with negligible error
KLT is optimal
Page 14 of 47

15. KLT Overview  Difficulties
Computational Complexity (no fast algorithm)
Dependency on the statistics of the current frame
Make uncorrelated not independent
Utilize KLT as a Benchmark in evaluating the performance of the other transforms.
Page 15 of 47

16. Papers Review A Signal Subspace Approach for S.E. [Ephraim 95]
On S.E. Algorithms based on Signal Subspace Methods [Hansen]
Extension of the Signal Subspace S.E. Approach to Colored Noise [Ephraim]
An Adaptive KLT Approach for S.E. [Gazor]
Incorporating the Human Hearing Properties in Signal Subspace Approach for S.E. [Jabloun]
An Energy-Constrained Signal Subspace Method for S.E. [Huang]
S.E. Based on the Subspace Method [Asano]
Page 16 of 47

17. A Signal Subspace Approach for S.E. [Ephraim 95]
Principal
Decompose the input vector of the noisy signal into a signal+noise subspace and a noise subspace by applying KLT
Enhancement Procedure
Removing the noise subspace
Estimating the clean signal from S+N subspace
Two linear estimators by considering:
Signal distortion
Residual noise energy
Page 17 of 47

18. A Signal Subspace Approach for S.E. [Ephraim 95]
Notes
Keeping the residual noise below some threshold to avoid producing musical noise
Since DFT & KLT are related, SS is a particular case of this method
if # of basis vectors (for linear combination of a vector) are less than the dim of the vector, then there are some zero eigenvalues for its correlation matrix
Page 18 of 47

19. A Signal Subspace Approach for S.E. [Ephraim 95]
Basics
speech signal : z=y+w , K-dimensional
If M=K, representation is always possible.
Else “damped complex sinusoid model” can be used.
Span( V ): produces all vector y
Are zero mean complex variables
Page 19 of 47

20. A Signal Subspace Approach for S.E. [Ephraim 95]
When M<K, all vectors y lie in a subspace of spanned by the columns of V
 SIGNAL+NOISE SUBSPACE
Covariance matrix of clean signal y
zero eigenvalues
Page 20 of 47

21. A Signal Subspace Approach for S.E. [Ephraim 95]
Covariance matrix of noise w : (K-Dim)
White noise vectors fill the entire Euclidean space RK
Thus the noise exists in both S+N subspace and complementary subspace
NOISE SUBSPACE RK n S n n
Page 21 of 47

22. A Signal Subspace Approach for S.E. [Ephraim 95]
The discussion indicates that Euclidean space of the noisy signal is composed of a signal subspace and a complementary noise subspace
This decomposition can be performed by applying KLT to the noisy signal :
Let
The covariance matrix of z is:
Page 22 of 47

23. A Signal Subspace Approach for S.E. [Ephraim 95]
Noise is additive
Let be the eigendecomposition of Rz
Where are eigenvectors of Rz and
Eigenvalues of Rw are
Page 23 of 47

24. A Signal Subspace Approach for S.E. [Ephraim 95]
Estimating Dimensions of Signal Subspace M
Because ,Hence is the orthogonal projector onto the S+N subspace
Let
: principal eigenvectors
Page 24 of 47

25. A Signal Subspace Approach for S.E. [Ephraim 95]
Thus a vector z of noisy signal can be decomposed as
is the Karhunen-Loeve Transform Matrix.
The vector does not contain signal information and can be nulled when estimating the clean signal.
However, M (dim of S+N subspace) must be calculated precisely
Page 25 of 47

26. A Signal Subspace Approach for S.E. [Ephraim 95]
Linear Estimation of the clean signal
Time Domain Constrained Estimator
Minimize signal distortion while constraining the energy of residual noise in every frame below a given threshold
Spectral Domain Constrained Estimator
Minimize signal distortion while constraining the energy of residual noise in each spectral component below a given threshold
Page 26 of 47

27. A Signal Subspace Approach for S.E. [Ephraim 95]
Time Domain Constrained Estimator
Having z=y+w
Let be a linear estimator of y
where H is a K*K matrix
The residual signal is
Representing signal distortion and residual noise respectively
Page 27 of 47

28. A Signal Subspace Approach for S.E. [Ephraim 95]
Defining Criterion
Solving :
Energy:
Energy:
Minimize signal distortion while constraining the energy of residual noise in the entire frame below a given threshold
Page 28 of 47

29. A Signal Subspace Approach for S.E. [Ephraim 95]
After solving the Constrained minimization by ‘‘Kuhn-Tucker’’ necessary conditions we obtain
Eigendecomposition of HTDC
Where is the Lagrange multiplier that must satisfy
Page 29 of 47

30. A Signal Subspace Approach for S.E. [Ephraim 95]
In order to null noisy components
If then HTDC=I, which means minimum distortion and maximum noise
Page 30 of 47

31. A Signal Subspace Approach for S.E. [Ephraim 95]
Spectral Domain Constrained Estimator
Minimize signal distortion while constraining the energy of residual noise in each spectral component below a given threshold.
Results:
Page 31 of 47

32. A Signal Subspace Approach for S.E. [Ephraim 95]
Notes
The most computational complexity is in Eigendecomposition of the estimated covariance.
Eigendecomposition of Toeplitz covariance matrix of the noisy vector is used as an approximate to KLT
Compromise between large T in estimating Rz ,and large K to satisfy M<K, while KT can not be too large
Page 32 of 47

33. A Signal Subspace Approach for S.E. [Ephraim 95]
Implementation Results
The improvement in SNR is proportional to K /M
The SDC estimator is more powerful than the TDC estimator
SNR improvements in Signal Subspace and SS are similar
Subjective Test
83.9 preferred Signal Subspace over noisy signal
98.2 preferred Signal Subspace over SS
Page 33 of 47

34. On S.E. Algorithms based on Signal Subspace Methods [Hansen]
The dimension of the signal subspace is chosen at a point with almost equal singular values
Gain matrices for different estimators
SDC
TDC MV
Lowest residual noise
LS  G=I
Lowest signal distortion and highest residual noise
K /M improvement in SNR
SDC improves the SNR in the range 0-20 db
Less sensitive to errors in the noise estimation
Musical noise
Page 34 of 47

35. Extension of the Signal Subspace S. E
Extension of the Signal Subspace S.E. Approach to Colored Noise [Ephraim]
Whitening approach is not desirable for SDC estimator.
Obtaining gain matrix H for SDC estimator
is not diagonal when the input noise is colored
Whitening  Orthogonal Transformation U’  modify components by
Page 35 of 47

36. An Adaptive KLT Approach for S.E. [Gazor]
Goal
Enhancement of speech degraded by additive colored noise
Novelty
Adaptive tracking based algorithm for obtaining KLT components
A VAD based on principle eigenvalues
Page 36 of 47

37. An Adaptive KLT Approach for S.E. [Gazor]
Objective
Minimize the distortion when residual noise power is limited to a specific level
Type of colored noise
Have a diagonal covariance matrix in KLT domain
Replaced by
Page 37 of 47

38. An Adaptive KLT Approach for S.E. [Gazor]
Adaptive KLT tracking algorithm
named “projection approximation subspace tracking”
reducing computational time
Eigendecomposition is considered as a constrained optimization problem
Solving the problem considering quasi-stationarity of speech
Then a recursive algorithm is planned to find a close approximation of eigenvectors of the noisy signal
Page 38 of 47

39. An Adaptive KLT Approach for S.E. [Gazor]
Voice activity detector
When the current principle components’ energy is above 1/12 its past minimum and maximum
Implementation Results
SNR (dB)
Non-Processed
Ephraim’s
Noise Type 10
85%
55%
white 5
75%
69%
64%
89%
73%
office
79%
68%
Page 39 of 47

40. Incorporating the Human Hearing Properties in the Signal Subspace Approach for S.E. [Jabloun]
Goal
Keep the residual noise as much as possible, in order to minimize signal distortion
Novelty
Transformation from Frequency to Eigendomain for modeling masking threshold.
Many masking models were introduced in frequency domain; like Bark scale
IFET
Masking
FET
eigendomain
Page 40 of 47

41. Incorporating the Human Hearing Properties in the Signal Subspace Approach for S.E. [Jabloun]
Use noise prewhitening to handle the colored noise
Implementation results
Input SNR
Compared with noisy signal
Compared with Signal Subspace
20 dB
92%
71%
10 dB
85%
78%
5 dB
Page 41 of 47

42. An Energy-Constrained Signal Subspace Method for S.E. [Huang]
Novelty
The colored noise is modelled by an AR process.
Estimating energy of clean signal to adjust the speech enhancement
Prewhitening filter is constructed based on the estimated AR parameters.
Optimal AR coeffs is given by [Key 98]
Page 42 of 47

43. An Energy-Constrained Signal Subspace Method for S.E. [Huang]
Implementation Results
Word Recognition Accuracy for noisy digits
Input SNR
0 dB
5 dB
10 dB
20 dB
Baseline
40 %
70 %
90 %
100 %
ECSS
SNR improvement for isolated noisy digits
Input SNR
0 dB
5 dB
10 dB
20 dB
Improvement
7.6
6.4
5.2
2.9
Page 43 of 47

44. S.E. Based on the Subspace Method [Asano]—Microphone Array
The input spectrum observed at the mth microphone
Vector notation for all microphones
(spatial) correlation matrix for xk is
Then Eigenvalue Decomposition
is applied to
Microphone array
Ambient Noise
Directional Sources
Page 44 of 47

45. S.E. Based on the Subspace Method [Asano]—Microphone Array
Procedure
Weighting the eigenvalues of spatial correlation matrix
Energy of D directional sources is concentrated on D largest eigenvalues
Ambient noise is reduced by weighting eigenvalues of the noise-dominant subspace
discarding M-D smallest eigenvalues when direct-ambient ratio is high
Using MV beamformer to extract directional component from modified spatial correlation matrix
Page 45 of 47

46. S.E. Based on the Subspace Method [Asano]—Microphone Array
Implementation results
Two directional speech signals + Ambient noise
Recognition Rate:
87.2%
81.5%
86.6%
81.1%
10 dB
78%
72.3%
71.5%
66.9%
5 dB B1 A
SNR
MV-NSR MV
Page 46 of 47

47. Thanks For Your Attention
The End
Page 47 of 47