1. Linear Prediction

2. Linear Prediction (Introduction):
The object of linear prediction is to estimate the output sequence from a linear combination of input samples, past output samples or both :
The factors a(i) and b(j) are called predictor coefficients.

3. Linear Prediction (Introduction):
Many systems of interest to us are describable by a linear, constant-coefficient difference equation :
If Y(z)/X(z)=H(z), where H(z) is a ratio of polynomials N(z)/D(z), then
Thus the predicator coefficient given us immediate access to the poles and zeros of H(z).

4. Linear Prediction (Types of System Model):
There are two important variants :
All-pole model (in statistics, autoregressive (AR) model ) :
The numerator N(z) is a constant.
All-zero model (in statistics, moving-average (MA) model ) :
The denominator D(z) is equal to unity.
The mixed pole-zero model is called the autoregressive moving-average (ARMA) model.

5. Linear Prediction (Derivation of LP equations):
Given a zero-mean signal y(n), in the AR model :
The error is :
To derive the predicator we use the orthogonality principle, the principle states that the desired coefficients are those which make the error orthogonal to the samples y(n-1), y(n-2),…, y(n-p).

6. Linear Prediction (Derivation of LP equations):
Thus we require that
Or,
Interchanging the operation of averaging and summing, and representing < > by summing over n, we have
The required predicators are found by solving these equations.

7. Linear Prediction (Derivation of LP equations):
The orthogonality principle also states that resulting minimum error is given by
Or,
We can minimize the error over all time :
where

8. Linear Prediction (Applications):
Autocorrelation matching :
We have a signal y(n) with known autocorrelation We model this with the AR system shown below : σ
1-A(z)

9. Linear Prediction (Order of Linear Prediction):
The choice of predictor order depends on the analysis bandwidth. The rule of thumb is :
For a normal vocal tract, there is an average of about one formant per kilohertz of BW.
One formant require two complex conjugate poles.
Hence for every formant we require two predicator coefficients, or two coefficients per kilohertz of bandwidth.

10. Linear Prediction (AR Modeling of Speech Signal):
True Model:
Pitch
Gain
s(n)
Speech
Signal DT
Impulse
generator
G(z)
Glottal
Filter
Voiced
U(n)
Voiced
Volume
velocity
H(z)
Vocal tract
Filter
R(z) LP
Filter V U
Uncorrelated
Noise
generator
Unvoiced
Gain

11. Linear Prediction (AR Modeling of Speech Signal):
Using LP analysis :
Pitch
Gain
estimate DT
Impulse
generator
Voiced
s(n)
Speech
Signal
All-Pole
Filter
(AR) V U
White
Noise
generator
Unvoiced
H(z)

12. 3.3 LINEAR PREDICTIVE CODING MODEL FOR SREECH RECOGNITION

13. Convert this to equality by including an excitation term:
3.3.1 The LPC Model
Convert this to equality by including an excitation term:

14. 3.3.2 LPC Analysis Equations
The prediction error:
Error transfer function:

15. 3.3.2 LPC Analysis Equations
We seek to minimize the mean squared error signal:

16. Terms of short-term covariance:
(*)
Terms of short-term covariance:
With this notation, we can write (*) as:
A set of P equations, P unknowns

17. 3.3.2 LPC Analysis Equations
The minimum mean-squared error can be expressed as:

18. 3.3.3 The Autocorrelation Method
w(m): a window zero outside 0≤m≤N-1
The mean squared error is:
And:

19. 3.3.3 The Autocorrelation Method

20. 3.3.3 The Autocorrelation Method

21. 3.3.3 The Autocorrelation Method

22. 3.3.3 The Autocorrelation Method

23. 3.3.3 The Autocorrelation Method

24. 3.3.4 The Covariance Method

25. 3.3.4 The Covariance Method
The resulting covariance matrix is symmetric, but not Toeplitz,
and can be solved efficiently by a set of techniques called
Cholesky decomposition

26. 3.3.6 Examples of LPC Analysis

27. 3.3.6 Examples of LPC Analysis

28. 3.3.7 LPC Processor for Speech Recognition

29. 3.3.7 LPC Processor for Speech Recognition
Preemphasis: typically a first-order FIR,
To spectrally flatten the signal
Most widely the following filter is used:

30. 3.3.7 LPC Processor for Speech Recognition
Frame Blocking:

31. 3.3.7 LPC Processor for Speech Recognition
Windowing
Hamming Window:
Autocorrelation analysis

32. 3.3.7 LPC Processor for Speech Recognition
LPC Analysis, to find LPC coefficients, reflection coefficients (PARCOR), the log area ratio coefficients, the cepstral coefficients, …
Durbin’s method

33. 3.3.7 LPC Processor for Speech Recognition

34. 3.3.7 LPC Processor for Speech Recognition
LPC parameter conversion to cepstral coefficients

35. 3.3.7 LPC Processor for Speech Recognition
Parameter weighting
Low-order cepstral coefficients are sensitive to overall spectral slope
High-order cepstral coefficients are sensitive to noise
The weighting is done to minimize these sensitivities

36. 3.3.7 LPC Processor for Speech Recognition

37. 3.3.7 LPC Processor for Speech Recognition
Temporal cepstral derivative

38. 3.3.9 Typical LPC Analysis Parameters
N number of samples in the analysis frame
M number of samples shift between frames
P LPC analysis order
Q dimension of LPC derived cepstral vector
K number of frames over which cepstral time derivatives are computed

39. N M p Q K 3 12 10 8 100 (10 msec) 80 (10 msec) 100 (15 msec)
Typical Values of LPC Analysis Parameters for Speech-Recognition System 3 K 12 Q 10 8 p
100 (10 msec)
80 (10 msec)
100 (15 msec) M
300 (30 msec)
240 (30 msec)
300 (45 msec) N