Speech and Audio Processing and Coding (cont.)

Slides:



Advertisements
Similar presentations
Chapter 19 Fast Fourier Transform (FFT) (Theory and Implementation)
Advertisements

Computer Graphics: 2D Transformations
Nature of light and theories about it
AP STUDY SESSION 2.
1
| Page Angelo Farina UNIPR | All Rights Reserved | Confidential Digital sound processing Convolution Digital Filters FFT.
Copyright © 2003 Pearson Education, Inc. Slide 1 Computer Systems Organization & Architecture Chapters 8-12 John D. Carpinelli.
Processes and Operating Systems
UNITED NATIONS Shipment Details Report – January 2006.
Contents In today’s lecture we’ll have a look at:
Autoregressive Integrated Moving Average (ARIMA) models
Pole Placement.
FET Biasing.
Stationary Time Series
Laplace Transform Math Review with Matlab:
Copyright © Cengage Learning. All rights reserved.
Digital Filter Banks The digital filter bank is set of bandpass filters with either a common input or a summed output An M-band analysis filter bank is.
ON THE REPRESENTATION OF VOICE SOURCE APERIODICITIES IN THE MBE SPEECH CODING MODEL Preeti Rao and Pushkar Patwardhan Department of Electrical Engineering,
PP Test Review Sections 6-1 to 6-6
Notes 18 ECE Microwave Engineering Multistage Transformers
The Fourier Transform I
Outline Minimum Spanning Tree Maximal Flow Algorithm LP formulation 1.
Numerical Analysis 1 EE, NCKU Tien-Hao Chang (Darby Chang)
Copyright © 2013, 2009, 2006 Pearson Education, Inc. 1 Section 5.5 Dividing Polynomials Copyright © 2013, 2009, 2006 Pearson Education, Inc. 1.
Copyright © 2012, Elsevier Inc. All rights Reserved. 1 Chapter 7 Modeling Structure with Blocks.
Mike Doggett Staffordshire University
1 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt Synthetic.
6.4 Best Approximation; Least Squares
Essential Cell Biology
12 System of Linear Equations Case Study
Chapter 8 Estimation Understandable Statistics Ninth Edition
PSSA Preparation.
Copyright © 2013 Pearson Education, Inc. All rights reserved Chapter 11 Simple Linear Regression.
Chapter 13: Digital Control Systems 1 ©2000, John Wiley & Sons, Inc. Nise/Control Systems Engineering, 3/e Chapter 13 Digital Control Systems.
Computer Vision Lecture 7: The Fourier Transform
16. Mean Square Estimation
and M-ary Quadrature Amplitude Modulation (M-QAM)
9. Two Functions of Two Random Variables
1 Decidability continued…. 2 Theorem: For a recursively enumerable language it is undecidable to determine whether is finite Proof: We will reduce the.
Math Review with Matlab:
ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: The Linear Prediction Model The Autocorrelation Method Levinson and Durbin.
Liner Predictive Pitch Synchronization Voiced speech detection, analysis and synthesis Jim Bryan Florida Institute of Technology ECE5525 Final Project.
Page 0 of 34 MBE Vocoder. Page 1 of 34 Outline Introduction to vocoders MBE vocoder –MBE Parameters –Parameter estimation –Analysis and synthesis algorithm.
Speech and Audio Processing and Recognition
Overview of Adaptive Multi-Rate Narrow Band (AMR-NB) Speech Codec
Representing Acoustic Information
EE513 Audio Signals and Systems Digital Signal Processing (Systems) Kevin D. Donohue Electrical and Computer Engineering University of Kentucky.
Topics covered in this chapter
1 CS 551/651: Structure of Spoken Language Lecture 8: Mathematical Descriptions of the Speech Signal John-Paul Hosom Fall 2008.
Speech Coding Using LPC. What is Speech Coding  Speech coding is the procedure of transforming speech signal into more compact form for Transmission.
1 Linear Prediction. 2 Linear Prediction (Introduction) : The object of linear prediction is to estimate the output sequence from a linear combination.
1 PATTERN COMPARISON TECHNIQUES Test Pattern:Reference Pattern:
1 Linear Prediction. Outline Windowing LPC Introduction to Vocoders Excitation modeling  Pitch Detection.
Speech Signal Representations I Seminar Speech Recognition 2002 F.R. Verhage.
Authors: Sriram Ganapathy, Samuel Thomas, and Hynek Hermansky Temporal envelope compensation for robust phoneme recognition using modulation spectrum.
ECE 5525 Osama Saraireh Fall 2005 Dr. Veton Kepuska
VOCODERS. Vocoders Speech Coding Systems Implemented in the transmitter for analysis of the voice signal Complex than waveform coders High economy in.
EEL 6586: AUTOMATIC SPEECH PROCESSING Speech Features Lecture Mark D. Skowronski Computational Neuro-Engineering Lab University of Florida February 27,
More On Linear Predictive Analysis
By Sarita Jondhale 1 Signal preprocessor: “conditions” the speech signal s(n) to new form which is more suitable for the analysis Postprocessor: operate.
EEL 6586: AUTOMATIC SPEECH PROCESSING Speech Features Lecture Mark D. Skowronski Computational Neuro-Engineering Lab University of Florida February 20,
PATTERN COMPARISON TECHNIQUES
Vocoders.
Linear Prediction.
1 Vocoders. 2 The Channel Vocoder (analyzer) : The channel vocoder employs a bank of bandpass filters,  Each having a bandwidth between 100 HZ and 300.
Linear Predictive Coding Methods
Digital Systems: Hardware Organization and Design
Linear Prediction.
Speech Processing Final Project
Presentation transcript:

Speech and Audio Processing and Coding (cont.) Dr Wenwu Wang Centre for Vision Speech and Signal Processing Department of Electronic Engineering w.wang@surrey.ac.uk http://personal.ee.surrey.ac.uk/Personal/W.Wang/teaching.html

Frequency response of H(z) The frequency response of the vocal tract filter can be best obtained by performing the DFT of the impulse response. It can also be directly calculated from the pole-zero plot, due to the relation between DFT and Z-transform. Essentially, the frequency response of the vocal tract can be obtained by sampling its Z-transform along the unit circle: The magnitude response can be obtained as: Exercise: derive the expression for the phase response of H(z).

How to Separate Spectral Envelop and Spectral Details From Spectrum? Spectral envelope (containing information about formant structure) Spectral details

Two Techniques Can Be Used Linear Prediction Cepstrum Analysis 4

Recall “resonance” effect in speech production Vocal tract acts as a variable resonator In simple term, resonance = “formants” Source: (Ellis 2013) 5

Resonance Source: (Ellis 2013) 6

Simulate resonance with a single pair-poles filter 7 Source: (Ellis 2013)

Source filter model Source: excitation signal defining the fine structure of the speech. Filter: subsequent shaping by physical resonances Source: (Ellis 2013) 8

The “resonance” in spectrum Changing slowly ~ 10-20 ms Source: (Ellis 2013) 9

Linear prediction In the time-domain, filtering of the excitation signal through the vocal tract filter described in previous slides can be written equivalently as: The above equation is known as autoregressive moving average (ARMA) process. That is, a sampled value at the output of a linear filter is a weighted average of its past output samples, past input samples and the current input sample. In general, both poles and zeros exist, and it is also referred to as a pole-zero filtering process which can be further split into AR process and MA process written respectively as: AR (all-pole filter) MA (all-zero filter) 10

Linear prediction (cont.) Linear prediction analysis or linear predicative coding (LPC) is a powerful method used in speech processing and coding for modelling the short-term correlations between speech samples. It is a mathematical operation where future values of a discrete-time signal are estimated as a weighted summation of previous samples, i.e. an AR process (all-pole filter), where G is a gain factor: This effectively exploits the redundancy between consecutive samples and encodes and transmits only the non-redundant information in the signal. Although nasals and fricatives can cause anti-resonances in the transfer function, suggesting the usage of a pole-zero filter, a reasonable approximation to the vocal tract filter can be obtained by the above all-pole filter with a sufficiently high order p.

Linear prediction (cont.) Linear prediction provides efficient methods for calculating the coefficients of an all-pole filter from real speech which leads to a compact representation of the formant structure. In speech coding, the LPC coefficients are quantised and encoded and then transmitted to a speech decoder, which reconstructs an approximation to the original speech. The output of a linear predictor with coefficients is defined as: The prediction error between the actual signal and its predicted value is given by: The coefficients are calculated by minimising the short-term mean-squared prediction error:

LPC coefficients estimation Autocorrelation method: The LPC coefficients can be solved using the set of linear equations known as the Yule-Walker equations: where is the short-time autocorrelation function: The above matrix is a Toeplitz matrix, and therefore the equation is usually solved by a recursive algorithm known as Durbin’s algorithm.

LPC coefficients estimation Covariance method: The LPC coefficients can be solved using the following set of linear equations: where measures the similarity over a fixed number of samples between two sequences which are slightly delayed with respect to one-another: The above matrix is not a Toeplitz matrix, and therefore the equation is usually solved by matrix inversion.

Inverse filtering Once the estimates of the linear prediction coefficients have been obtained, the excitation signal can be obtained from the speech signal using inverse filtering where is an inverse filter of the vocal tract transfer function Inverse filtering results in a residual with a flatter spectrum than the original speech. However, the residual still contains useful information about the speech signal, e.g. whether it is voiced/unvoiced, and the periodicity of the speech. For voiced sounds, it has a pulse-like nature, so it is common to apply periodicity estimation to the residual rather than the original speech.

Inverse filtering (cont.) The LPC residual of a voiced segment obtained by inverse filtering

Order selection of the LPC model Roughly speaking, the number of peaks in the magnitude response is equal to the number of poles. As p increases, the error between the AR fit and the speech signal decreases, and the estimated coefficients result in a closer fit to the actual spectral shape, and vice versa. In practice, LPC modelling is usually used to model the formant structure, so the spectral detail obtained from the higher order is unnecessary, and LPC orders of around 20-30 are sufficient to model the first few formants in speech. Generally, LPC spectrum provides a better match to the spectral peaks than the spectral valleys, due to the usage of an all-pole filter to model the spectrum, without any zeros to model the spectral valleys.

Order selection of the LPC model

Estimate formant frequency Estimate from the LPC spectrum Estimate from the Z-transform of the vocal tract transfer function 19

LPC analysis 20ms Source: (Ellis 2013) 20

Prediction gain The prediction error has a smaller variance compared with the original signal. Therefore if we quantise and encode the prediction error instead of the original signal, the error in the quantisation/encoding scheme can be reduced, for a given bit rate. Let d[n] be the quantisation error, the signal-to-quantising noise ratio of the system is: Prediction gain SNR of the quantiser

Speech synthesis from the LPC model After obtaining the LPC coefficients, it is a simple matter to re-synthesize the original speech Generating periodic impulses or white noise. Convolving the LPC coefficients with the generated excitation signals. Source: (Ellis 2013)

Cepstral analysis LPC modelling has been widely used in speech processing due to its simplicity and computational efficiency. The LPC spectral envelope provides a poor fit to the spectral valleys An assumption is made that the excitation is Gaussian noise or a single impulse, while multiple glottal pulses could exist within the window used for estimating the prediction coefficients, especially for female or high pitched speech. A more accurate but less efficient approach to separate the excitation signal from the vocal tract transfer function is cepstral deconvolution. Essentially, this is a process whereby fine spectral detail can be separated from the smooth spectral shape (formant structure).

Definition of cepstrum Complex cesptrum The complex cepstrum is defined as the inverse Fourier transform of the logarithm of the phase unwrapped spectrum. Real cesptrum The real cepstrum is simply obtained by discarding the phase information Both complex and real cepstrum are real valued, as s[n] is real, and S[k] is complex conjugate symmetric. The main drawback with the real cepstrum is that the phase information is discarded and it is not possible to reconstruct the original spectrum, while it is possible to do this from the complex cepstrum by using an inverse DFT.

Cepstrum is a special case of homomorphic filtering Homomorphic system: Homomorphic system can be split into a cascade of three homomorphic systems: Source: (Rabiner & Schafer, 1978)

Cepstrum is a special case of homomorphic filtering Liftering Source: (Rabiner & Schafer, 1978)

Cepstrum Procedure for computing cepstrum 27 Figure source: http://cnx.org/content/m12469/latest/

Cepstrum: an example 28 Source: (Taylor, 2009) = DFT

Phase unwrapping is the phase of the speech spectrum in the k-th frequency bin, and is measured by taking the angle of S[k], which has a range of . Any substitution produces the same complex value of S[k]. To produce a smooth phase spectrum from bin to bin, the phase needs to be unwrapped, i.e. adding integer multiples of to the initial phase estimates. Convolution Theorem DFT

Phase unwrapping Phase unwrapping

Cepstrum Even through the cepstrum involves a transform from time to frequency domains, and then an inverse transformation, it is not accurate to say that it is back to the time domain, due to the nonlinear logarithmic function that is used in the operation. The axis of the cepstrum is therefore given the name quefrency, in the similar spirit as creating “cepstrum” versus “spectrum”. Any regular periodicity in the spectrum would produce a peak in the cepstrum, in the same way that periodicity in the time domain produces a peak in the frequency domain. Harmonics are equally spaced apart in frequency, so they would produce a peak along the quefrency axis at a position (and an integer multiple of this position) which would be proportional to the pitch period. The low end of the quefrency axis corresponds to the smooth spectral shape or formant structure, and the middle and high ends correspond to harmonics and spectral detail. In other words, the contributions of the excitation signal and the vocal tract transfer function occupy different regions of the quefrency axis.

Terminology Terns for Spectrum Derived Terms for Cepstrum Spectrum Frequency Quefrency Harmonics Rahmonics Magnitude Gamnitude Phase Saphe Filter Lifter Low-pass filter Short-pass lifter High-pass filter Long-pass lifter The terminology was invented by Bogert et al. (1960) 32 Figure source: http://cnx.org/content/m12469/latest/

Cepstrum Real cesptrum of a short vowel segment

Cepstrum Real cesptrum of other short vowel segments (only half quefrencies are shown). Horizontal axis is in ms, and vertical axis shows the magnitudes. 34 Figure source: http://cnx.org/content/m12469/latest/

Spectrum vs Cepstrum Waveform of a vowel segment (512 samples) Spectrum of the vowel Cepstrum of the vowel segment The first 20 cepstral coefficients Examples from: http://mi.eng.cam.ac.uk/~ajr/SA95/node33.html

Liftering Liftering in cepstrum is similar to “filtering” in the spectrum. All cepstrum coefficients are set to zero, except the low end coefficients (such as the first 20 coefficients together with their reflected counterparts at negative frequencies). This process is called low-pass liftering. The spectral envelope which contains the formant structural information can be estimated by liftering. The spectral envelope obtained from cepstrum is a good fit in both peaks and valleys, in contrast to LPC, where a better fit was obtained to the peaks than to the valleys. 36

Liftering: demonstration Plot of Source: Rabiner & Schafer, 2007 37

Liftering Spectrum of a vowel segment and its spectral envelop obtained by liftering the first 20 cepstrum coefficients

Liftering Liftered log-magnitude spectrum obtained by using only the first K cepstrum coefficients. It can be seen that with more coefficients, more spectral details can be obtained (similar effect to the use of different orders in LPC modelling to get different spectral details). Source: (Taylor 2009).

An application example of cepstrum: pitch frequency estimation Window (frame) number 40 Source: Rabiner & Schafer, 2007

An application example of cepstrum: pitch frequency estimation 41

An application example of cepstrum liftering: echo/reverberation removal 42

An application example of cepstrum liftering: echo/reverberation removal Figure due to Guido Gebl 43

Mel-frequency cepstral coefficients (MFCCs) MFCC is similar to the computation of the real cepstrum, the difference is that instead of using the DFT, where frequency bins are linearly-spaced apart, the signal is passed through a filter bank in which frequency bands are non-linearly spaced apart, but equi-distant on the mel-scale. The mel-scale is roughly linear below 500Hz, and roughly logarithmic above this, and is based on the perceptual model of pitch sensitivity. The MFCCs features are frequently used in speech recognition, and an advantage of MFCCs is that the amplitude spectrum can be summarised by a relatively small number (commonly around 13) of perceptually-relevant features.

Filter bank for computing MFCCs Speech signal is analysed using STFT. The DFT values at various frequency bands are grouped together in each critical band. They are then weighted according to the weighting function defined by the triangular shape as shown in the next slide. 45

Filter bank for computing MFCCs 46

Filter bank for computing MFCCs (cont.) FFT FFT FFT … … FFT Spectrum … … … … Mel-filters & Cepstrum Analysis … … 47

Comparison of smoothing techniques based on LPC, Cepstrum and Mel cepstrum 48 Source: Rabiner & Schafer, 2007

Summary Source filter model + resonance LPC model LPC analysis & synthesis Cepstrum analysis Liftering MFCCs 49