Prosody modification in speech signals Project by Edi Fridman & Alex Zalts supervision by Yizhar Lavner

Prosody: the "non-textual" aspects of the speech signal ”Segmental" aspects: timing, duration, rhythm, stress, and metrical structure. The duration of each individual "segment" is under the control of the speaker to varying degrees, and varies with stress and rate. The relative strength of an individual syllable, word, or phrase may be realized in a number of ways, including lengthening (or shortening and cliticization), changes in pitch, and amplitude, and spectral character.

Project goals Prosody modification with TDPSOLA algorithm Prosody modification with HNM model Conversion of male voice to female voice & vice versa

Four steps in prosody modification Time-scale modification Pitch-scale modification Energy envelope modification Modification of distribution of utterancers

TDPSOLA Approach (*) Based on Overlapp-and-Add idea (*) Synchronization with original pitch by: 1) Setting up pitch marks in analysis signal 2) Setting up new pitch marks in synthesis signal according to time-scale and pitch-scale factors (0.6 for pitch 1.3 for time) (*) Building synthesis signal using OLA

Let us define time instants in analysis signal t a (s) as original pitch marks and pitch contour as P(t) The stream of synthesis pitch-marks t s (u) is determined from t a (s) according to desired time-scale modification (t  D(t)) and pitch- scale modification F p (P) by: t s (u+1)- t s (u) =  P`(t) dt t s `(u+1)-t s `(u) t s `(u+1) t s `(u) 1 with t s (u+1) = D(t s `(u+1)) Setting up new pitch marks P`(t) = F p (P(t))

Problem of TDPSOLA: Impossible to change pitch contour because algorithm is based on original pitch marks Problem: too many pitch marks are not counted in, resulting bad sound quality original pitch-marks new pitch-marks

HNM Approach Speech signal is modeled as harmonics of pitch plus noise Harmonics and noise are treated in different ways Synthesis and analysis are performed in pitch synchronous way

Let X(n) be the speech segment. According to HNM model can be found and written as: where the complex constants h k and z k are defined as: h k - complex amplitude of harmonic K f k - frequency of harmonic K T - sampling period W(n) - noise To minimize error

Amplitudes and phases of pitch-harmonics computed with Prony algorithm by minimizing least square error between harmonics and original signal yielding: Harmonic K is set to be K*F0 where F0 is pitch that found by PDA In each voiced speech fragment maximum voiced frequency Fm is calculated and noise part obtained by filtering signal with HP filter with cutoff frequency Fm In unvoiced fragments signal’s specturm is modeled by pth-order all-pole filter H(z). The noise is synthesized by filtering a unit variance gaussian noise through H(z) When pitch scaling is done there is a need to re-compute amplitudes and phases of modified pitch-harmonics. For this purpose a frequency-continuous spectral and phase envelope is necessary.

Comparing between TDPSOLA & HNM

The only target in pitch-scaling was to change F0 preserving other formants There was an attempt to change spectral envelope in order to change male voice to female voice and vice versa New algorithm was proposed