IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 1 ICA 2004, Kyoto, April 4-9, 2004 / Session: SPP02, Paper No (Th.P3.17) Harmonic Plus Noise Model Based Speech Synthesis in Hindi and Pitch Modification By P.K. Lehana P.C. Pandey IIT Bombay, India

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 2 ABSTRACT In harmonic plus noise model (HNM), each segment of speech is modeled as two bands: a lower "harmonic" part represented as amplitudes and phases of the harmonics of a fundamental and an upper noise part using an all-pole filter excited by random white noise, with dynamically varying band boundary. HNM based synthesis can be used for good quality output with relatively small number of parameters and it permits pitch and time scaling without explicit estimation of vocal tract parameters. We have investigated its use for synthesis in Hindi which has aspirated stops and lacks voiced fricatives. It was found that good quality synthesis could be carried out, including those of aspirated stops. The upper band of HNM was needed only for the palatal and alveolar fricatives. Sensitivity of output quality to the errors in glottal closure instants was studied and random perturbations exceeding 4% of the local pitch period resulted in noticeable degradation. Synthesis with pitch scaling showed that the frequency scale of the amplitudes and phases of the harmonics of the original signal needed to be modified by a speaker dependent warping function, obtained by studying the relationship between pitch frequency and formant frequencies for the three cardinal vowels spoken with different pitches.

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 3 OVERVIEW  Introduction  Harmonic plus noise model (HNM)  Methodology  Results  Conclusions

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 4 INTRODUCTION Research Objective Use of harmonic plus noise model (HNM) based pitch synchronous synthesis to study: Speech synthesis with phoneme sets in Hindi (“aspiration” a feature for stops) Effect of perturbations in glottal crossing instants (GCI’s) on speech quality, by using electroglottogram (EGG) for accurate specifications of GCI’s Speaker modification

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 5 HARMONIC PLUS NOISE MODEL (1/3) Harmonic plus Noise Model (HNM) of Speech (Stylianou, 1995; 2001) Speech signal divided into: Harmonic part Noise part Parameters: Max. voiced frequency V/UV & pitch Harm. ampl. & phases Noise parameters

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 6 HARMONIC PLUS NOISE MODEL (2/3) Analysis

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 7 HARMONIC PLUS NOISE MODEL (3/3) Synthesis

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 8 METHODOLOGY (1/3) Synthesis with Hindi Phoneme Sets Material : Recordings of speech and electroglottogram (EGG) for Isolated Vowels Syllables Words Sentences Processing : Analysis/synthesis of recorded material using HNM

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 9 METHODOLOGY (2/3) Effect of Pitch Perturbation on Speech Quality Material : 2-channel recording of vowels for male and female speakers Speech signal EGG from imp. glottograph Processing: – Estimation of pitch periods from speech signal EGG – Analysis of vowels for HNM parameters – Resynthesis, with % perturbation in GCIs – Assessment of quality of resynthesized vowels

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 10 METHODOLOGY (3/3) Spectral Modifications Material : Sustained vowels at different notes by male and female speakers Processing: Study of F0 & formants in cardinal vowels Formant synthesis after interchanging the notes Scaling of HNM parameters by pitch-scaling factor

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 11 RESULTS (1/7) ANALYSIS/SYNTHESIS RESULTS Synthesis with Hindi Phoneme Sets All vowels and VCV natural & intelligible, if synthesized using harmonic part only, except / a∫a / and / asa / which require the noise part also. GCIs obtained from glottal signal (EGG) give better synthesis.

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 12 RESULTS (2/7) Example1: Synthesis of / ata / Recorded Synth. (H) Synth. (H+N)

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 13 RESULTS (3/7) Example2: Synthesis of /a∫a/ Recorded Synth. (H) Synth. (H+N)

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 14 RESULTS (4/7) Effect of Pitch Perturbation, Example: vowel / a / Recorded GCIs from Speech GCIs from EGG Syn. Syn. With  (GCI) < 4%

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 15 RESULTS (5/7) Effect of Pitch Perturbation on Vowel Quality Quality  (GCI) GCI’s from SpeechGCI’s from EGG Acceptable< 4 %< 6 % Noticeable degradation % % Unacceptable> 8 %> 10 %

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 16 RESULTS (6/7) F0 and Formant Relations F1 monotonically increases with F0. Interchanging the notes results in unnatural output -> proper relation between F0 and F’s necessary. Speaker dependent relationship between F0 and formants.

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 17 RESULTS (7/7) Scaling of HNM parameters by speaker dependent scaling factor gives more natural o/p. Quality Scaling factor Natural< 1.5 Degradation Unacceptable> 2.0 Recorded / a / ( male, 117Hz) / a / pitch-scaled by 2.1

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 18 CONCLUSIONS Conclusions  HNM based synthesis provided good quality synthesis in Hindi.  GCI perturbations > 4 % → quality degradation. GCIs from EGG → better output, indicating HNM’s sensitivity to pitch estimation errors.  Modest pitch modification possible with linear frequency scaling of HNM parameters

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 19

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 20 ICA 2004, Kyoto, April 4-9, 2004 / Session: SPP02, Paper No (Th.P3.17) Harmonic Plus Based Speech Hindi and Pitch

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 21 Noise Model Synthesis in Modification

IIT Bombay ICA 2004, Kyoto, Japan, April 4 - 9, 2004   Introdn HNM Methodology Results Conclusions IntrodnHNM MethodologyResults Conclusions 22 By P.K. Lehana P.C. Pandey EE Dept, IIT Bombay, India