Spotting Multilingual Consonant-Vowel Units of Speech using Neural Network Models Suryakanth V.Gangashetty, C. Chandra Sekhar, and B.Yegnanarayana Speech and Vision Laboratory Department of Computer Science and Engineering Indian Institute of Technology Madras, Chennai – India {svg,chandra,

isbuleTinki mu khyasa mAchAr mu nnAL mu dalameiccarselvijeylali ta IrOjuvAr ta lolu mukhya m sa lu Speech Signal-to-Symbol Transformation Phonetic engine: Capable of speech signal-to-symbol transformation independent of vocabulary and language

Approaches to Speech Signal-to-Symbol Transformation Based on segmentation and labeling –Segmentation of continuous speech signal into regions of subword units –Assignment of labels to the segmented regions using a subword unit classifier Based on spotting subword units in continuous speech –Detection of anchor points in continuous speech –Assignment of labels to the segments around the anchor points using a subword unit classifier

Spotting CV Units in Continuous Speech CV type units have the highest frequency of occurrence in speech in Indian languages Subword units of CCV, CCCV and CVC types also contain CV segments Vowel onset point (VOP) can be used as an anchor point for recognition of CV units Detection of VOPs using distributions of feature vectors of C and V regions Models for classification of CV segments

Significant Events in a CV Unit

VOP Detection using AANN Models AANN models for capturing the distribution of data One AANN for the consonant region of a CV unit Another AANN for the vowel region of a CV unit

System for Detection of VOPs using AANNs

Illustration of Detection of VOPs (a) Waveform, (b) Hypothesised region labels for each frame, (c)Hypothesised VOPs, and (d) Manually marked (actual) VOPs for the Tamil language sentence /kArgil pahudiyilirundu UDuruvalkArarhaL/

Broadcast News Corpus of Indian Languages Description (Number of) Language TamilTeluguHindiMultilingual Bulletins Training bulletins Testing bulletins CV classes considered Training CV segments43,541 41,725 20,2361,05,502 Sentences for testing1,4161, ,094

Performance for Detection of VOPs Matching hypothesis: A hypothesis with a deviation upto 25 msecs from an actual VOP Missing hypothesis: There is no hypothesis with a deviation upto 25 msecs from an actual VOP Spurious hypothesis: –Multiple hypotheses with a deviation upto 25 msecs –A hypothesis with a deviation greater than 25 msecs VOP Hypotheses (in %) Matching Missing Spurious

Classification of CV Segments using SVMs

System for Spotting CV Units The system gives a 5-best performance of about 74.63% for spotting CV units in 300 test sentences containing 3,924 syllable-like units

Illustration of Spotting CV Units VOP locations (Sample numbers) Lattice of 5-best hypothesised CVs Actual syllable ActualHypothesised pA kA vAhashukAr kApAhAnApa gi yE hi yayaigil hApA pa sAsapa hu gumuvupuhu bIviTiNi dI di yi lAlizi tIyi li nirujalaili VOP Missedrun du RujadE rAdu mumu kU vapO vAU Du dadAnAtuDu VOP Missedru va dakai hivAval kA kagachazAkA VOP Missedrar ha kAkaga sahaL

Summary and Conclusions Spotting multilingual CV units in continuous speech AANN models for detecting VOPs SVM classifier for recognition of CV units around the VOPs Need to reduce # missing VOPs Further processing of hypothesised CV lattice

Thank You